I'm extremely new to Spring, and even more-so to Spring Integration, so apologies if this is a very basic question.
I'm wanting to build a very basic log file processor to learn the ropes. Very similar to this: example
I'm also wanting to use a java config approach, and most of the examples I've been following are all XML driven and I'm having a hard time doing the translation.
Ultimately I'd like to recursively poll a source directory for log files and use a persistence store to keep track of what's been found.
Then, copy those files to be processed to a processing folder, and then kick off a spring batch job to process the contents of the file.
When everything completes the processed file can be deleted from the processing location.
I can't seem to figure out the proper way to wire up (using general java config of SpEL) the flow. Also, I'm still very unsure of what the proper pieces should be.
Again something along these basic, high-level lines for the file moving:
file:inbound-channel-adapter -> channel -> file:outbound-adapter
basic sample
Here is what I have so far
EDIT
I've updated with Artem's solution. My source files are now properly copied to the destination location. Thanks Artem!
Ultimately I am still facing the same problem. The files to be scanned are found immediately (and the metadata-store.properties files is populated immediately) but the files are slowly copied to the destination folder. If a crash happens, any source files that have not been copied to the destination folder will essentially be "lost". Perhaps I need to look at other forms of persistence stores, like a custom jdbcfilter.
@Value("${logProcessor.filenamePattern}")
private String filenamePattern;
@Value("${logProcessor.sourceDirectory}")
private String sourceDirectory;
@Value("${logProcessor.processingDirectory}")
private String processingDirectory;
@Bean
@InboundChannelAdapter(channel = "sourceFileChannel", poller = @Poller(fixedRate = "5000"))
public MessageSource<File> sourceFiles() {
CompositeFileListFilter<File> filters = new CompositeFileListFilter<>();
filters.addFilter(new SimplePatternFileListFilter(filenamePattern));
filters.addFilter(persistentFilter());
FileReadingMessageSource source = new FileReadingMessageSource();
source.setAutoCreateDirectory(true);
source.setDirectory(new File(sourceDirectory));
source.setFilter(filters);
source.setUseWatchService(true);
return source;
}
@Bean
@InboundChannelAdapter(channel = "processingFileChannel", poller = @Poller(fixedRate = "5000"))
public MessageSource<File> processingFiles() {
CompositeFileListFilter<File> filters = new CompositeFileListFilter<>();
filters.addFilter(new SimplePatternFileListFilter(filenamePattern));
FileReadingMessageSource source = new FileReadingMessageSource();
source.setAutoCreateDirectory(true);
source.setDirectory(new File(processingDirectory));
source.setFilter(filters);
return source;
}
@Bean
@ServiceActivator(inputChannel = "sourceFileChannel")
public MessageHandler fileOutboundChannelAdapter() {
FileWritingMessageHandler adapter = new FileWritingMessageHandler(new File(processingDirectory));
adapter.setDeleteSourceFiles(false);
adapter.setAutoCreateDirectory(true);
adapter.setExpectReply(false);
return adapter;
}
@Bean
public MessageChannel sourceFileChannel() {
return new DirectChannel();
}
@Bean
public MessageChannel processingFileChannel() {
return new DirectChannel();
}
@Bean
public DefaultDirectoryScanner defaultDirectoryScanner() {
return new DefaultDirectoryScanner();
}
@Bean
public FileSystemPersistentAcceptOnceFileListFilter persistentFilter() {
FileSystemPersistentAcceptOnceFileListFilter fileSystemPersistentAcceptOnceFileListFilter = new FileSystemPersistentAcceptOnceFileListFilter(metadataStore(), "");
fileSystemPersistentAcceptOnceFileListFilter.setFlushOnUpdate(true);
return fileSystemPersistentAcceptOnceFileListFilter;
}
@Bean
public PropertiesPersistingMetadataStore metadataStore(){
PropertiesPersistingMetadataStore metadataStore = new PropertiesPersistingMetadataStore();
metadataStore.setBaseDirectory("C:\\root\\code\\logProcessor");
return metadataStore;
}