Spring SFTP Intergation: Handling moving files to another directory

php editor Xiaoxin introduces Spring SFTP Integration is a powerful framework that can help developers handle the task of moving files from one directory to another. Whether on a local server or a remote server, this framework makes it easy to move and manage files. It provides a series of APIs and configuration options that allow developers to customize the logic of moving files and handle various exceptions. Whether in enterprise-level applications or personal projects, Spring SFTP Integration is a very practical tool worth exploring.

Question content

I am new to spring intergation. I need to process files from sftp server from user1/upload directory and then move them to user1/processed directory. My code generally works fine, but I have two problems:

1. When I restart the application, the directory user1/processed is deleted along with all the files that existed before. I just want to write more files there without clearing the directory every time.

2. Every time I start the application I receive files (as I see their names and I print to the console) I receive old files which have been moved to processed in the directory. This seems really weird because I don't see these files when I connect to sftp via other tools like winscp. Has the old file list been honored somewhere?

@Value("${cielo.sftp.host}")
    private String sftpHost;
    @Value("${cielo.sftp.port}")
    private int sftpPort;
    @Value("${cielo.sftp.user}")
    private String sftpUser;
    @Value("${cielo.sftp.pass}")
    private String sftpPasword;
    @Value("${cielo.sftp.remotedir}")
    private String sftpRemoteDirectoryDownload;

    @Value("${cielo.sftp.localdir}")
    private String sftpLocalDirectoryDownload;
    @Value("${cielo.sftp.filter}")
    private String sftpRemoteDirectoryDownloadFilter;

    @Bean
    @Order(Ordered.HIGHEST_PRECEDENCE)
    public SessionFactory<LsEntry> sftpSessionFactory() {
        DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
        factory.setHost(sftpHost);
        factory.setPort(sftpPort);
        factory.setUser(sftpUser);
        factory.setPassword(sftpPasword);

        factory.setAllowUnknownKeys(true); //Set to true to allow connections to hosts with unknown (or changed) keys. Its default is 'false'. If false, a pre-populated knownHosts file is required.
        return new CachingSessionFactory<>(factory);
    }


    @Bean
    @Order(Ordered.HIGHEST_PRECEDENCE - 1)
    public SftpInboundFileSynchronizer sftpInboundFileSynchronizer(final SessionFactory<LsEntry> sftpSessionFactory) {
        SftpInboundFileSynchronizer fileSynchronizer =
                new SftpInboundFileSynchronizer(sftpSessionFactory);
        fileSynchronizer.setDeleteRemoteFiles(false);
        fileSynchronizer.setRemoteDirectory(sftpRemoteDirectoryDownload);
        fileSynchronizer
                .setFilter(new SftpSimplePatternFileListFilter(sftpRemoteDirectoryDownloadFilter)); //todo maybe use RegexPatternFileListFilter?
        return fileSynchronizer;
    }

    @Bean
    @Order(Ordered.HIGHEST_PRECEDENCE - 2)
    @InboundChannelAdapter(channel = "fromSftpChannel", poller = @Poller(fixedDelay = "1000"))
    //@InboundChannelAdapter(channel = "fromSftpChannel", poller = @Poller(cron = "${cielo.sftp.poller.cron}"))

    public MessageSource<File> sftpMessageSource(final SftpInboundFileSynchronizer sftpInboundFileSynchronizer) {
        SftpInboundFileSynchronizingMessageSource source =
                new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer);
        source.setLocalDirectory(new File("/tmp/local"));
        source.setAutoCreateLocalDirectory(true);
        source.setLocalFilter(new AcceptOnceFileListFilter<File>());
        return source;
    }

    @Bean
    @ServiceActivator(
            inputChannel = "fromSftpChannel")
    public MessageHandler resultFileHandler() {
        return new MessageHandler() {
            @Override
            public void handleMessage(final Message<?> message) throws MessagingException {
                String payload = String.valueOf(message.getPayload());
                System.err.println(payload);
            }
        };
    }

    private static final SpelExpressionParser PARSER = new SpelExpressionParser();

    @Bean(name="fromSftpChannel")
    public MessageChannel fromSftpChannel() {
        return new PublishSubscribeChannel();
    }

    @Bean
    @ServiceActivator(inputChannel = "fromSftpChannel")
    @Order(Ordered.LOWEST_PRECEDENCE)
    public MessageHandler moveFile() {
        SftpOutboundGateway sftpOutboundGateway = new  SftpOutboundGateway(sftpSessionFactory(), AbstractRemoteFileOutboundGateway.Command.MV.getCommand(), "'/user1/upload/'.concat(" + PARSER.parseExpression("payload.getName()").getExpressionString() + ")");
        sftpOutboundGateway.setRenameExpressionString("'/user1/processed/'.concat(" + PARSER.parseExpression("payload.getName()").getExpressionString() + ")");
        sftpOutboundGateway.setRequiresReply(false);
        sftpOutboundGateway.setOutputChannelName("nullChannel");
        sftpOutboundGateway.setOrder(Ordered.LOWEST_PRECEDENCE);
        sftpOutboundGateway.setAsync(true);
        return sftpOutboundGateway;
    }

thanks for your help!

I looked at the examples in spring integration - sftp to rename or move files in remote server after copying and it helped me a lot

I also checked the official spring intergation sftp documentation

Workaround

I'm not sure how to clear the remote directory on startup. This behavior needs to be debugged on your end. But I can tell you why you are seeing old files. You rename remotely after processing, but local copies of the files are still stored on source.setLocalDirectory(new File("/tmp/local"));. Once you've finished renaming or rebooting, consider cleaning it.

You can also look at SftpStreamingMessageSource to replace your logic: https ://docs.spring.io/spring-integration/reference/sftp/streaming.html

The above is the detailed content of Spring SFTP Intergation: Handling moving files to another directory. For more information, please follow other related articles on the PHP Chinese website!