println it

Software blog about tools, builds and making it all work

Ant FTP task: adding progress indicator and timeout

Ant FTP task provides no option to display how much the download has progressed so far, similarly to wget or curl verbose options. Setting connection timeout isn’t possible either.

But I had to implement it one day and here is what I have found out:

The basic idea is therefore to redefine an FTP task with improved version of FTPClient making use of CopyStreamListener when calling Util.copyStream(). Let’s do it!

1. New FTP task:

The biggest problem in the whole story is the following line of code: ftp = new FTPClient(). I wish there were a simple IoC-like way to “inject” an FTPClient instance here but Ant does not provide too many ways to customize its default configuration, something like RestTemplate.setRequestFactory(). Alas!

I really wish more people would embrace Spring/IoC way of thinking: everything is a Strategy and therefore can be “injected”, extended, modified, and customized.

So my first attempt was to extend Ant’s org.apache.tools.ant.taskdefs.optional.net.FTP class, override execute() method, copy its original code and replace the assignment. As much as I have no taste for copying someone else’s code in order to modify a line or two, sometimes this is the only way to go. But it didn’t work! execute() method accesses lots of private members, not accessible from a sub-class.

The next option was to copy an entire FTP class and then replace the assignment. If anyone can suggest a better way to do the same I’m all ears. So I ended up with:

public class FTP extends Task
{
    ...
    public void execute() throws BuildException {
        ...
        org.apache.commons.net.ftp.FTPClient ftp = null;
        ...
        ftp = ( verbose ? new FTPClient( getProject()) :
                          new org.apache.commons.net.ftp.FTPClient());
        ftp.setDataTimeout( 5 * 60 * 1000 ); // 5 minutes
        ...
    }
}

Line 08 sets a customized version of FTPClient or a default one, line 10 sets up FTPClient timeout.

2. New FTPClient:

public class FTPClient extends org.apache.commons.net.ftp.FTPClient
{
    private final Project project; // org.apache.tools.ant.Project
    
    ...

    @Override
    public boolean retrieveFile(String remote, OutputStream local)
    throws IOException
    {
        ...
        Util.copyStream( input, 
                         local, 
                         getBufferSize(),
                         CopyStreamEvent.UNKNOWN_STREAM_SIZE, 
                         new CopyStreamListener( this.project ),
                         false );
        ...
    }
}

Again, I had to copy the original code and “patch” Util.copyStream() call to use CopyStreamListener implementation. Also, I had to copy and slightly modify an FTPConfigurator class to make it accept my new FTPClient.

3. CopyStreamListener:

public class CopyStreamListener implements org.apache.commons.net.io.CopyStreamListener
{
    private final Project project;
    private       long    mbTransferred = 0;

    ...

    @Override
    public void bytesTransferred ( long totalBytes, int bytes, long streamSize )
    {
        long mb = ( totalBytes / ( 1024 * 1024 ));
        if ( mb > mbTransferred )
        {
            mbTransferred = mb;
            this.project.log( "[" + new Date() + "]: [" + mbTransferred + "] Mb transferred" );
        }
    }
}

4. Plugging in new definition of FTP task with ClfAntBuilder:

public class ClfAntBuilder extends AntBuilder
{
    public ClfAntBuilder ()
    {
        super();
        getProject().addTaskDefinition( "ftp", FTP.class );
    }
}

5. Running it:

new ClfAntBuilder().ftp( action         : 'get',
                         server         : host,
                         userid         : username,
                         ... )
{
    fileset( dir       : '...',
             includes  : '...',
             excludes  : '...' )
}

6. The final result:

Trying to override old definition of task ftp
      [ftp] getting files
      [ftp] transferring SomeFile.xml.zip to C:\Temp\SomeFile.xml.zip
[Tue Aug 17 20:43:07 IDT 2010]: [1] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [2] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [3] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [4] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [5] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [6] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [7] Mb transferred
...

Nice!

, , ,

3 Responses to “Ant FTP task: adding progress indicator and timeout”

  • Rakesh says:

    Exactly what I was looking for and thinking of doing. I was trying to find a way to track the progress of File Uploading, as the current API is not providing it, tried to find other ways, and finally decided to extend FTPClient and provide my own implementation. And your solution is exactly what I’d wanted to do :)

    Thank You for your post.

  • gravura says:

    Hello i try to open your blog in safari and its looks funny, i tink that the problem is from your hosting ,or maybe from me but still you have a nice setup for the ads, i writing in this post because you will see it when you are validating comments, Keep up the good work Andrei from Romania

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>