It seems that its only possible to upload files over FTP using:
https://wiki.jenkins-ci.org/display/JENKINS/Publish+Over+FTP+Plugin
But I need to download a file over FTP, I could write a groovy plugin for this but it seems strange that there is no plugin for this
Maybe the reason is that it is somewhat trivial to download a file during a build using a one-liner like:
curl -O ftp://server/path/to/file
The curl tool is available for just about any operating system.
If you only need to download one file over FTP and do not need to check out any other files, you could also use the URL SCM plugin.
There is now an SSH2Easy plugin available that supports sftp to/from remote systems. My company has moved to requiring use of sftp and won't allow ftp any longer.
This plugin, at the time of this reply, still needs some polishing, but it does work.