How to recursively download a folder via FTP on Li

Question

I'm trying to ftp a folder using the command line ftp client, but so far I've only been able to use 'get' to get individual files.

Answer 1

If you can, I strongly suggest you tar and bzip (or gzip, whatever floats your boat) the directory on the remote machine—for a directory of any significant size, the bandwidth savings will probably be worth the time to zip/unzip.

Answer 2

If you want to stick to command line FTP, you should try NcFTP. Then you can use get -R to recursively get a folder. You will also get completion.

Answer 3

Just to complement the answer given by Thibaut Barrère.

I used

wget -r -nH --cut-dirs=5 -nc ftp://user:pass@server//absolute/path/to/directory

Note the double slash after the server name. If I don't put an extra slash the path is relative to the home directory of user.

• -nH avoids the creation of a directory named after the server name
• -nc avoids creating a new file if it already exists on the destination (it is just skipped)
• --cut-dirs=5 allows me to take the content of /absolute/path/to/directory and to put it in the directory where I launch wget. The number 5 is used to filter out the 5 components of the path. The double slash means an extra component.

Answer 4

You should not use ftp. Like telnet it is not using secure protocols, and passwords are transmitted in clear text. This makes it very easy for third parties to capture your username and password.

To copy remote directories remotely, these options are better:

• rsync is the best-suited tool if you can login via ssh, because it copies only the differences, and can easily restart in the middle in case the connection breaks.

• ssh -r is the second-best option to recursively copy directory structures.

See:

• rsync man page

• ssh man page

Answer 5

If you can use scp instead of ftp, the -r option will do this for you. I would check to see whether you can use a more modern file transfer mechanism than FTP.

Answer 6

If lftp is installed on your machine, use mirror dir. And you are done. See the comment by Ciro below if you want to recursively download a directory.