Download all files and sub-directories from FTP fo

Question

I searched all over the internet about my question but I found answer for C# only, but none for VB.NET.

I want to download all the files and sub-directories of a directory on my FTP server.

I am currently doing it by downloading a ZIP file from my FTP server and extracting it but this is not a good method.

Thanks!

Answer 1

Translating my answer to C# Download all files and subdirectories through FTP to VB.NET:

The FtpWebRequest does not have any explicit support for recursive file operations (including downloads). You have to implement the recursion yourself:

• List the remote directory
• Iterate the entries, downloading files and recursing into subdirectories (listing them again, etc.)

Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the FtpWebRequest. The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.

Your options are:

• Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name". If that succeeds, it's a file, if that fails, it's a directory.
• You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
• You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)

Sub DownloadFtpDirectory(
        url As String, credentials As NetworkCredential, localPath As String)
    Dim listRequest As FtpWebRequest = WebRequest.Create(url)
    listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails
    listRequest.Credentials = credentials

    Dim lines As List(Of String) = New List(Of String)

    Using listResponse As FtpWebResponse = listRequest.GetResponse(),
          listStream As Stream = listResponse.GetResponseStream(),
          listReader As StreamReader = New StreamReader(listStream)
        While Not listReader.EndOfStream
            lines.Add(listReader.ReadLine())
        End While
    End Using

    For Each line As String In lines
        Dim tokens As String() =
            line.Split(New Char() {" "}, 9, StringSplitOptions.RemoveEmptyEntries)
        Dim name As String = tokens(8)
        Dim permissions As String = tokens(0)

        Dim localFilePath As String = Path.Combine(localPath, name)
        Dim fileUrl As String = url + name

        If permissions(0) = "d" Then
            If Not Directory.Exists(localFilePath) Then
                Directory.CreateDirectory(localFilePath)
            End If
            DownloadFtpDirectory(fileUrl + "/", credentials, localFilePath)
        Else
            Dim downloadRequest As FtpWebRequest = WebRequest.Create(fileUrl)
            downloadRequest.Method = WebRequestMethods.Ftp.DownloadFile
            downloadRequest.Credentials = credentials

            Using downloadResponse As FtpWebResponse = downloadRequest.GetResponse(),
                  sourceStream As Stream = downloadResponse.GetResponseStream(),
                  targetStream As Stream = File.Create(localFilePath)
                Dim buffer As Byte() = New Byte(10240 - 1) {}
                Dim read As Integer
                Do
                    read = sourceStream.Read(buffer, 0, buffer.Length)
                    If read > 0 Then
                        targetStream.Write(buffer, 0, read)
                    End If
                Loop While read > 0
            End Using
        End If
    Next
End Sub

Use the function like:

Dim credentials As NetworkCredential = New NetworkCredential("user", "mypassword")
Dim url As String = "ftp://ftp.example.com/directory/to/download/"
DownloadFtpDirectory(url, credentials, "C:\target\directory")

---

If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.

For example with WinSCP .NET assembly you can download whole directory with a single call to the Session.GetFiles:

' Setup session options
Dim SessionOptions As SessionOptions = New SessionOptions
With SessionOptions
    .Protocol = Protocol.Ftp
    .HostName = "ftp.example.com"
    .UserName = "user"
    .Password = "mypassword"
End With

Using session As Session = New Session()
    ' Connect
    session.Open(SessionOptions)

    ' Download files
    session.GetFiles("/directory/to/download/*", "C:\target\directory\*").Check()
End Using

Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.

The Session.GetFiles method is recursive by default.

(I'm the author of WinSCP)

Answer 2

I KNOW YOU MIGHT BE LAZY TO READ ALL THAT, BUT THIS IS THE ANSWER!!! THE CODE IS AT THE END :D

Hello,

It is very simple to do using the WinSCP library which is available for C# and VB.NET - here is the full answer:

Firsly, install the WinSCP library to your project, see here: https://winscp.net/eng/docs/library#downloading_and_installing_the_assembly

Or if you are lazy to do it, then just download the library and add the DLL file as a reference in your project.

And now let's download the whole directory including its files and its sub-directories to the local storage.

You have 2 options:

• Session.GetFiles to download even if the files & directories are already downloaded.

• Session.SynchronizeDirectories to download the files & directories if they does not exists, and also will download the the modified files which already exists. (BEST and better than the first option).

I used the second option of course because it is the best one ever. However, if you want to use the Session.GetFiles method... you can see the VB.NET example here: https://winscp.net/eng/docs/library_session_getfiles#vbnet

And here is how I used the second option: https://winscp.net/eng/docs/faq_script_modified_files

As you can see, everything is explained great!

Basically use synchronize local instead of get and synchronize remote instead of put.

With WinSCP .NET assembly that means, use Session.SynchronizeDirectories, with argument mode set to SynchronizationMode.Remote or SynchronizationMode.Local instead of Session.GetFiles or Session.PutFiles respectively.

And as I wanted to download the files from the FTP Server to my local PC, then here is what I used (SynchronizeDirectories):

1. So yes! Of course Imports WinSCP first :-)

2. Use this code, because the code on https://winscp.net/eng/docs/library_session_synchronizedirectories#vbnet is for SFTP (port 22) and not FTP (port 21) ... and also it uses SynchronizeMode.Remote which uploads from your PC to the FTP server , so in the code below I replaced .Remote with .Local

   Public Shared Function Main() As Integer
   
       Try 
           ' Setup session options
           Dim sessionOptions As New SessionOptions
           With sessionOptions
               .Protocol = Protocol.Ftp
               .HostName = "example.com"
               .UserName = "user"
               .Password = "mypassword"
           End With
   
           Using session As New Session
               ' Will continuously report progress of synchronization
               AddHandler session.FileTransferred, AddressOf FileTransferred
   
               ' Connect
               session.Open(sessionOptions)
   
               ' Synchronize files
               Dim synchronizationResult As SynchronizationResult
               synchronizationResult = _
                   session.SynchronizeDirectories( _
                       SynchronizationMode.Local, "d:\www", "/home/martin/public_html", False)
   
               ' Throw on any error
               synchronizationResult.Check()
           End Using
   
           Return 0
       Catch e As Exception
           Console.WriteLine("Error: {0}", e)
           Return 1
       End Try
   
   End Function
   
   Private Shared Sub FileTransferred(ByVal sender As Object, ByVal e As TransferEventArgs)
   
       If e.Error Is Nothing Then
           'Console.WriteLine("Upload of {0} succeeded", e.FileName)
       Else
           'Console.WriteLine("Upload of {0} failed: {1}", e.FileName, e.Error)
       End If
   
       If e.Chmod IsNot Nothing Then
           If e.Chmod.Error Is Nothing Then
               'Console.WriteLine("Permisions of {0} set to {1}", e.Chmod.FileName, e.Chmod.FilePermissions)
           Else
               'Console.WriteLine("Setting permissions of {0} failed: {1}", e.Chmod.FileName, e.Chmod.Error)
           End If
       Else
           'Console.WriteLine("Permissions of {0} kept with their defaults", e.Destination)
       End If
   
       If e.Touch IsNot Nothing Then
           If e.Touch.Error Is Nothing Then
               'Console.WriteLine("Timestamp of {0} set to {1}", e.Touch.FileName, e.Touch.LastWriteTime)
           Else
               'Console.WriteLine("Setting timestamp of {0} failed: {1}", e.Touch.FileName, e.Touch.Error)
           End If
       Else
           ' This should never happen during "local to remote" synchronization
           'Console.WriteLine("Timestamp of {0} kept with its default (current time)", e.Destination)
       End If
   
   End Sub
   

Don't forget to replace the credentials and the paths..

One more thing? Good luck on your project! :-)