ftplib - file creation very slow: SSLError: The re

Question

import ftplib
import csv
import StringIO

sio = StringIO.StringIO()
data = csv.writer(sio)
data.writerows(data_to_export)
sio.seek(0)
ftps.storbinary("STOR " + 'blah.csv', sio)
ftps.close()

So the file I'm creating is only 56 lines, it creates very slowly. And when it finishes creating, I get this error:

    ftps.storbinary("STOR " + 'blah.csv', sio)
  File "/usr/lib/python2.7/ftplib.py", line 752, in storbinary
    conn.unwrap()
  File "/usr/lib/python2.7/ssl.py", line 384, in unwrap
    s = self._sslobj.shutdown()
SSLError: The read operation timed out

I don't get it why it is so slow and why it gives time out, even though it creates that file.

P.S. please ask if I need to provide additional information

Update I also tried cString, but it does not improve anything as I guess its not related with writing speed.

If there is any difference, connection to ftp is made using Implicit SSL/TLS mode (that ftp does not support Explicit SSL/TLS).

Update Digging deeper. That is what debugging shows for ftp:

*cmd* 'TYPE I'
*resp* '200 Binary mode selected.'
*cmd* 'PASV'
*resp* '227 Entering Passive Mode (*,*,*,*,*,*)'
*cmd* 'STOR blah.csv'
*resp* '125 Secure data connection open; transfer starting.'

And then stucks at that last output till I get connection time out.

Note. My internet connection is very good, but that ftp is kind of slow. Still I guess such small file should be handled much faster than this.

Update2 That is some weird ftp. Trying unsecure ftp connection, files are uploaded properly and fast. That Implicit connection has something to do with this slow performance and time outs.

Answer 1

In regards to speed there are too many factors to guess at, so i would adjust the code and try the following code which i recently created for customer ftp delivery and can state it works without error.

Note this is from a much larger file but have extracted the relevant sections so ignore some of the variables ie args.var, you may have to adjust the blocksize dependant on bandwidth etc try taking it down to 1024 then working up.

Hope it helps

def initiate_ftp_connection(ftp_host, user, passwd, ftp_dir):
    ftp_session = ftplib.FTP()
    ftp_session.connect(ftp_host, 21)
    #uncomment for debugging.
    #ftp_session.set_debuglevel(2)
    ftp_session.login(user=user, 
                      passwd=passwd)
    #cd to correct remote directory
    ftp_session.cwd(ftp_dir)
    return ftp_session



def upload_deliverables(session, file_and_path):
    working_dir = os.path.dirname(file_and_path)
    #strip to remove any newlines
    filename = os.path.basename(file_and_path).strip()
    totalSize = os.path.getsize(file_and_path)
    #instantiate progress tracker for status updates
    uploadTracker = FtpUploadTracker(int(totalSize),filename)
    #change dir to working_dir
    os.chdir(working_dir)
    '''
     Trigger the ftp upload (storbinary) for the deliverable.
      Args:
       1: FTP KEYWORD and FILE
       2: File IO
       3: Blocksize
       4: Callback
    '''
    session.storbinary('STOR ' + filename, 
                        open(filename,'r'), 
                        8192, 
                        uploadTracker.ftp_callback)


#connect to server
ftp_session = initiate_ftp_connection(args.ftp_host, 
                                      args.ftp_user, 
                                      args.ftp_pass,
                                      args.ftp_dir)

#start ftp delivery
upload_deliverables(ftp_session, args.asset)
#quit the ftp session
ftp_session.quit()   
#close any file handles.
ftp_session.close()