Zip and ftp a string on the fly with Python

Question

I want to zip a string(could be very big) and send it through FTP. So far I am using ftplib and ziplib, but they are not getting along too well.

ftp = FTP(self.host)
ftp.login(user=self.username, passwd=self.password)
ftp.cwd(self.remote_path)

buf = io.BytesIO(str.encode("This string could be huge!!"))

zip = ZipFile.ZipFile(buf, mode='x')
# Either one of the two lines 
ftp.storbinary("STOR " + self.filename, buf) # Works perfectly!
ftp.storbinary("STOR " + self.filename, zip) # Doesnt Work

ftp.quit()

The line that doesn't work throws me the following error.

KeyError: 'There is no item named 8192 in the archive'

I tried to zip the file to bytesio without success.

I need to do this all in memory. I cant write the zip file on the server first and then ftp.

Also, I need to do it through pure FTP, no SFTP nor SSH.

Answer 1

I think you're taking the problem the wrong way round.

ftp.storbinary needs a bytes object, not a ZipFile object. You need to create bytes object with compressed data out of your uncompressed data, and pass that to ftp.storbinary. Plus you have to provide a name for the file in the archive.

this snippet creates such an object from a string (standalone example)

import zipfile,io

output_io = io.BytesIO()

zipfile_ob = zipfile.ZipFile(output_io,"w",zipfile.ZIP_DEFLATED)
zipfile_ob.writestr("your_data.txt",b"big string to be compressed"*20)
zipfile_ob.close()

now adapted to your context:

ftp = FTP(self.host)
ftp.login(user=self.username, passwd=self.password)
ftp.cwd(self.remote_path)

buf = str.encode("This string could be huge!!")
output_io = io.BytesIO()

zipfile_ob = zipfile.ZipFile(output_io,"w",zipfile.ZIP_DEFLATED)
zipfile_ob.writestr("your_data.txt",buf)
zipfile_ob.close()
output_io.seek(0)   # rewind the fake file
ftp.storbinary("STOR " + self.filename, output_io)

ftp.quit()

The seek part is needed else you're passing the output_io file-like object while at the end of the file (you just wrote to it so current position is: end of stream). Using seek(0) rewinds the file-like object so it can be read from the start.

note that for only one file, it may be better to use a Gzip object.