Uploading ZIP file to S3, use EC2 to Unzip

问题:

I have a Codeigniter web app that is uploading many tiny files every hour to Amazon S3, which is causing my S3 request charges to shoot up real fast. One way to overcome this will be to zip up the file, upload the zip file to S3, then unzip it when it is on S3.

Can this be done using EC2? Or is there a better method to achieve this? Thank you!!

EDIT: If I were to use EC2, do I use PHP to trigger the creation of a EC2 instance, upload the PHP file required to unzip the zipped files, copy the uncompressed files to S3, then destroy the EC2 instance?

回答1:

If you have an EC2 machine in the same region I would suggest you upload it there zipped and then it drop it to s3 from there unzipped. S3 cannot unzip it on its own as its all static.

Theres no charges between ec2 and s3 so ec2 can handle the unzipping and then write it out into your s3 bucket without additional transfer charges.

回答2:

You can write code in a lambda to unzip a file of S3 bucket, you just have to use it, AWS Lambda will do this for you.

Referece:

1. https://github.com/carloscarcamo/aws-lambda-unzip-py/blob/master/unzip.py
2. https://github.com/mehmetboraezer/aws-lambda-unzip

回答3:

S3 is just storage. Whatever file you upload is the file that is stored. You cannot upload a zip file then extract it once its in S3. If you wrote the application the best thing I could say is to try to re-design how you store the files. S3 requests are pretty cheap... you must be making a lot of requests.

回答4:

I have been using this service to unzip files full of thousands of tiny image files, each zip I upload is about 4GB, and costs around $1 to unzip using http://www.cloudzipinc.com/service/s3_unzip, maybe that might help someone.

Having said that, you might find it easier to use Python with the Boto library. That will work far more efficiently than PHP.