Batch upload of Blobs to Google App Engine

Question

I'm currently migrating a site to Google App Engine. Previously, all the images and static resources (100s of MBs of the stuff) was just part of the deployment.

Now, it's clear, I should use Blob Storage for this in GAE (otherwise updating the app will be mega-painful!).

I need to batch upload these images from my disk to GAE and suspect I need some kind of script to do this. Any ideas or suggestions of the best approach?

Answer 1

The current bulkloader doesn't support this, but it's possible to write one yourself. The process goes something like this:

1. Write a handler that handles uploaded blobs, described here. Have it write the key of the newly created blob to the datastore, or return it via a redirect to the uploader, or whatever is most suitable for your app.
2. Write an upload script. You can use remote_api to call the blobstore create_upload_url function, then use urllib to upload the file. Be sure to encode the file in a multipart form for the blobstore to recognize it correctly.

As a side note, uploading an app uploads only the modified files, so you can store large amounts of static content, and they'll only be uploaded if they've changed. This probably doesn't help you here, though, as apps are limited to 300mb in size.

Answer 2

It's easier than that. There is a freeware app that lets you drag and drop files into your cloud storage. Check out CloudBerry