Programmatically emulating “gsutil mv” on appengin

Question

I would like to implement a mv (copy-in-the-cloud) operation on google cloud storage that is similar to how gsutil does it (http://developers.google.com/storage/docs/gsutil/commands/mv).

I read somewhere earlier that this involves a read and write (download and reupload) of the data, but I cannot find the passages again.

Is this the correct way to move a file in cloud storage, or does one have to go a level down to the boto library to avoid copying the data over the network for renaming the file?

istream = cloudstorage.open(src, mode='r')
ostream = cloudstorage.open(dst, content_type=src_content, mode='w')

while True:
    buf = istream.read(500000)
    if not buf:
        break

    ostream.write(buf)

istream.close()
ostream.close()

Update: I found the rest api that supports copy and compose operations and much more. It seems that there is hope that we do not have to copy data across continents to rename something.

Useful Links I have found sofar ...

• Boto based approach: https://developers.google.com/storage/docs/gspythonlibrary
• GCS Clinet Lib: https://developers.google.com/appengine/docs/python/googlecloudstorageclient/
• GCS Lib: https://code.google.com/p/appengine-gcs-client
• RAW JSON API: https://developers.google.com/storage/docs/json_api

Answer 1

Use the JSON API, there is a copy method. Here is the official example for Python, using the Python Google Api Client lib :

# The destination object resource is entirely optional. If empty, we use
# the source object's metadata.
if reuse_metadata:
    destination_object_resource = {}
else:
    destination_object_resource = {
            'contentLanguage': 'en',
            'metadata': {'my-key': 'my-value'},
    }
req = client.objects().copy(
        sourceBucket=bucket_name,
        sourceObject=old_object,
        destinationBucket=bucket_name,
        destinationObject=new_object,
        body=destination_object_resource)
resp = req.execute()
print json.dumps(resp, indent=2)