404 error when saving to GCS with client library o

Question

I have a python GAE app that uses GCS for file storage. It works perfectly in production, but fails completely when running locally on dev_appserver.

Any help getting to the bottom of this would be appreciated.

Thanks.

CODE:

BUCKET = '/mybucket'
filename = BUCKET + '/foo.jpg'
gcs_file = gcs.open(filename, 'w', content_type=content_type)
gcs_file.write(file)
gcs_file.close()

ERROR:

INFO:root:default: "POST /_ah/gcs/mybucket/foo.jpg HTTP/1.1" 404 52
ERROR 2013-11-12 07:45:05,905 webapp2.py:1552] Expect status [201] from Google Storage. But got status 404.
Path: u'/mybucket/foo.jpg'.
Request headers: {'x-goog-resumable': 'start', 'x-goog-api-version': '2', 'content-type': u'image/jpeg', 'accept-encoding': 'gzip, *'}.
Response headers: {'date': 'Tue, 12 Nov 2013 07:45:05 GMT', 'expires': 'Fri, 01 Jan 1990 00:00:00 GMT', 'content-type': 'text/plain; charset=UTF-8', 'content-length': '52', 'server': 'Development/2.0', 'cache-control': 'no-cache'}.
Extra info: None.
Traceback (most recent call last):
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
      rv = self.handle_exception(request, response, e)
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
      rv = self.router.dispatch(request, response)
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
INFO:root:default: "POST /admin/projects HTTP/1.1" 500 3587
        return route.handler_adapter(request, response)
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
        return handler.dispatch()
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
        return self.handle_exception(e, self.app.debug)
    File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
        return method(*args, **kwargs)
    File "/Users/me/myapp/admin.py", line 194, in post
        photo = self.create_image_entity(img_object)
    File "/Users/me/myapp/admin.py", line 234, in create_image_entity
        self.write_file_to_gcs(filename=filename, file=img_data, content_type=img_object['mimetype'])
    File "/Users/me/myapp/admin.py", line 276, in write_file_to_gcs
       gcs_file = gcs.open(filename, 'w', content_type=content_type)
    File "/Users/me/myapp/external/cloudstorage/cloudstorage_api.py", line 74, in open
       return storage_api.StreamingBuffer(api, filename, content_type, options)
    File "/Users/me/myapp/external/cloudstorage/storage_api.py", line 597, in __init__
        errors.check_status(status, [201], path, headers, resp_headers)
    File "/Users/me/myapp/external/cloudstorage/errors.py", line 108, in check_status
        raise NotFoundError(msg)
NotFoundError: Expect status [201] from Google Storage. But got status 404.
Path: u'/mybucket/foo.jpg'.
Request headers: {'x-goog-resumable': 'start', 'x-goog-api-version': '2', 'content-type': u'image/jpeg', 'accept-encoding': 'gzip, *'}.
Response headers: {'date': 'Tue, 12 Nov 2013 07:45:05 GMT', 'expires': 'Fri, 01 Jan 1990 00:00:00 GMT', 'content-type': 'text/plain; charset=UTF-8', 'content-length': '52', 'server': 'Development/2.0', 'cache-control': 'no-cache'}.
Extra info: None.

Answer 1

I think the issue is that the gcs file writing requires authorization , when u are running on the production then u are using an app that is authorized to write on the gcs by this step (as described in documentation)

API calls to BigQuery and Cloud Storage require authorization. We will be using the App Engine service account authorization method in this codelab. To set up your App Engine app to make calls to these API:

Copy the App Engine service account name that we noted above (in the form of [e-mail]).
Visit your API console's Team tab, and add the service account name to the project teammate with Can edit permissions.

while working on local server you have not this authorization.

Answer 2

I got the same problem. I am using google_appengine_1.8.7 (from zip) and appengine-gcs-client-python-r127.zip.

The end point "_ah/gcs" is not handled in dev_appserver.

In fact i think it is related to this change (https://code.google.com/p/appengine-gcs-client/source/detail?r=125) and seems that the GAE 1.8.7. is not align with this change.

I use this appengine-gcs-client-python-r65.zip and it works fine now!