Bad response to a BigQuery query: kind:discovery#r

Question

I'm getting a response I'm totally not expecting to a query I'm doing from appengine.

Instead of a response with the kind "bigquery#queryResults" which is expected according to the BigQuery docs, I'm getting

u'basePath': u'/bigquery/v2/', u'kind': u'discovery#restDescription',....

It seems to only happen with queries that would have taken 10-15 seconds and up, and I suspect this is some weird interaction with the Appengine dynamic 60 second timeout.

I'm running on python27, and use the google-api-python-client for the requests.

Update

Tried a few other scenarios:

• running it on a backend - where no appengine request timeouts apply - same thing: ~9.8s into the request the discovery json is returned

• ran it in dev (localhost) again, to be sure: everything is fine

• most interesting, running it with with a shorter timeout of 4s (as opposed to 35s previously) results in the same discovery reply after the same 9.8s interval

Answer 1

From the App Engine quota docs:

Each incoming HTTP request can be no larger than 32MB.

It might be possible that your queries are returning greater than 32 MB of data and this is causing HTTP 500 errors, resulting in the API providing the output you see above (which is the BigQuery API Discovery doc information - this response may be related to a separate issue with how the Python Client lib handles error responses on App Engine).

Suggested workaround: If your query responses are returning a great deal of data, you can use the maxResults parameter to limit the amount of rows returned in a response page to a smaller amount. Then, you can make successive calls to the API to retrieve subsequent pages of data.

Answer 2

This is a known and fixed bug in the underlying httplib2 library.

Make sure you are using version 0.8 or later:

Check for in httplib2/__init__.py:

__version__ = "0.8"