If Google App Engine cron jobs have a 10 minute li

Question

According to https://developers.google.com/appengine/docs/python/config/cron cron jobs can run for 10 minutes. However, when I try and test it by going to the url for the cron job when signed in as an admin, it times out with a DeadlineExceededError. Best I can tell this happens about 30 seconds in, which is the non-cron limit for requests. Do I need to do something special to test it with the cron rules versus the normal limits?

Here's what I'm doing:

• Going to the url for the cron job
• This calls my handler which calls a single function in my py script
• This function does a database call to google's cloud sql and loops through the resulting rows, calling a function on each row that use's ebay's api to get some data
• The data from the ebay api call is stored in an array to all be written back to the database after all the calls are done.
• Once the loop is done, it writes the data to the database and returns back to the handler
• The handler prints a done message

It always has issues during the looping ebay api calls. It's something like 500 api calls that have to be made in the loop.

Any idea why I'm not getting the full 10 minutes for this?

Edit: I can post actual code if you think it would help, but I'm assuming it's a process that I'm doing wrong, rather than an error in the code since it works just fine if I limit the query to about 60 api calls.

Answer 1

Here is how you can clarify the exception and tell if it's a urlfetch problem. If the exception is:

* google.appengine.runtime.DeadlineExceededError: raised if the overall request times out, typically after 60 seconds, or 10 minutes for task queue requests;
* google.appengine.runtime.apiproxy_errors.DeadlineExceededError: raised if an RPC exceeded its deadline. This is typically 5 seconds, but it is settable for some APIs using the 'deadline' option;
* google.appengine.api.urlfetch_errors.DeadlineExceededError: raised if the URLFetch times out.

then see https://developers.google.com/appengine/articles/deadlineexceedederrors as it's a urlfetch issue.

If it's the urlfetch that's timing out, try setting a longer duration (ex 60 sec.):

 result = urlfetch.fetch(url, deadline=60)

Answer 2

The way GAE executes a cron job allows it to run for 10 min. This is probably done (i'm just guessing here) through checking the user-agent, IP address, or some other method. Just because you setup a cron job to hit a URL in your application doesn't mean a standard HTTP request from your browser will allow it to run for 10 minutes.

The way to test if the job works is to do so on the local dev server where there is no limit. Or wait until your cron job executes and check the logs for any errors.

Hope this helps!