Google App Engine time out?

Question

In a Google App Engine app I used the following lines to read a page from a site :

  String Url="http://...",line,Result="";

  URL url=new URL(Url+"?r="+System.currentTimeMillis());
  BufferedReader reader=new BufferedReader(new InputStreamReader(url.openStream()));

  while ((line=reader.readLine())!=null) { Result+=line+"\n"; }
  reader.close();

But I got the following error :

Uncaught exception from servlet
com.google.apphosting.api.DeadlineExceededException: This request (f5e2889605d27d42) started at 2011/09/07 03:20:41.458 UTC and was still executing at 2011/09/07 03:21:30.888 UTC.
    at sun.misc.Unsafe.park(Native Method)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
    at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276)
    at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:82)
    at com.google.appengine.tools.development.TimedFuture.get(TimedFuture.java:55)
    at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:177)
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:56)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:150)
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:148)
    at java.security.AccessController.doPrivileged(Native Method)

Seems it took longer than it would like to wait, what can I do if that site is slow ?

Answer 1

If your code is running inside a request handler, then by default there is an App-Engine-enforced 60 second deadline. You can't change it. See the "Deadlines" row / "automatic scaling" column of the chart on this page under "Scaling types":

https://developers.google.com/appengine/docs/java/modules/

However, this code will be able to run for some hours if you change your module to use "manual scaling" and a "B1"-"B4" instance. Example:

default/src/main/webapp/WEB-INF/appengine-web.xml:

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
  <application>myapp</application>
  <module>default</module>
  <version>1</version>
  <threadsafe>true</threadsafe>

  <instance-class>B1</instance-class>
  <manual-scaling>
    <instances>1</instances>
  </manual-scaling>
</appengine-web-app>

On this type of instance, your requests will typically not time out for hours. (The documentation claims that the deadline is "indefinite".)

Answer 2

A DeadlineExceededException is thrown when your code, handling the request to your web application takes over 30 seconds to process. Presumably your code is taking a while to process because of the length of time it had to wait to receive data from some other site.

You can create a task on a task queue to fetch and process that data, and change your web request/response flow to reply with progress on your task.