GAE Starting dynamic instance to serve requests in

Question

I ve checked a bunch of other post about the same subject like this one : “Resident instance doesn't seem to work” but I couldn t find a working answer for my problem

I have 4 F2 resident and whenever someone goes to my application, instead of using a resident instance, a new dynamic instance is mounted to answer the request. And so the person who is attempting to use my application will wait 20 seconds. Then all the new requests will come to the new dynamic instance. The response time on my resident instances are inferior to my min pending latency (5 sec). My app is in production since yesterday and if no one uses it for 5 minutes then the next guy who will connect will have to wait 20 sec !

From what I ve read, some would say that if I set min idle instance to 4 ( in order to have my resident instances) GAE will try to have 4 iddle instances all the time and so we ll start new instances; but then I would always have to wait 35 sec for all my first connections After 2 minutes of using the application everything is fine. A guy working at Google told me that there is a bug in the management of resident instances and that I have to have 3 resident instances to have a quick answer for my users. I m sure I haven t understood something. Anyone could help me let my users have a fair response time when they connect to my application and no one have connected for 5 minutes please ?

Thank you very much

Answer 1

Check the GAE mailing list on google groups, you'll see this question come up over and over and over again.

The short answer is, GAE is designed for high loads. Resident instances exist to handle spikes in workloads, although I haven't really seen a formal definition of a "spike". If GAE's mysterious load balancing algorithm doesn't determine your traffic as a spike, you'll get a new dynamic instance.

If you throw hundreds or thousands of requests at your app, then some of them will get directed to the resident instances while waiting for dynamic instances to spin up.

There's not too much you can do but join the long list of people who have complained about this. There's at least two issues filed against this:

https://code.google.com/p/googleappengine/issues/detail?id=7865&q=scheduler&colspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log

https://code.google.com/p/googleappengine/issues/detail?id=7706&q=scheduler&colspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log

Alternatively you can rewrite your app so that you don't have a 20s startup time. There's a few things you can do, like defer much of your loading, but it's painful to do in Java.

As another aside, python and Go runtimes seem to start up much faster.

Answer 2

I reached a practical solution that seems to be Ok for now : i ve made the following tests wich could be usefull for low traffic applications on GAE :

• test 1 -> 5xF2 iddle wich 5sec pending latency results : i m still getting a warmup right away as a user –
• test 2 -> 8xF2 iddle with 5 sec pending latency : same as above
• test 3 ->8xF1 residents with 5 sec pending latency : same as above
• test 4 -> ****1xF4 resident with 15 sec pending latency**** : Way better. the firt instance to answer (the 50 to 100 first request) is the resident, then one dynamic instance is opened. and close after 2 minutes without using my app. Almost as I would have dream that GAE would work – So for the last 4 days it worked fine with the 1xF4 resident and 15 sec pending lantency. –

If new problems occur, i m thinking this days that the best way to avoid cold startups on users is to ping my app every 30 sec and i ll work to have a faster cold startup time

Answer 3

Here's the related issue (there may be others):

http://code.google.com/p/googleappengine/issues/detail?id=7706

It seems someone had success with 5, however from your research and other comments I've read it may be just that the chance is better with more.