Play framework 2.0.x AskTimeoutException

Question

I'm trying to do some load testing on my Play 2.0.2 app, but I keep running into the following exception:

Cannot invoke the action, eventually got an error: Thrown(akka.pattern.AskTimeoutException: Timed out)

I'm using http://blitz.io/ to do my load testing.

My tech stack is Play + mysql.

I'm not using any promises in my app.

Here's the customized akka configuration:

#default timeout for promises
promise.akka.actor.typed.timeout=10s

play {

    akka {
        event-handlers = ["akka.event.slf4j.Slf4jEventHandler"]
        loglevel = ERROR

        actor {

            deployment {

                /actions {
                    router = round-robin
                    nr-of-instances = 100
                }

                /promises {
                    router = round-robin
                    nr-of-instances = 100
                }

            }

            retrieveBodyParserTimeout = 10 second

            actions-dispatcher = {
                fork-join-executor {
                    parallelism-factor = 100
                    parallelism-max = 200
                }
            }

        promises-dispatcher = {
                fork-join-executor {
                    parallelism-factor = 100
                    parallelism-max = 200
        }
            }

            websockets-dispatcher = {
                fork-join-executor {
            parallelism-factor = 100
                    parallelism-max = 200
                }
            }

            default-dispatcher = {
                fork-join-executor {
                    parallelism-factor = 100
                    parallelism-max = 200
                }
            }

        }

    }

}   

One more thing: After the application runs for a day or so, it starts slowing down. Upon checking the resources (using htop), I see that the cpu utilization is 100%, so its clear that the node is stuck doing too much work that it's response gets delayed immensely. Furthermore, I can see that a lot of java instances have been spawned (8-10) on average. Why is that? Is it natural for a Play app to spawn so many java instances? About mem usage, the java play instance gets 512MB-1536MB at the time of spawning and I can see from the stats that the memory is about 50-70% used on average so probably thats not where the problem is (possible mem leaks etc).

I'm running on an extremely tight schedule, so I would appreciate any help/pointers I can get into investigating (or possibly fixing) this issue.

Thanks a bunch!

Answer 1

i think you might have some blocking IO, which is why you need the retrieveBodyParserTimeout = 10 second hack. Try making it async.