Akka Actor Priorities

Question

I have an actor-based system that performs periodic, cpu-intensive data ingests as well as serves RESTful endpoints. I'm using Akka actors to signal/control the various stages of the ingest process, and Spray (which is of course built on Akka) to serve my restful endpoints.

My problem is this: When the ingest kicks off it consumes most of the CPU, starving the RESTful endpoints until its done.

What's the best way to lower the priority of the ingest? Right now the ingest and the Spray module share the same ActorSystem, but they could be separated if that helps the solution.

Answer 1

It seems the different actors in your system need to live in different dispatchers. Create a new dispatcher for the CPU-intensive actors and leave the web service actors in the default dispatcher (or maybe move those to another dispatcher as well, if you see fit)

You may want to tweak the newly created dispatcher - for example, if you say your ingest actors perform computationally intensive jobs you should reduce the degree of parallelism of the dispatcher to something closer to 1.0

Separating your actor system into different dispatchers prevents problems similar to the one you have - if some actors start to hog the underlying threads they end up saturating the dispatcher that runs them. By having the web actors in another dispatcher you limit the impact that the CPU-intensive actors have on the rest of the system. This is somewhat similar to the concept of "bulkheading".

Here's some more info on Akka dispatchers: http://doc.akka.io/docs/akka/2.2.0/scala/dispatchers.html

To configure new dispatcher it's also worthwhile to take a look at the configuration section of the documentation: http://doc.akka.io/docs/akka/2.2.0/general/configuration.html

Answer 2

There are priority mailboxes that can be built to define which messages are dealt with in which priority. You can also stash/unstash messages which works nicely with hotswap if you want to stash messages and deal with them once a certain state is encountered. Info on stash is here: http://doc.akka.io/docs/akka/snapshot/scala/actors.html