Spray.io: When (not) to use non-blocking route han

Question

If we are thinking of production grade REST API, should we use non-blocking as much as possible, e.g.

def insertDbAsync(rows: RowList): Future[Unit] = ...
...
val route =
path("database" / "insertRowList") {
  post {
    entity(as[RowList]) { rows =>
      log.info(s"${rows.length} rows received")
      val async = insertDbAsync(rows)
      onComplete(async) {
        case Success(response) =>
          complete("success")
        case Failure(t) =>
          complete("error")
      }
    }
  }
}

I'm thinking that the answer will most likely be a 'yes', but what are some guidelines in deciding what should and should not be a blocking code, and why?

Answer 1

If you're using spray everything must be non-blocking as a matter of correctness - otherwise you'll block the (very small number of) dispatch threads and your server will stop responding.

Answer 2

Spray uses Akka as underlying platform, so recommendations are same as for actors (Blocking Needs Careful Management). Blocking code may require too much threads, which may:

• kill actor's lightweightness: millions of actors may operate on one thread by default. Let's say one non-blocked actor requires 0.001 threads for example. One blocked actor (which blocking time is, let's say, 100 times more than usual) will take 1 thread avg (not always same thread). First, The more threads you have - the more memory you loose - every blocked thread holds full callstack allocated before blocking, including references from stack (so GC can't erase them). Second, if you have more than number_of_processors threads - you will loose the performance. Third, if you use some dynamical pool - adding new thread may take some significant amount of time.

• cause thread's starvation - you may have pool filled with threads, which doing nothing - so new tasks can't be processed before blocking operation complete (0 % CPU load, but 100500 messages waiting to be processed). It may even cause deadlocks. However, Akka uses Fork-Join-Pool by default so if your blocking code is managed (surrounded with scala.concurrent.blocking - Await.result have such surrounding inside ) - it will prevent starvation by cost of creating new thread instead of blocked one, but it won't compensate other problems.

• traditionally cause deadlocks, so it's bad for design

If code is blocking from outside, you can surround it with future:

 import scala.concurrent._
 val f = Future {
     someNonBlockingCode()
     blocking { //mark this thread as "blocked" so fork-join-pool may create another one to compensate
        someBlocking()
     }  
 }

Inside separate actor:

 f pipeTo sender //will send the result to `sender` actor

Inside spray routing:

 onComplete(f) { .. }

It's better to execute such futures inside separate pool/dispatcher (fork-join-pool based).

P.S. As an alternative to futures (they may not be much convinient from design perspectvive) you may consider Akka I/O, Continuations/Coroutines, Actor's pools (also inside separate dispatcher), Disruptor etc.