Scala's “for comprehension” with futures

Question

I am reading through the Scala Cookbook (http://shop.oreilly.com/product/0636920026914.do)

There is an example related to Future use that involves for comprehension.

So far my understanding about for comprehension is when use with a collection it will produce another collection with the same type. For example, if each futureX is of type Future[Int], the following should also be of type Future[Int]:

for {
   r1 <- future1
   r2 <- future2
   r3 <- future3
} yield (r1+r2+r3)

Could someone explain me what exactly happening when use <- in this code? I know if it was a generator it will fetch each element by looping.

Answer 1

To elaborate those existing answers here a simple result to demonstrate how for comprehension works.

Its bit lengthy functions yet they worth taking look into it.

A function that give us a range of integers

scala> def createIntegers = Future{
             println("INT "+ Thread.currentThread().getName+" Begin.")
             val returnValue = List.range(1, 256)
             println("INT "+ Thread.currentThread().getName+" End.")
             returnValue
         }
createIntegers: createIntegers: scala.concurrent.Future[List[Int]]

A function that give us a range of chars

scala> def createAsciiChars = Future{
             println("CHAR "+ Thread.currentThread().getName+" Begin.")
             val returnValue = new ListBuffer[Char]
             for (i <- 1 to 256){
                  returnValue += i.toChar
             }
             println("CHAR "+ Thread.currentThread().getName+" End.")
             returnValue
          }
createAsciiChars: scala.concurrent.Future[scala.collection.mutable.ListBuffer[Char]]

Using these function calls within the for comprehension.

scala> val result = for{
                        i <- createIntegers
                        s <- createAsciiChars
                    } yield i.zip(s)
       Await.result(result, Duration.Inf)
result: scala.concurrent.Future[List[(Int, Char)]] = Future(<not completed>)

For these below lines we can make out that all the function calls are synchronous i.e. createAsciiChars function call is not executed until createIntegers completes its execution.

scala> INT scala-execution-context-global-27 Begin.
       INT scala-execution-context-global-27 End.
       CHAR scala-execution-context-global-28 Begin.
       CHAR scala-execution-context-global-28 End.

Making these function createAsciiChars, createIntegers calls outside the for comprehensions will be asynchronous execution.

Answer 2

First about for comprehension. It was answered on SO many many times, that it's an abstraction over a couple of monadic operations: map, flatMap, withFilter. When you use <-, scalac desugars this lines into monadic flatMap:

r <- monad into monad.flatMap(r => ... )

it looks like an imperative computation (what a monad is all about), you bind a computation result to the r. And yield part is desugared into map call. Result type depends on the type of monad's.

Future trait has a flatMap and map functions, so we can use for comprehension with it. In your example can be desugared into the following code:

future1.flatMap(r1 => future2.flatMap(r2 => future3.map(r3 => r1 + r2 + r3) ) )

Parallelism aside

It goes without saying that if execution of future2 depends on r1 then you can't escape sequential execution, but if the future computations are independent, you have two choices. You can enforce sequential execution, or allow for parallel execution. You can't enforce the latter, as the execution context will handle this.

val res = for {
   r1 <- computationReturningFuture1(...)
   r2 <- computationReturningFuture2(...)
   r3 <- computationReturningFuture3(...)
} yield (r1+r2+r3)

will always run sequentially. It can be easily explained by the desugaring, after which the subsequent computationReturningFutureX calls are only invoked inside of the flatMaps, i.e.

computationReturningFuture1(...).flatMap(r1 => 
    computationReturningFuture2(...).flatMap(r2 => 
        computationReturningFuture3(...).map(r3 => r1 + r2 + r3) ) )

However this is able to run in parallel and the for comprehension aggregates the results:

val future1 = computationReturningFuture1(...)
val future2 = computationReturningFuture2(...)
val future3 = computationReturningFuture3(...)

val res = for {
   r1 <- future1
   r2 <- future2
   r3 <- future3
} yield (r1+r2+r3)

Answer 3

It allows r1, r2, r3 to run in parallel, if possible. It may not be possible, depending things like how many threads are available to execute Future computations, but by using this syntax you are telling the compiler to run these computations in parallel if possible, then execute the yield() when all have completed.