When are scala's for-comprehensions lazy?

Question

In Python, I can do something like this:

lazy = ((i,j) for i in range(0,10000) for j in range(0,10000))
sum((1 for i in lazy))

It will take a while, but the memory use is constant.

The same construct in scala:

(for(i<-0 to 10000; j<-i+1 to 10000) yield (i,j)).count((a:(Int,Int)) => true)

After a while, I get a java.lang.OutOfMemoryError, even though it should be evaluated lazily.

Answer 1

Laziness comes not from the for-comprehension, but from the collection itself. You should look into the strictness characteristics of the collection.

But, for the lazy :-), here's a summary: Iterator and Stream are non-strict, as are selected methods of the view of any collection. So, if you want laziness, be sure to .iterator, .view or .toStream your collection first.

Answer 2

Nothing's inherently lazy about Scala's for-comprehension; it's syntactic sugar* which won't change the fact that the combination of your two ranges will be eager.

If you work with lazy views of your ranges, the result of the comprehension will be lazy too:

scala> for(i<-(0 to 10000).view; j<-(i+1 to 10000).view) yield (i,j)
res0: scala.collection.SeqView[(Int, Int),Seq[_]] = SeqViewN(...)

scala> res0.count((a: (Int, Int)) => true)
res1: Int = 50005000

The laziness here is nothing to do with the for-comprehension, but because when flatMap or map (see below) are called on some type of container, you get back a result in the same type of container. So, the for-comprehension will just preserve the laziness (or lack of) of whatever you put in.

---

*for something like:

(0 to 10000).flatMap(i => (i+1 to 10000).map(j => (i, j)))