Out of memory error during Scala compilation

Question

I'm using the macro-heavy scala-pickling along with shapeless, and I keep crashing the Scala 2.10.3 compiler with an apparent out of memory error. The tail of the error message looks like this:

[error]     <tpt> // tree.tpe=tasks.anonfun$218
[error]     Block( // tree.tpe=Unit
[error]       Apply( // def <init>(): scala.runtime.AbstractFunction1 in class AbstractFunction1, tree.tpe=scala.runtime.AbstractFunction1
[error]         SimpleMiddlebury$$anonfun$218.super."<init>" // def <init>(): scala.runtime.AbstractFunction1 in class AbstractFunction1, tree.tpe=()scala.runtime.AbstractFunction1
[error]         Nil
[error]       )
[error]       ()
[error]     )
[error]   )
[error] )
[error] 
[error] == Expanded type of tree ==
[error] 
[error] ConstantType(value = Constant(anon$326))
[error] 
[error] uncaught exception during compilation: java.io.IOException
[error] Cannot allocate memory

I'm monitoring my system memory, and there's plenty, so if this is really a problem with memory allocation, I'm guessing there's some JVM flag I can set somewhere to make the problem go away.

However, I have tried increasing the available memory, by adjusting the heap setting in paulp's sbt script to -Xmx8g, and I still get this error. Is this the right flag?

Ideas?

EDIT: I added the "scala-pickling" and "shapeless" tags, because this is something other users of those libraries may have encountered.

Answer 1

in your sbt installation directory lives a file ./bin/sbt

update the -Xss settings in this file to a higher threshold. Setting it to -Xss8M was enough for my shapeless stack overflow compilation issues to go away