Why am I Failing to combine Enumerators in SimpleR

Question

Im trying to achieve something similar to the first example under "Sending large amounts of data" at http://www.playframework.org/documentation/2.0/ScalaStream.

What I am doing differently is that I have more than one file that I want to concatenate in the response. It looks like this:

def index = Action {

  val file1 = new java.io.File("/tmp/fileToServe1.pdf")
  val fileContent1: Enumerator[Array[Byte]] = Enumerator.fromFile(file1)    

  val file2 = new java.io.File("/tmp/fileToServe2.pdf")
  val fileContent2: Enumerator[Array[Byte]] = Enumerator.fromFile(file2)   

  SimpleResult(
    header = ResponseHeader(200),
    body = fileContent1 andThen fileContent2
  )

}

What happens is that only the contents of the first file is in the response.

Something slightly simpler like below works fine though:

fileContent1 = Enumerator("blah" getBytes)
fileContent2 = Enumerator("more blah" getBytes)

SimpleResult(
  header = ResponseHeader(200),
  body = fileContent1 andThen fileContent2
)

What am I getting wrong?

Answer 1

I've got most of this from reading through the play code so I might have misunderstood, but from what I can see: the Enumerator.fromFile function(Iteratee.scala [docs][src]) creates an enumerator which when applied to an Iteratee[Byte, Byte] appends an Input.EOF to the output of the Iteratee when it's done reading from the file.

According to the Play! docs example that you linked to you're supposed to manually append an Input.EOF (alias of Enumerator.eof) at the end of the enumerator. I would think that, having an Input.EOF automatically appended to the end of the file's byte array is the reason that you're getting only one file returned.

The equivalent simpler example in the play console would be:

scala> val fileContent1 = Enumerator("blah") andThen Enumerator.eof
fileContent1: play.api.libs.iteratee.Enumerator[java.lang.String] = play.api.libs.iteratee.Enumerator$$anon$23@2256deba

scala> val fileContent2 = Enumerator(" and more blah") andThen Enumerator.eof
fileContent2: play.api.libs.iteratee.Enumerator[java.lang.String] = play.api.libs.iteratee.Enumerator$$anon$23@7ddeef8a

scala> val it: Iteratee[String, String] = Iteratee.consume[String]()
it: play.api.libs.iteratee.Iteratee[String,String] = play.api.libs.iteratee.Iteratee$$anon$18@6fc6ce97

scala> Iteratee.flatten((fileContent1 andThen fileContent2) |>> it).run.value.get
res9: String = blah

The fix, although I haven't tried this yet would be to go a few levels deeper and use the Enumerator.fromCallback function directly and pass it a custom retriever function that keeps returning Array[Byte]s until all the files that you want to concatenate have been read. Refer to the implementation of the fromStream function to see what the default is and how to modify it.

An example of how to do this (adapted from Enumerator.fromStream):

def fromFiles(files: List[java.io.File], chunkSize: Int = 1024 * 8): Enumerator[Array[Byte]] = {
  val inputs = files.map { new java.io.FileInputStream(_) }
  var inputsLeftToRead = inputs
  Enumerator.fromCallback(() => {
    def promiseOfChunkReadFromFile(inputs: List[java.io.FileInputStream]): Promise[Option[Array[Byte]]] = {
      val buffer = new Array[Byte](chunkSize)
      (inputs.head.read(buffer), inputs.tail.headOption) match {
        case (-1, Some(_)) => {
          inputsLeftToRead = inputs.tail
          promiseOfChunkReadFromFile(inputsLeftToRead)
        }
        case (-1, None) => Promise.pure(None)
        case (read, _) => {
          val input = new Array[Byte](read)
          System.arraycopy(buffer, 0, input, 0, read)
          Promise.pure(Some(input))
        }
      }
    }
    promiseOfChunkReadFromFile(inputs)
  }, {() => inputs.foreach(_.close())})
}

Answer 2

This is the play framework 2.2 code that I am using to play together a bunch of audio files one after another. You can see the use of a scala Iterator that gets converted to a java Enumeration for java.io.SequenceInputStream.

def get_files(name: String) = Action { implicit request =>
  import java.lang.IllegalArgumentException
  import java.io.FileNotFoundException
  import scala.collection.JavaConverters._
  try {
    val file1 = new java.io.File("songs/phone-incoming-call.ogg")
    val file2 = new java.io.File("songs/message.ogg")
    val file3 = new java.io.File("songs/desktop-login.ogg")
    val inputStream1 = new java.io.FileInputStream(file1)
    val inputStream2 = new java.io.FileInputStream(file2)
    val inputStream3 = new java.io.FileInputStream(file3)
    val it = Iterator(inputStream1, inputStream2, inputStream3)
    val seq = new java.io.SequenceInputStream(it.asJavaEnumeration)
    val fileContent: Enumerator[Array[Byte]] = Enumerator.fromStream(seq)
    val length: Long = file1.length + file2.length + file3.length
    SimpleResult(
      header = ResponseHeader(200, Map(CONTENT_LENGTH -> length.toString)),
      body = fileContent
    )
  }
  catch {
    case e: IllegalArgumentException =>
      BadRequest("Could not retrieve file. Error: " + e.getMessage)
    case e: FileNotFoundException =>
      BadRequest("Could not find file. Error: " + e.getMessage)
  }
}

The only problem with my code (in terms of what I am looking for) is that it is a stream meaning that I can't resume playing from a previous point in time even though I am sending the CONTENT_LENGTH. Maybe I am calculating the total length in a wrong way.

The other issue with it is that the audio files need to be of the same type (expected), same amount of channels and same sample rate (took me a while to figure out), e.g all have to be ogg 44100Hz 2 channels.