How to use OutputStreams with chunked response in Play 2.1

loops, outputstream, playframework, scala

Solution

In general you can't safely use any `OutputStream` with the Enumerator/Iteratee framework because `OutputStream` doesn't support non-blocking pushback. However, if you can control the writing to the `OutputStream` you can hack together something like:

val baos = new ByteArrayOutputStream
val zos = new ZipOutputStream(baos)

val enumerator = Enumerator.generateM {
  Future.successful {
    if (moreDateToWrite) {
      // Write data into zos
      val r = Some(baos.toByteArray)
      baos.reset()
      r
    } else None
  }
}

If all you need is compression, take a look at the `Enumeratee` instances provided in `play.filters.gzip.Gzip` and the `play.filters.gzip.GzipFilter` filter.

Problem

I need to use to `java.util.zip.ZipOutputStream` to respond with a compressed file archive. The data is several hundred megabytes uncompressed, so I would like to store as little of it as possible. It is coming from a serialization of SQL results. I see examples of using an `OutputStream` to return a chunked result using `Enumerator.outputStream`: - http://greweb.me/2012/11/play-framework-enumerator-outputstream/ - Play/Akka integration with Java OutputStreams but those seem ill-advised when I read the documentation (emphasis mine) Create an Enumerator of bytes with an OutputStream. Not that calls to write will not block, so if the iteratee that is being fed to is slow to consume the input, the OutputStream will not push back. This means it should not be used with large streams since there is a risk of running out of memory. Clearly, I can't use that. Or at least not without modification. How can I create a response with an `OutputStream` (in this case, a gzipped archive) while being assured that only portions of it will be stored in memory? I recognize the difference between `InputStream`s/`OutputStream`s and Play's `Enumerator`/`Iteratee` paradigm, so I expect there will be a specific way in which I need to generate my source data (serialization of SQL results) so that it doesn't outpace the rate of download. I don't know what it is.

Original source