What InputStream type should be used to handle URLConnection streams that have HTTP Content-Encoding set to deflate?
For a Content-Encoding of gzip or zip I use a GZIPInputStream, no problem.
For a Content-Encoding of "deflate" I have tried using InflaterInputStream and DeflaterInputStream but I get
java.util.zip.ZipException: unknown
compression method
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:147)
My understanding is that "deflate" encoding refers to Zlib compression, and according to the docs this should be handled by InflaterInputStream.
In HTTP/1.1, Content-encoding: deflate
actually refers to the DEFLATE compression algorithm, as defined by RFC 1951, wrapped in the zlib data format, as defined by RFC 1950.
However some vendors just implement the DEFLATE algorithm as defined RFC 1951, completely ignoring RFC 1950 (no zlib headers).
Others have been hit by the same issue:
- http://www.mail-archive.com/www-talk@w3.org/msg01000.html
- Internet Explorer 8 + Deflate
In order to work around this, try to instantiate the InflaterInputStream
passing an Inflater
that was created with the nowrap
parameter set to true
:
in = new InflaterInputStream(conn.getInputStream()), new Inflater(true));
Unfortunately, using the InflaterInputStream with an Inflater object did not always produce the correct decompression. I had to detect the headers and tell the Inflater where the offset to the payload was.
http://thushw.blogspot.com/2014/05/decoding-html-pages-with-content.html