Java: Stream Contents of Zipfile via HTTP

Question

I have quite some amount of streamable data (>100MB), which, for the sake of compression, i would like to host packed in a zipfile on an http-server. So this zipfile contains a single file.

Now is it possible for a java-client to stream the data via http, even though it is packed in a zipfile?

According to wikipedia, ZIPs are not sequentially...

http://en.wikipedia.org/wiki/ZIP_(file_format)#Structure

If this is still possible somehow, then how?

edit: about gzip: as i said, i use a custom java client (not a webbrowser) is gzip available in the java http implementation?

Answer 1

Would it make more sense to let the web server do the zipping? If you are simply trying to reduce the amount of bandwidth being used, rather than really wanting to store the file zipped up on the server, this would simply be a matter of configurations, for example see:

http://tomcat.apache.org/tomcat-5.5-doc/config/http.html

for HTTP/1.1 GZIP compression. The server can force the response to the client to be zipped.

See also http://en.wikipedia.org/wiki/HTTP_compression.

The client will receive zipped packets and handle the unzipping. It should be possible to stream the file too, so the client doesn't need all the file before it can do something useful, because the server can zip individual chunks.

Answer 2

Yes you can, Stream the zip and use the MIME type as application/zip

If you actually want to play stream music on the other end, then it can't be done trivially as you can only unpack once the entire zip is available on client.

If size is you concern, you can either turn down your mp3 bit-rate or use formats such as ogg/vorbis

Answer 3

Java supports the gzip format with the GZipInputStream (decompressing) and GZipOutputStream (compressing). Both zip and gzip use the same compressing format internally, the main difference is in the metadata: zip has it at the end of the file, gzip at the beginning (and gzip only supports one enclosed file easily).

For your of streaming one big file, using gzip will be the better thing to do - even more as you don't need access to the metadata.

I'm not sure if the HTTPConnection sends Accept-Encoding: gzip and then handles inflating the content automatically if the server delivers it with Content-Encoding: gzip, but you surely can do it manually if the server simply sends a the .gz file as such (i.e. with Content-Encoding: identity).

(By the way, make sure to read from the stream with not too small buffers, as each deflate call will have a native call overhead, since Java's GZipInputStream uses the native zlib implementation.)

Answer 4

Use GZIP and then you can stream. Gzip uses the default compression algorithm of zip anyway.

Answer 5

Here's a snippet of code (that works) that the client can use to read from the zipped stream:

static void processZippedInputStream(InputStream in, String entryNameRegex)
throws IOException
{
    ZipInputStream zin = new ZipInputStream(in);
    ZipEntry ze;
    while ((ze = zin.getNextEntry()) != null)
    {
        if (ze.getName().matches(entryNameRegex))
        {
            // treat zin as a normal input stream - ie read() from it till "empty" etc
            break;
        }
        zin.closeEntry();
    }
    zin.close();
}

The main difference with a normal InputStream is iterating through the entries. You may know, for example, that you want the first entry, so no need for the name matching parameter etc.