Chunked http decoding in java?

Question

I am decoding http packets. And I faced a problem that chunk problem. When I get a http packet it has a header and body. When transefer-encoding is chunked I don't know what to do ?

Is there a useful API or class for dechunk the data in JAVA ?

And if someone , experienced about http decoding , please show me a way how to do this ?

Answer 1

Use a fullworthy HTTP client like Apache HttpComponents Client or just the Java SE provided java.net.URLConnection (mini tutorial here). Both handles it fully transparently and gives you a "normal" InputStream back. HttpClient in turn also comes with a ChunkedInputStream which you just have to decorate your InputStream with.

If you really insist in homegrowing a library for this, then I'd suggest to create a class like ChunkedInputStream extends InputStream and write logic accordingly. You can find more detail how to parse it in this Wikipedia article.

Answer 2

If you are looking for a simple API try Jodd Http library (http://jodd.org/doc/http.html). It handles Chunked transfer encoding for you and you get the whole body as a string back.

From the docs:

HttpRequest httpRequest = HttpRequest.get("http://jodd.org");
HttpResponse response = httpRequest.send();

System.out.println(response);

Answer 3

Here is quick-and-dirty alternative that requires no dependency except Oracle JRE:

private static byte[] unchunk(byte[] content) throws IOException {
    ByteArrayInputStream bais = new ByteArrayInputStream(content);
    ChunkedInputStream cis = new ChunkedInputStream(bais, new HttpClient() {}, null);
    return readFully(cis);
}

It uses the same sun.net.www.http.ChunkedInputStream as java.net.HttpURLConnection does behind the scene.

This implementation doesn't provide detailed exceptions (line numbers) on wrong content format.

It works with Java 8 but could fail in with next release. You've been warned.

Could be useful for prototyping though.

You can choose any readFully implementation from Convert InputStream to byte array in Java.

Answer 4

Apache HttpComponents

Oh, and if we are talking about the client side, HttpUrlConnection does this as well.