Keep client from closing connection to a netty ser

Question

I'm debugging a problem with a netty proxy I'm writing, and I've noticed that even if I skip the 'proxy' aspect and implement a simple http server and send two requests serially from a commons httpclient, the commons httpclient closes the connection and the second request is made in a different connection. If I proxy the request on the other hand, the second request uses the same connection, but I get a 'connection reset' exception when I try to write the response to the second request to the client's channel.

Code for my pipeline and basic handler:

    ChannelFactory factory =
            new NioServerSocketChannelFactory(
                    Executors.newCachedThreadPool(),
                    Executors.newCachedThreadPool());
    ServerBootstrap sb = new ServerBootstrap(factory);

    sb.setPipelineFactory(new ChannelPipelineFactory() {
        public ChannelPipeline getPipeline() {
            return Channels.pipeline(
                    new HttpRequestDecoder(),
                    new HttpResponseEncoder(),
                    new RequestHandler());
        }
    });

private static class RequestHandler extends SimpleChannelHandler {

    @Override
    public void messageReceived(ChannelHandlerContext ctx, final MessageEvent e) {
        HttpResponse clientResponse = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
        clientResponse.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/html; charset=UTF-8");
        clientResponse.setContent(ChannelBuffers.wrappedBuffer(new byte[] {1, 2, 3}));
        System.out.println("here: " + e.getChannel());
        e.getChannel().write(clientResponse);
    }
}

Here is tcpdump on port 2080 showing the client closing the connection (open-handshake, push, push, close-handshake, open-handshake, push, push, close-handshake):

[master] sudo tcpdump -i any '(port 2080)'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
15:13:22.396482 IP localhost.localdomain.45724 > localhost.localdomain.autodesk-nlm: Flags [S], seq 3122841582, win 32792, options [mss 16396,sackOK,TS val 880723828 ecr 0,nop,wscale 7], length 0
15:13:22.396499 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.45724: Flags [S.], seq 604436385, ack 3122841583, win 32768, options [mss 16396,sackOK,TS val 880723829 ecr 880723828,nop,wscale 7], length 0
15:13:22.396511 IP localhost.localdomain.45724 > localhost.localdomain.autodesk-nlm: Flags [.], ack 1, win 257, options [nop,nop,TS val 880723829 ecr 880723829], length 0
15:13:22.406805 IP localhost.localdomain.45724 > localhost.localdomain.autodesk-nlm: Flags [P.], seq 1:1600, ack 1, win 257, options [nop,nop,TS val 880723839 ecr 880723829], length 1599
15:13:22.406817 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.45724: Flags [.], ack 1600, win 256, options [nop,nop,TS val 880723839 ecr 880723839], length 0
15:13:22.446068 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.45724: Flags [P.], seq 1:63, ack 1600, win 256, options [nop,nop,TS val 880723878 ecr 880723839], length 62
15:13:22.446083 IP localhost.localdomain.45724 > localhost.localdomain.autodesk-nlm: Flags [.], ack 63, win 257, options [nop,nop,TS val 880723878 ecr 880723878], length 0
15:13:22.449192 IP localhost.localdomain.45724 > localhost.localdomain.autodesk-nlm: Flags [F.], seq 1600, ack 63, win 257, options [nop,nop,TS val 880723881 ecr 880723878], length 0
15:13:22.449360 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.45724: Flags [F.], seq 63, ack 1601, win 256, options [nop,nop,TS val 880723881 ecr 880723881], length 0
15:13:22.449371 IP localhost.localdomain.45724 > localhost.localdomain.autodesk-nlm: Flags [.], ack 64, win 257, options [nop,nop,TS val 880723881 ecr 880723881], length 0
15:13:22.449716 IP localhost.localdomain.35737 > localhost.localdomain.autodesk-nlm: Flags [S], seq 929672323, win 32792, options [mss 16396,sackOK,TS val 880723882 ecr 0,nop,wscale 7], length 0
15:13:22.449729 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.35737: Flags [S.], seq 1626218986, ack 929672324, win 32768, options [mss 16396,sackOK,TS val 880723882 ecr 880723882,nop,wscale 7], length 0
15:13:22.449737 IP localhost.localdomain.35737 > localhost.localdomain.autodesk-nlm: Flags [.], ack 1, win 257, options [nop,nop,TS val 880723882 ecr 880723882], length 0
15:13:22.449986 IP localhost.localdomain.35737 > localhost.localdomain.autodesk-nlm: Flags [P.], seq 1:1600, ack 1, win 257, options [nop,nop,TS val 880723882 ecr 880723882], length 1599
15:13:22.449992 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.35737: Flags [.], ack 1600, win 256, options [nop,nop,TS val 880723882 ecr 880723882], length 0
15:13:22.453566 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.35737: Flags [P.], seq 1:63, ack 1600, win 256, options [nop,nop,TS val 880723886 ecr 880723882], length 62
15:13:22.453582 IP localhost.localdomain.35737 > localhost.localdomain.autodesk-nlm: Flags [.], ack 63, win 257, options [nop,nop,TS val 880723886 ecr 880723886], length 0
15:13:22.475867 IP localhost.localdomain.35737 > localhost.localdomain.autodesk-nlm: Flags [F.], seq 1600, ack 63, win 257, options [nop,nop,TS val 880723908 ecr 880723886], length 0
15:13:22.475998 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.35737: Flags [F.], seq 63, ack 1601, win 256, options [nop,nop,TS val 880723908 ecr 880723908], length 0
15:13:22.476012 IP localhost.localdomain.35737 > localhost.localdomain.autodesk-nlm: Flags [.], ack 64, win 257, options [nop,nop,TS val 880723908 ecr 880723908], length 0

Here is what happens if I use the proxy, which essentially saves the channel from messageReceived(), and looks up this channel a little later to write the response. Note that here the channel is not closed, but is instead reset by the client, leading to an IOException 'connection reset':

[master] sudo tcpdump -i any '(port 2080)'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
15:11:02.055316 IP localhost.localdomain.58266 > localhost.localdomain.autodesk-nlm: Flags [S], seq 1055230627, win 32792, options [mss 16396,sackOK,TS val 880583487 ecr 0,nop,wscale 7], length 0
15:11:02.055333 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.58266: Flags [S.], seq 596566447, ack 1055230628, win 32768, options [mss 16396,sackOK,TS val 880583487 ecr 880583487,nop,wscale 7], length 0
15:11:02.055344 IP localhost.localdomain.58266 > localhost.localdomain.autodesk-nlm: Flags [.], ack 1, win 257, options [nop,nop,TS val 880583487 ecr 880583487], length 0
15:11:02.066169 IP localhost.localdomain.58266 > localhost.localdomain.autodesk-nlm: Flags [P.], seq 1:1600, ack 1, win 257, options [nop,nop,TS val 880583498 ecr 880583487], length 1599
15:11:02.066188 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.58266: Flags [.], ack 1600, win 256, options [nop,nop,TS val 880583498 ecr 880583498], length 0
15:11:02.071439 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.58266: Flags [P.], seq 1:1568, ack 1600, win 256, options [nop,nop,TS val 880583503 ecr 880583498], length 1567
15:11:02.071450 IP localhost.localdomain.58266 > localhost.localdomain.autodesk-nlm: Flags [.], ack 1568, win 256, options [nop,nop,TS val 880583503 ecr 880583503], length 0
15:11:02.076384 IP localhost.localdomain.58266 > localhost.localdomain.autodesk-nlm: Flags [P.], seq 1600:3199, ack 1568, win 256, options [nop,nop,TS val 880583508 ecr 880583503], length 1599
15:11:02.080625 IP localhost.localdomain.autodesk-nlm > localhost.localdomain.58266: Flags [P.], seq 1568:3135, ack 3199, win 256, options [nop,nop,TS val 880583513 ecr 880583508], length 1567
15:11:02.102018 IP localhost.localdomain.58266 > localhost.localdomain.autodesk-nlm: Flags [R.], seq 3199, ack 3135, win 256, options [nop,nop,TS val 880583534 ecr 880583513], length 0

Is there something I'm missing in my handler to keep the client from closing the connection?

EDIT: Reading the response seems to cause the connection reset to go away and the connection to be torn down normally. Any idea why this println prevents the connection reset after the second request?

httpClient.executeMethod(method);
System.out.println(method.getResponseBodyAsString());

Answer 1

Your handler is sending a HTTP v1.1 response where keep-alive is the default connection type, so as long as your client is expecting v1.1 and not v1.0, and behaves correctly, then you shouldn't have to do anything more to indicate to the client to keep the connection open.

If your client is expecting HTTP v1.0 you could add a "Connection: keep-alive" header.

Answer 2

Finally figured this out. There were a couple of issues. The most important issue was that I was not calling method.releaseConnection() from the commons HttpClient that initiated the call to this proxy. For some reason, calling getResponseBodyAsString() or getResponseBodyAsStream.close() both eliminated the reset problem as well.