Creating an idle timeout in Go?

Question

I use CloudFlare for one of my high volume websites, and it sits in front of my stack.

The thing is CloudFlare leaves idle connections open in addition to creating new ones, and it's not a setting I can change.

When I have Varnish or Nginx sitting in front listening on port 80 they have out of the box configuration to hang up the idle connections.

This is fine until I had to add a proxy written in Go to the front of my stack. It uses the net/http standard library.

I'm not a Go wizard but based on what people are telling me there are only read and write timeout settings but not hanging up idle connections.

Now my server will fill up with connections and die unless I set a set read and write timeouts, but the problem with this is my backend takes a long time sometimes and it's causing good requests to be cut off when they shouldn't.

What is the proper way to handle idle connections with Go http?

Edit 1: To be more clear, I'm using httputil.NewSingleHostReverseProxy to construct a proxy, which exposes transport options but only for the upstream. The problems I am having are downstream, they need to be set on the http.Server object that uses the ReverseProxy as a handler. http.Server does not expose transport.

Edit 2: I would prefer an idle timeout to a read timeout, since the later would apply to an active uploader.

Thanks

Answer 1

The proper way to hangup idle connections in the Go http server is to set the read timeout.

It is not necessary to set the write timeout to hang up on idle clients. Don't set this value or adjust it up if it's cutting off responses.

If you have long uploads, then use a connection state callback to implement separate idle and read timeouts:

server.ConnState = func(c net.Conn, cs http.ConnState) {
    switch cs {
    case http.StateIdle, http.StateNew:
        c.SetReadDeadline(time.Now() + idleTimeout)
    case http.StateActive:
        c.SetReadDeadline(time.Now() + activeTimeout)
    }
}

Answer 2

See the net/http.Transport docs. The Transport type has some options for dealing with idle HTTP connections in the keep-alive state. From reading your question, the option that seems most relevant to your problem is the MaxIdleConnsPerHost field:

MaxIdleConnsPerHost, if non-zero, controls the maximum idle (keep-alive) to keep per-host. If zero, DefaultMaxIdleConnsPerHost is used.

Reading the code, the default is 2 per host.

The Transport type also has a method to zap all idle connections: CloseIdleConnections.

CloseIdleConnections closes any connections which were previously connected from previous requests but are now sitting idle in a "keep-alive" state. It does not interrupt any connections currently in use.

You can specify a Transport on any http client:

tr := &http.Transport{
    TLSClientConfig:    &tls.Config{RootCAs: pool},
    DisableCompression: true,
    MaxIdleConnsPerHost: 1,
}
client := &http.Client{Transport: tr}
resp, err := client.Get("https://example.com")

Another thing worth noting: the docs recommend that you keep a single http client object that is re-used across all your requests (i.e. like a global variable).

Clients and Transports are safe for concurrent use by multiple goroutines and for efficiency should only be created once and re-used.

If you are creating many http client objects in your proxy implementation, it might explain unbounded growth of idle connections (just guessing at how you might be implementing this, though).

EDIT: Reading a little bit more, the net/httputil package has some convenience types for reverse proxies. See the ReverseProxy type. That struct also allows you to supply your own Transport object, allowing you to control your proxy's idle client behavior via this helper type.