Memory leak in Go http standard library?

Question

Have a Go binary implement an http server:

package main

import (
    "net/http"
)

func main() {
    http.ListenAndServe(":8080", nil)
}

It will start with ~850 kb or so of memory. Send it some requests via your web browser. Observe it quickly rises to 1 mb. If you wait, you'll see it never goes down. Now hammer it with Apache Bench (using the script below) and see your memory usage continually increase. After sometime it will eventually plateau at around 8.2 MB or so.

Edit: It doesn't seem to stop at 8.2, rather it slows down significantly. It's currently at 9.2 and still rising.

In short, why is this happening? I used this shell script:

while [ true ]
do
    ab -n 1000 -c 100 http://127.0.0.1:8080/
    sleep 1
end

While trying to get to the bottom of this, I've tried to tweak the settings. I've tried closing using r.Close = true to prevent Keep-Alive. Nothing seems to work.

I found this originally while trying to determine if there was a memory leak in a program I'm writing. It has a lot of http handlers and I/O calls. After checking I had closed all my database connections I kept seeing it's memory usage rise. My program to plateau at around 433 MB.

Here's the output of Goenv:

GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mark/Documents/Programming/Go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
TERM="dumb"
CC="clang"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fno-common"
CXX="clang++"
CGO_ENABLED="1"

Answer 1

From the heap pprof you have provided in comments, it looks like you are leaking memory via gorilla/sessions and gorilla/context (almost 400MB).

Refer to this ML thread: https://groups.google.com/forum/#!msg/gorilla-web/clJfCzenuWY/N_Xj9-5Lk6wJ and this GH issue: https://github.com/gorilla/sessions/issues/15

Here is a version that leaks extremely quickly:

package main

import (
    "fmt"
    // "github.com/gorilla/context"
    "github.com/gorilla/sessions"
    "net/http"
)

var (
    cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
    http.HandleFunc("/", defaultHandler)
    http.ListenAndServe(":8080", nil)
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
    cookieStore.Get(r, "leak-test")
    fmt.Fprint(w, "Hi there")
}

Here is one that cleans up and has a relatively static RSS:

package main

import (
    "fmt"
    "github.com/gorilla/context"
    "github.com/gorilla/sessions"
    "net/http"
)

var (
    cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
    http.HandleFunc("/", defaultHandler)
    http.ListenAndServe(":8080", context.ClearHandler(http.DefaultServeMux))
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
    cookieStore.Get(r, "leak-test")
    fmt.Fprint(w, "Hi there")
}