Timeout for WaitGroup.Wait()

Question

What is an idiomatic way to assign a timeout to WaitGroup.Wait() ?

The reason I want to do this, is to safeguard my 'scheduler' from potentially awaiting an errant 'worker' for ever. This leads to some philosophical questions (i.e. how can the system reliably continue once it has errant workers?), but I think that's out of scope for this question.

I have an answer which I'll provide. Now that I've written it down, it doesn't seem so bad but it still feels more convoluted than it ought to. I'd like to know if there's something available which is simpler, more idiomatic, or even an alternative approach which doesn't use WaitGroups.

Ta.

Answer 1

I wrote a library that encapsulates the concurrency logic https://github.com/shomali11/parallelizer which you can also pass a timeout.

Here is an example without a timeout:

func main() {
    group := parallelizer.DefaultGroup()

    group.Add(func() {
        for char := 'a'; char < 'a'+3; char++ {
            fmt.Printf("%c ", char)
        }
    })

    group.Add(func() {
        for number := 1; number < 4; number++ {
            fmt.Printf("%d ", number)
        }
    })

    err := group.Run()

    fmt.Println()
    fmt.Println("Done")
    fmt.Printf("Error: %v", err)
}

Output:

a 1 b 2 c 3 
Done
Error: <nil>

Here is an example with a timeout:

func main() {
    options := &parallelizer.Options{Timeout: time.Second}
    group := parallelizer.NewGroup(options)

    group.Add(func() {
        time.Sleep(time.Minute)

        for char := 'a'; char < 'a'+3; char++ {
            fmt.Printf("%c ", char)
        }
    })

    group.Add(func() {
        time.Sleep(time.Minute)

        for number := 1; number < 4; number++ {
            fmt.Printf("%d ", number)
        }
    })

    err := group.Run()

    fmt.Println()
    fmt.Println("Done")
    fmt.Printf("Error: %v", err)
}

Output:

Done
Error: timeout

Answer 2

I did it like this: http://play.golang.org/p/eWv0fRlLEC

go func() {
    wg.Wait()
    c <- struct{}{}
}()
timeout := time.Duration(1) * time.Second
fmt.Printf("Wait for waitgroup (up to %s)\n", timeout)
select {
case <-c:
    fmt.Printf("Wait group finished\n")
case <-time.After(timeout):
    fmt.Printf("Timed out waiting for wait group\n")
}
fmt.Printf("Free at last\n")

It works fine, but is it the best way to do it?

Answer 3

Mostly your solution you posted below is as good as it can get. Couple of tips to improve it:

• Alternatively you may close the channel to signal completion instead of sending a value on it, a receive operation on a closed channel can always proceed immediately.
• And it's better to use defer statement to signal completion, it is executed even if a function terminates abruptly.
• Also if there is only one "job" to wait for, you can completely omit the WaitGroup and just send a value or close the channel when job is complete (the same channel you use in your select statement).
• Specifying 1 second duration is as simple as: timeout := time.Second. Specifying 2 seconds for example is: timeout := 2 * time.Second. You don't need the conversion, time.Second is already of type time.Duration, multiplying it with an untyped constant like 2 will also yield a value of type time.Duration.

I would also create a helper / utility function wrapping this functionality. Note that WaitGroup must be passed as a pointer else the copy will not get "notified" of the WaitGroup.Done() calls. Something like:

// waitTimeout waits for the waitgroup for the specified max timeout.
// Returns true if waiting timed out.
func waitTimeout(wg *sync.WaitGroup, timeout time.Duration) bool {
    c := make(chan struct{})
    go func() {
        defer close(c)
        wg.Wait()
    }()
    select {
    case <-c:
        return false // completed normally
    case <-time.After(timeout):
        return true // timed out
    }
}

Using it:

if waitTimeout(&wg, time.Second) {
    fmt.Println("Timed out waiting for wait group")
} else {
    fmt.Println("Wait group finished")
}

Try it on the Go Playground.

Answer 4

This is not an actual answer to this question but was the (much simpler) solution to my little problem when I had this question.

My 'workers' were doing http.Get() requests so I just set the timeout on the http client.

urls := []string{"http://1.jpg", "http://2.jpg"}
wg := &sync.WaitGroup{}
for _, url := range urls {
    wg.Add(1)
    go func(url string) {
        client := http.Client{
            Timeout: time.Duration(3 * time.Second), // only want very fast responses
        }
        resp, err := client.Get(url)
        //... check for errors
        //... do something with the image when there are no errors
        //...

        wg.Done()
    }(url)

}
wg.Wait()