How to test unlikely concurrent scenarios?

Question

For example, map access like this:

func (pool *fPool) fetch(url string) *ResultPromise {
    pool.cacheLock.RLock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.RUnlock()
        return rp
    }
    pool.cacheLock.RUnlock()
    pool.cacheLock.Lock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.Unlock()
        // Skip adding url if someone snuck it in between RUnlock an Lock
        return rp
    }
    rp := newPromise()
    pool.cache[url] = rp
    pool.cacheLock.Unlock()
    pool.c <- fetchWork{rp, url}
    return rp
}

Here, the contents of the second if condition are not covered. However, by placing breakpoints it's trivial to end up in that block.

The example isn't contrived, because:

1. If we skip the RLock, the map will be unnecessarily locked when the workload is mostly reads.
2. If we skip the second if,the most expensive work (handled by pool.c <- fetchWork{rp, url} in this case) can happen more than once for the same key, which is unacceptable.

Answer 1

I. Mocking pool.cacheLock.Lock()

One way to cover that branch would be to mock pool.cacheLock.Lock(), and the mocked version could insert the url into the map. So checking again after this call, it would be found and execution would enter the body of the 2nd if statement.

Mocking by using interface

One way to mock pool.cacheLock.Lock() would be to make pool.cacheLock an interface, and in tests you can set a mocked value whose Lock() method will do the "dirty insert" into the map.

Here's a simplified version of your code that uses an interface for pool.cacheLock:

type rwmutex interface {
    Lock()
    RLock()
    RUnlock()
    Unlock()
}

type fPool struct {
    cache     map[string]string
    cacheLock rwmutex
}

func (pool *fPool) fetch(url string) string {
    pool.cacheLock.RLock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.RUnlock()
        return rp
    }
    pool.cacheLock.RUnlock()
    pool.cacheLock.Lock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.Unlock()
        // Skip adding url if someone snuck it in between RUnlock an Lock
        return rp
    }
    rp := url + "~data"
    pool.cache[url] = rp
    pool.cacheLock.Unlock()
    return rp
}

Its normal usage would be:

pool := fPool{
    cache:     map[string]string{},
    cacheLock: &sync.RWMutex{},
}
fmt.Println(pool.fetch("http://google.com"))

And a test case that will trigger the body of the 2nd if:

type testRwmutex struct {
    sync.RWMutex // Embed RWMutex so we don't have to implement everything
    customLock   func()
}

func (trw *testRwmutex) Lock() {
    trw.RWMutex.Lock()
    if trw.customLock != nil {
        trw.customLock()
    }
}

func TestFPoolFetch(t *testing.T) {
    trw := &testRwmutex{RWMutex: sync.RWMutex{}}
    pool := &fPool{
        cache:     map[string]string{},
        cacheLock: trw,
    }

    exp := "http://google.com~test"
    trw.customLock = func() {
        pool.cache["http://google.com"] = exp
    }

    if got := pool.fetch("http://google.com"); got != exp {
        t.Errorf("Expected: %s, got: %s", exp, got)
    }
}

Mocking by using a function field

Another way to mock pool.cacheLock.Lock() would be to "outsource" this functionality to a field of function type, which tests can replace to a function which–besides calling this–also does the "dirty insert".

Again your simplified example:

func NewFPool() *fPool {
    mux := &sync.RWMutex{}
    return &fPool{
        cache:     map[string]string{},
        cacheLock: mux,
        lock:      mux.Lock,
    }
}

type fPool struct {
    cache     map[string]string
    cacheLock *sync.RWMutex
    lock      func()
}

func (pool *fPool) fetch(url string) string {
    pool.cacheLock.RLock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.RUnlock()
        return rp
    }
    pool.cacheLock.RUnlock()
    pool.lock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.Unlock()
        // Skip adding url if someone snuck it in between RUnlock an Lock
        return rp
    }
    rp := url + "~data"
    pool.cache[url] = rp
    pool.cacheLock.Unlock()
    return rp
}

Normal usage would be:

pool := NewFPool()
fmt.Println(pool.fetch("http://google.com"))

And a test case that will trigger the body of the 2nd if:

func TestFPoolFetch(t *testing.T) {
    pool := NewFPool()
    oldLock := pool.lock

    exp := "http://google.com~test"
    pool.lock = func() {
        oldLock()
        pool.cache["http://google.com"] = exp
    }

    if got := pool.fetch("http://google.com"); got != exp {
        t.Errorf("Expected: %s, got: %s", exp, got)
    }
}

II. Using a simple test flag

The idea here is that to support easy testing you build a simple test flag into the implementation of fPool (e.g. it can be a field of fPool), and the code you want to test deliberately checks for this flag:

type fPool struct {
    cache     map[string]string
    cacheLock *sync.RWMutex
    test      bool
}

func (pool *fPool) fetch(url string) string {
    pool.cacheLock.RLock()
    if rp, pres := pool.cache[url]; pres {
        pool.cacheLock.RUnlock()
        return rp
    }
    pool.cacheLock.RUnlock()
    pool.cacheLock.Lock()
    if rp, pres := pool.cache[url]; pres || pool.test {
        pool.cacheLock.Unlock()
        // Skip adding url if someone snuck it in between RUnlock an Lock
        return rp
    }
    rp := url + "~data"
    pool.cache[url] = rp
    pool.cacheLock.Unlock()
    return rp
}

Now if you want to test the body of the 2nd if, all you gotta do is:

func TestFPoolFetch(t *testing.T) {
    pool := NewFPool()
    pool.test = true

    exp := ""
    if got := pool.fetch("http://google.com"); got != exp {
        t.Errorf("Expected: %s, got: %s", exp, got)
    }
}