Async/Await and Caching

Question

My service layer is caching alot of Db requests to memcached, does this make it impossible to use Async/Await?? For example how could I await this?

public virtual Store GetStoreByUsername(string username)
{
        return _cacheManager.Get(string.Format("Cache_Key_{0}", username), () =>
        {
                return _storeRepository.GetSingle(x => x.UserName == username);
        });
}

Note: If the key exists in the cache it will return a "Store" (not a Task<Store>), if the key does not exist in the cache it will execute the lambda. If I change the Func to

return await _storeRepository.GetSingleAsync(x => x.UserName == username);

And the method signature to

public virtual async Task<Store> GetStoreByUsername(string username)

This will not work obviously because of the cache return type.

Answer 1

Here's a way to cache results of asynchronous operations that guarantees no cache misses and is thread-safe.

In the accepted answer, if the same username is requested many times in a loop or from multiple threads the DB request will keep getting sent until there's a response that gets cached, at which point the cache will start getting used.

The method below creates a SemaphoreSlim object for each unique key. This will prevent the long running async operation from running multiple times for the same key while allowing it to be running simultaneously for different keys. Obviously, there's overhead keeping SemaphoreSlim objects around to prevent cache misses so it may not be worth it depending on the use case. But if guaranteeing no cache misses is important then this accomplishes that.

private readonly ConcurrentDictionary<string, SemaphoreSlim> _keyLocks = new ConcurrentDictionary<string, SemaphoreSlim>();
private readonly ConcurrentDictionary<string, Store> _cache = new ConcurrentDictionary<string, Store>();

public async Task<Store> GetStoreByUsernameAsync(string username)
{   
    Store value;
    // get the semaphore specific to this username
    var keyLock = _keyLocks.GetOrAdd(username, x => new SemaphoreSlim(1));
    await keyLock.WaitAsync().ConfigureAwait(false);
    try
    {
        // try to get Store from cache
        if (!_cache.TryGetValue(username, out value))
        {
            // if value isn't cached, get it from the DB asynchronously
            value = await _storeRepository.GetSingleAsync(x => x.UserName == username).ConfigureAwait(false);

            // cache value
            _cache.TryAdd(username, value);
        }
    }
    finally
    {
        keyLock.Release();
    }
    return value;
}

Note: To further optimize this method, an additional cache check could be performed before the lock acquisition step.

Answer 2

It looks like the cache-manager does all the "check it exists, if not run the lambda then store". If so, the only way to make that async is to have a GetAsync method that returns a Task<Store> rather than a Store, i.e.

public virtual Task<Store> GetStoreByUsernameAsync(string username)
{
    return _cacheManager.GetAsync(string.Format("Cache_Key_{0}", username), () =>
    {
        return _storeRepository.GetSingleAsync(x => x.UserName == username);
    });
}

Note that this doesn't need to be marked async as we aren't using await. The cache-manager would then do something like:

public async Task<Store> GetAsync(string key, Func<Task<Store>> func)
{
    var val = cache.Get(key);
    if(val == null)
    {
        val = await func().ConfigureAwait(false);
        cache.Set(key, val);
    }
    return val;
}