Pre-load values for a Guava Cache

Question

I have a requirement where we are loading static data from a database for use in a Java application. Any caching mechanism should have the following functionality:

• Load all static data from the database (once loaded, this data will not change)
• Load new data from the database (data present in the database at start-up will not change but it is possible to add new data)

Lazy loading of all the data isn't an option as the application will be deployed to multiple geographical locations and will have to communicate with a single database. Lazy loading the data will make the first request for a specific element too slow where the application is in a different region to the database.

I have been using the MapMaker API in Guava with success but we are now upgrading to the latest release and I can't seem to find the same functionality in the CacheBuilder API; I can't seem to find a clean way of loading all data at start-up.

One way would be to load all keys from the database and load those through the Cache individually. This would work but would result in N+1 calls to the database, which isn't quite the efficient solution I'm looking for.

public void loadData(){
    List<String> keys = getAllKeys();
    for(String s : keys)
        cache.get(s);
}

Or the other solution is to use a ConcurrentHashMap implementation and handle all of the threads and missing entries myself? I'm not keen on doing this as the MapMaker and CacheBuilder APIs provide the key-based thread locking for free without having to provide extra testing. I'm also pretty sure the MapMaker/CacheBuilder implementations will have some efficiencies that I don't know about/haven't got time to investigate.

public Element get(String key){
    Lock lock = getObjectLock(key);
    lock.lock();
    try{
        Element ret = map.get(key)
        if(ret == null){
            ret = getElement(key); // database call
            map.put(key, e);
        }
        return ret;
    }finally {
        lock.unlock();
    } 
}

Can anyone think of a better solution to my two requirements?

---

Feature Request

I don't think pre-loading a cache is an uncommon requirement, so it would be nice if the CacheBuilder provided a configuration option to pre-load the cache. I think providing an Interface (much like CacheLoader) which will populate the cache at start-up would be an ideal solution, such as:

CacheBuilder.newBuilder().populate(new CachePopulator<String, Element>(){

    @Override
    public Map<String, Element> populate() throws Exception {
        return getAllElements();
    }

}).build(new CacheLoader<String, Element>(){

    @Override
    public Element load(String key) throws Exception {       
        return getElement(key);
    }

});

This implementation would allow the Cache to be pre-populated with all relevant Element objects, whilst keeping the underlying CustomConcurrentHashMap non-visible to the outside world.

Answer 1

I'd load all static data from the DB, and store it in the Cache using cache.asMap().put(key, value) ([Guava 10.0.1 allows write operations on the Cache.asMap() view][1]).

Of course, this static data might get evicted, if your cache is configured to evict entries...

The CachePopulator idea is interesting.

Answer 2

In the short-term I would just use Cache.asMap().putAll(Map<K, V>).

Once Guava 11.0 is released you can use Cache.getAll(Iterable<K>), which will issue a single bulk request for all absent elements.