alternative to memcached that can persist to disk

Question

I am currently using memcached with my java app, and overall it's working great.

The features of memcached that are most important to me are:

• it's fast, since reads and writes are in-memory and don't touch the disk
• it's just a key/value store (since that's all my app needs)
• it's distributed
• it uses memory efficiently by having each object live on exactly one server
• it doesn't assume that the objects are from a database (since my objects are not database objects)

However, there is one thing that I'd like to do that memcached can't do. I want to periodically (perhaps once per day) save the cache contents to disk. And I want to be able to restore the cache from the saved disk image.

The disk save does not need to be very complex. If a new key/value is added while the save is taking place, I don't care if it's included in the save or not. And if an existing key/value is modified while the save is taking place, the saved value should be either the old value or the new value, but I don't care which one.

Can anyone recommend another caching solution (either free or commercial) that has all (or a significant percentage) of the memcached features that are important to me, and also allows the ability to save and restore the entire cache from disk?

Answer 1

What about Terracotta?

Answer 2

You can use GigaSpaces XAP which is a mature commercial product which answers your requirements and more. It is the fastest distributed in-memory data grid (cache++), it is fully distributed, and supports multiple styles of persistence methods.

Guy Nirpaz, GigaSpaces

Answer 3

Have you looked at BerkeleyDB?

• Fast, embedded, in-process data management.
• Key/value store, non-relational.
• Persistent storage.
• Free, open-source.

However, it fails to meet one of your criteria:

• BDB supports distributed replication, but the data is not partitioned. Each node stores the full data set.

Answer 4

EhCache has a "disk persistent" mode which dumps the cache contents to disk on shutdown, and will reinstate the data when started back up again. As for your other requirements, when running in distributed mode it replicates the data across all nodes, rather than storing them on just one. other than that, it should fit your needs nicely. It's also still under active development, which many other java caching frameworks are not.

Answer 5

Try go-memcached - memcache server written in Go. It persists cached data to disk out of the box. Go-memcached is compatible with memcache clients. It has the following features missing in the original memcached:

• Cached data survive server crashes and/or restarts.
• Cache size may exceed available RAM size by multiple orders of magnitude.
• There is no 250 byte limit on key size.
• There is no 1Mb limit on value size. Value size is actually limited by 2Gb.
• It is faster than the original memcached. It also uses less CPU when serving incoming requests.

Here are performance numbers obtained via go-memcached-bench:

-----------------------------------------------------
|            |  go-memcached   | original memcached |
|            |      v1         |      v1.4.13       |
| workerMode ----------------------------------------
|            | Kqps | cpu time |  Kqps  | cpu time  |
|----------------------------------------------------
| GetMiss    | 648  |    17    |  468   |   33      |
| GetHit     | 195  |    16    |  180   |   17      |
| Set        | 204  |    14    |  182   |   25      |
| GetSetRand | 164  |    16    |  157   |   20      |
-----------------------------------------------------

Statically linked binaries for go-memcached and go-memcached-bench are available at downloads page.

Answer 6

I think membase is what you want.