Object cache for C#

Question

I'm doing a document viewer for some document format. To make it easier, let's say this is a PDF viewer, a Desktop application. One requirement for the software is the speed in rendering. So, right now, I'm caching the image for the next pages while the user is scrolling through the document.

This works, the UI is very responsive and it seems like the application is able to render the pages almost instantly....at a cost : the memory usage sometimes goes to 600MB. I cache it all in memory.

Now, I can cache to disk, I know, but doing that all the time is noticeably slower. What I would like to do is implement some cache (LRU?), where some of the cached pages (image objects) are on memory and most of them are on disk.

Before I embark on this, is there something in the framework or some library out there that will do this for me? It seems a pretty common enough problem. (This is a desktop application, not ASP.NET)

Alternatively, do you have other ideas for this problem?

Answer 1

For .NET 4.0, you can also use the MemoryCache from System.Runtime.Caching.

http://msdn.microsoft.com/en-us/library/system.runtime.caching.aspx

Answer 2

Caching application block and ASP.NET cache are both options however, although they do LRU, the only kind of disk utilization that happens is by memory paging. I think there are ways you can optimize this that are more specific to your goal to get a better output. Here are some thoughts:

• Since it's an ASP.NET app, why not generate the images, write them to disk, and when the browser requests the next page have IIS serve it up. That keeps your ASP.NET worker process lower while letting IIS do what it's good at.
• Use statistics about how a user interacts. LRU as implemented in the ASP.NET cache will typically apply to the individual image files - not much use for this scenario by the sounds of it. Instead, maybe some logic like: "If the user has scrolled X pages in the last Y seconds, then general the next X*Y images". Users scrolling quickly will get more pages generated; users reading slowly will need less cached.
• Have IIS serve images from disk, and use a custom HTTP handler for images you really want to control the caching of.
• Have the browser request the files ahead of time too, and rely on browser caching.
• Once an image has been served to the client, is it fair to say it can pretty much be removed from the cache? That could reduce the footprint substantially.

I'd certainly avoid using a plain hash table though.

Answer 3

There's patterns & practices Enterprise Library (more specifically, Caching Application Block), but it IMO tends to be over-engineered and overly complex.

Answer 4

To addition to  rsbarro's answer to use MemoryCache I recommend to use PostSharp AOP as described in 

http://www.codeproject.com/Articles/493971/Leveraging-MemoryCache-and-AOP-for-expensive-calls

Answer 5

A classic trade-off situation. Keeping everything in memory will be fast at the cost of massively increased memory consumption, whilst retrieving from disc decreases memory consumption, but isn't as performant. However, you already know all this!

The built-in System.Web.Caching.Cache class is great, and I've used it to good effect many times myself in my ASP.NET applications (although mostly for database record caching), however, the drawback is that the cache will only run on one machine (typically a sole web server) and cannot be distributed across multiple machines.

If it's possible to "throw some hardware" at the problem, and it doesn't necessarily need to be expensive hardware, just boxes with plenty of memory, you could always go with a distributed caching solution. This will give you much more memory to play with whilst retaining (nearly) the same level of performance.

Some options for a distributed caching solution for .NET are:

Memcached.NET

indeXus.Net

or even Microsoft's own Velocity project.

Answer 6

There is an efficient, open sourced RAM virtualizer that uses MRU algorithm to keep freshest referenced objects in-memory and uses a fast, lightweight backing store (on Disk) for "paging".

Here is link in Code Project for a mini-article about it: http://www.codeproject.com/Tips/827339/Virtual-Cache

I hope you find it useful.