Memory management in memory intensive application

Question

If you are developing a memory intensive application in C++ on Windows, do you opt to write your own custom memory manager to allocate memory from virtual address space or do you allow CRT to take control and do the memory management for you ? I am especially concerned about the fragmentation caused by the allocation and deallocation of small objects on heap. Because of this, I think the process will run out of memory eventhough there is enough memory but is fragmented.

Answer 1

There used to be excellent third-party drop-in heap replacement library for VC++, but I don't remember the name any more. Our app got 30% speed-up when we started using it.

Edit: it's SmartHeap - thanks, ChrisW

Answer 2

It depends very much on your memory allocation patterns. From my personal experience there are generally one or two classes in a project that needs special considerations when it comes to memory management because they are used frequently in the part of the code where you spend lots of time. There might also be classes that in some particular context needs special treatment, but in other contexts can be used without bothering about it.

I often end up managing those kind of objects in a std::vector or something similar and explicit rather than overriding the allocation routines for the class. For many situations the heap is really overkill and the allocation patterns are so predictable that you don't need to allocate on the heap but in some much simpler structure that allocates larger pages from the heap that has less bookkeeping overhead than allocating every single instance on the heap.

These are some general things to think about:

First, small objects that's allocated and destroyed quickly should be put on the stack. The fastest allocation are the ones that are never done. Stack allocation is also done without any locking of a global heap which is good for multi threaded code. Allocating on the heap in c/c++ can be relatively expensive compared to GC languages like java so try to avoid it unless you need it.

If you do a lot of allocation you should be careful with threading performance. A classic pitfall is string classes that tends to do alot of allocation hidden to the user. If you do lots of string processing in multiple threads, they might end up fighting about a mutex in the heap code. For this purpose, taking control of the memory management can speed up things alot. Switching to another heap implementation is generally not the solution here since the heap will still be global and your threads will fight about it. I think google has a heap that should be faster in multithreaded environments though. Haven't tried it myself.

Answer 3

there's a solution used by some open source software like doxygen, the idea is to store some instances into file when you exceeds a specific amount of memory. And after get from file your data when you need them.