Memory management in memory intensive application

Question

If you are developing a memory intensive application in C++ on Windows, do you opt to write your own custom memory manager to allocate memory from virtual address space or do you allow CRT to take control and do the memory management for you ? I am especially concerned about the fragmentation caused by the allocation and deallocation of small objects on heap. Because of this, I think the process will run out of memory eventhough there is enough memory but is fragmented.

Answer 1

From my experience, fragmentation is mostly a problem when you are continuously allocating and freeing large buffers (like over 16k) since those are the ones that will ultimately cause an out of memory, if the heap cannot find a big enough spot for one of them.

In that case, only those objects should have special memory management, keep the rest simple. You can use buffer reusing if they always have the same size, or more complex memory pooling if they vary in size.

The default heap implementations shouldn't have any problem finding some place for smaller buffers between previous allocations.

Answer 2

Although most of you indicate that you shouldn't write your own memory manager, it could still be useful if:

• you have a specific requirement or situation in which you are sure you can write a faster version
• you want to write you own memory-overwrite logic (to help in debugging)
• you want to keep track of the places where memory is leaked

If you want to write your own memory manager, it's important to split it in the following 4 parts:

1. a part that 'intercepts' the calls to malloc/free (C) and new/delete (C++). This is quite easy for new/delete (just global new and delete operators), but also for malloc/free this is possible ('overwrite' the functions of the CRT, redefine calls to malloc/free, ...)
2. a part that represents the entry point of your memory manager, and which is called by the 'interceptor' part
3. a part that implements the actual memory manager. Possibly you will have multiple implementations of this (depending on the situation)
4. a part that 'decorates' the allocated memory with information of the call stack, overwrite-zones (aka red zones), ...

If these 4 parts are clearly separated, it also becomes easy to replace one part by another, or add a new part to it e.g.:

• add the memory manager implementation of Intel Tread Building Blocks library (to part 3)
• modify part 1 to support a new version of the compiler, a new platform or a totally new compiler

Having written a memory manager myself, I can only indicate that it can be really handy having an easy way to extend your own memory manager. E.g. what I regularly have to do is finding memory leaks in long-running server applications. With my own memory manager I do it like this:

• start the application and let it 'warm up' for a while
• ask your own memory manager to dump an overview of the used memory, including the call stacks at the moment of the call
• continue running the application
• make a second dump
• sort the two dumps alphabetically on call stack
• look up the differences

Although you can do similar things with out-of-the-box components, they tend to have some disadvantages:

• often they seriously slow down the application
• often they can only report leaks at the end of the application, not while the application is running

But, also try to be realistic: if you don't have a problem with memory fragmentation, performance, memory leaks or memory overwrites, there's no real reason to write your own memory manager.

Answer 3

no, I would not.

The chances of me writing a better code then the CRT with the who know how many hundreds of man year invested in it are slim.

I would search for a specialized library instead of reinventing the wheel.

Answer 4

Was it SmartHeap from MicroQuill?

Answer 5

you opt to write your own custom memory manager to allocate memory from virtual address space or do you allow CRT to take control and do the memory management for you?

The standard library is often good enough. If it isn't then, instead of replacing it, a smaller step is to override operator new and operator delete for specific classes, not for all classes.

Answer 6

I think your best bet is to not implement one until profiles prove that the CRT is fragmenting memory in a way that damages the performance of your application. CRT, core OS, and STL guys spend a lot of time thinking about memory management.

There's a good chance that your code will perform quite fine under existing allocators with no changes needed. There's certainly a better chance of that, than there is of you getting a memory allocator right the first time. I've written memory allocators before for similar circumstances and it's a monsterous task to take on. Not so suprisingly, the version I inherited was rife with fragmentation problems.

The other advantage of waiting until a profile shows it's a problem is that you will also know if you've actually fixed anything. That's the most important part of a performance fix.

As long as you're using standard collection classes an algorihtmns (such as STL/BOOST) it shouldn't be very hard to plug in a new allocator later on in the cycle to fix the portions of your code base that do need to be fixed. It's very unlikely that you will need a hand coded allocator for your entire program.