C# not releasing memory after task complete

Question

The following code is a simplified example of an issue I am seeing. This application consumes approx 4GB of memory before throwing an exception as the dictionary is too big.

 class Program
 {
    static void Main(string[] args)
    {
        Program program = new Program();

        while(true)
        {
            program.Method();
            Console.ReadLine();
        }
    }

    public void Method()
    {
        WasteOfMemory memory = new WasteOfMemory();
        Task tast = new Task(memory.WasteMemory);
        tast.Start();
    }


}

public class WasteOfMemory
{
     public void WasteMemory()
     {
         Dictionary<string, string> aMassiveList = new Dictionary<string, string>();

         try
         {
             long i = 0;
             while (true)
             {
                 aMassiveList.Add(i.ToString(), "I am a line of text designed to waste space.... I am exceptionally useful........");
                 i++;
             }

         }
         catch(Exception e)
         {
             Console.WriteLine("I have broken myself");
         }
     }
}

This is all as expected, although what we cannot currently work out is when this memory should be released from the CLR.

We have let the task complete and then simulated a memory overload situation, but the memory consumed by the dictionary is not released. As the OS is running out of memory, is it not putting pressure on the CLR to release the memory?

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

So, why is the memory not being released? And how can we get the CLR to release the memory?

Any explanations or solutions would be greatly appreciated.

EDIT: Following replies, particularly Beska's, it is obvious my description of the issue is not the the clearest, so I will try to clarify.

The code may not be the best example, sorry! It was a quick crude piece of code to try to replicate the issue.

The dictionary is used here to replicate the fact we have a large custom data object, which fills a large chunk of our memory and it is not then released after the task has completed.

In the example, the dictionary fills up to the limit of the dictionary and then throws an exception, it does NOT keep filling forever! This is well before our memory is full, and it does not cause an OutOfMemoryException. Hence the result is a large object in memory, and then the task completes.

At this point we would expect the dictionary to be out of scope, as both the task and the method 'Method' have completed. Hence, we would expect the dictionary to be garbage collected and the memory reclaimed. In reality, the memory is not freed until 'Method' is called again, creating a new WasteOfMemory instance and starting a new task.

Hopefully that will clarify the issue a bit

Answer 1

The memory is not being released because the scope aMassiveList is never finished. When a function returns, it releases all non-referenced resources created inside it.

In your case, aMassiveList never leaves context. If you want your function never to return you have to find a way to 'process' your info and release it instead of storing all of them forever.

If you create a function that increasingly allocates resources and never release it you will end up consuming all the memory.

Answer 2

Okay, I've been following this...I think there are a couple issues, some of which people have touched on, but I think not answering the real question (which, admittedly, took me a while to recognize, and I'm not sure I'm answering what you want even now.)

This is all as expected, although what we cannot currently work out is when this memory should be released from the CLR.

As others have said, while the task is running, the dictionary will not be released. It's being used. It gets bigger until you run out of memory. I'm pretty sure you understand this.

We have let the task complete and then simulated a memory overload situation, but the memory consumed by the dictionary is not released. As the OS is running out of memory, is it not putting pressure on the CLR to release the memory?

Here, I think, is the real question.

If I understand you correctly, you're saying you set this up to fill up memory. And then, after it crashes (but before you hit return to start a new task) you're trying other things outside of this program, such as running other programs in Windows to try to get the GC to collect the memory, right? Hoping that the OS would talk to the GC, and start pressuring it to do it's thing.

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

I think you answered your own question...it has not been necessarily been released until you hit return to start a new task. The new task needs memory, so it goes to the GC, and the GC happily collects the memory from the previous task, which has now ended (after throwing from full memory).

So, why is the memory not being released? And how can we get the CLR to release the memory?

I don't know that you can force the GC to release memory. Generally speaking, it does it when it wants (though some hacker types might know some slick way to force its hand.) Of course, .NET decides when to run the GC, and since nothing is happening while the program is just sitting there, it may well be deciding that it doesn't need to. As to whether the OS can pressure the GC to run, it seems from your tests the answer is "no". A bit counter-intuitive perhaps.

Is that what you were trying to get at?

Answer 3

Change your method to be a using statement

Example:

Using (WateOfMemory memory = new WateOfMemory())
{
    Task tast = new Task(memory.WasteMemory); 
    tast.Start();
 }

And add disposible WateOfMemoryClass (by the way your constructor is WasteOfMemory)

#region Dispose
    private IntPtr handle;
    private Component component = new Component();
    private bool disposed = false;

    public WateOfMemory()
    {

    }

    public WateOfMemory(IntPtr handle)
    {
        this.handle = handle;
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    private void Dispose(bool disposing)
    {
        if(!this.disposed)
        {
            if(disposing)
            {
            component.Dispose();
            }

            CloseHandle(handle);
            handle = IntPtr.Zero;            
        }
        disposed = true;         
    }

    [System.Runtime.InteropServices.DllImport("Kernel32")]
    private extern static Boolean CloseHandle(IntPtr handle);

    ~WateOfMemory()      
    {
        Dispose(false);
    }
    #endregion

Answer 4

GC will only release unreferenced objects, so as the dictionary is being referenced by your program it can't be released by the GC

Answer 5

The garbage collector only frees locations in memory that are no longer in use that are objects which have no pointer pointing to them.

(1) your program runs infinitely without termination and

(2) you never change the pointer to your dictionary, so the GC has certainly no reason to touch the dictionary.

So for me your program is doing exactly what it is supposed to do.

Answer 6

The way you've written the WasteMemory method, it will never exit (unless the variable "i" overflows, which won't happen this year) and BECAUSE IT WILL NEVER EXIT it will keep IN USE the reference to the internal Dictionary.

Daniel White is right, you should read about how GC works.

If the references are in use, GC will not collect the referenced memory. Otherwise, how would any program work?

I don't see what you expect the CLR/GC to do here. There's nothing to garbage-collect inside one run of your WasteMemory method.

However and even more confusing, if we wait until the task has completed, then hit enter to run the task again the memory is released, so obviously the previous dictionary has been garbage collected (hasn't it?).

When you press Enter, a new task is created and started. It's not the same task, it's a new task - a new object holding a reference to a new WasteOfMemory instance.

The old task will keep running and the memory it uses will NOT be collected because the old task keeps running in background and it keeps USING that memory.

I'm not sure why - and most importantly HOW - you observe the memory of the old task being released.