System.OutOfMemoryException' was thrown - WebC

Question

I'm writing a poor mans load tester and I thought I was managing my resources correctly (thread pool) but when I run the following code I get an OutOfMemoryException on my call to WebClient.DownloadStringAsynch.

Using .net4.0 but could move to 4.5.

ASKs:

• What is the fix?
• How could I use HttpWebRequest and send that asynch as an alternative to webclient?

• What about using .net 4.5 using await (any difference with how .net4 manages threads with asynch calls?

  static void Main(string[] args)
  {
      System.Net.ServicePointManager.DefaultConnectionLimit = 200;
      while (true)
      {
          for (int i = 0; i < 100; i++)
          {
              Task.Factory.StartNew(LoadTestAsynchNET40);
          }
          Console.WriteLine(".........................sleeping...............................");
          Thread.Sleep(2);
      }
  }
  
  static void LoadTestAsynchNET40()
  {
    string url = "http://mysrv.com/api/dev/getthis?stuff=thestuff" + "&_=" + DateTime.Now.Ticks;  // <--- somtimes throws here...
      using (var client = new WebClient())
      {
          DateTime dt1 = DateTime.Now;
          client.Headers["Accept"] = "text/xml";
          client.DownloadStringCompleted += DownloadStringCompleted;
          Console.WriteLine(DateTime.Now.ToString("ss:fff") + ", Sent Ad Request...");
          client.DownloadStringAsync(new Uri(url), dt1); //<---throws here...
      }
  }
  
  static void DownloadStringCompleted(object sender, DownloadStringCompletedEventArgs e)
  {
      Console.WriteLine("Received reponse...");
  
  }
  

Answer 1

DownloadStringAsync will create a single giant string containing the entire response.
If you call that for lots of big responses, you will run out of memory.

Instead, you should use HttpWebRequest directly.
Its GetResponse() (or BeginGetResponse()) method gives you a stream that allows you to read the response directly from the server without buffering it in memory.

If you still want asyncrony, you should move the .Net 4.5, which adds the easier-to-use GetResponseAsync() method (as opposed to the old APM-based BeginGetResponse())