Parallel tasks with a long pause

Question

I have a function which is along the lines of

   private void DoSomethingToFeed(IFeed feed)
   {
      feed.SendData(); // Send data to remote server
      Thread.Sleep(1000 * 60 * 5); // Sleep 5 minutes
      feed.GetResults(); // Get data from remote server after it's processed it
   }

I want to parallelize this, since I have lots of feeds that are all independent of each other. Based on this answer, leaving the Thread.Sleep() in there is not a good idea. I also want to wait after all the threads have spun up, until they've all had a chance to get their results.

What's the best way to handle a scenario like this?

Edit, because I accidentally left it out: I had originally considered calling this function as Parallel.ForEach(feeds, DoSomethingToFeed), but I was wondering if there was a better way to handle the sleeping when I found the answer I linked to.

Answer 1

check WaitHandle for waiting on tasks.

Answer 2

Unless you have an awful lot of threads, you can keep it simple. Create all the threads. You'll get some thread creation overhead, but since the threads are basically sleeping the whole time, you won't get too much context switching.

It'll be easier to code than any other solution (unless you're using C# 5). So start with that, and improve it only if you actually see a performance problem.

Answer 3

I think you should take a look at the Task class in .NET. It is a nice abstraction on top of more low level threading / thread pool management.

In order to wait for all tasks to complete, you can use Task.WaitAll.

An example use of Tasks could look like:

IFeed feedOne = new SomeFeed();
IFeed feedTwo = new SomeFeed();

var t1 = Task.Factory.StartNew(() => { feedOne.SendData(); });

var t2 = Task.Factory.StartNew(() => { feedTwo.SendData(); });

// Waits for all provided tasks to finish execution
Task.WaitAll(t1, t2);

However, another solution would be using Parallel.ForEach which handles all Task creation for you and does the appropriate batching of tasks as well. A good comparison of the two approaches is given here - where it, among other good points is stated that:

Parallel.ForEach, internally, uses a Partitioner to distribute your collection into work items. It will not do one task per item, but rather batch this to lower the overhead involved.

Answer 4

private void DoSomethingToFeed(IFeed feed)
{
    Task.Factory.StartNew(() => feed.SendData())
        .ContinueWith(_ => Delay(1000 * 60 * 5)
                          .ContinueWith(__ => feed.GetResults())
                     );
}

//http://stevenhollidge.blogspot.com/2012/06/async-taskdelay.html
Task Delay(int milliseconds)      
{
    var tcs = new TaskCompletionSource<object>();
    new System.Threading.Timer(_ => tcs.SetResult(null)).Change(milliseconds, -1);
    return tcs.Task;
}