Limit number of tasks with TPL

问题:

I'm looking into creating tasks which need to be executed on multiple threads. I could have a large number of tasks being created i.e. 2000 for example. I want to limit the number of tasks queued and executed simultaneously. Is there a way to create a certain number of tasks and then create new ones as they complete? Trying to work out if the task scheduler helps with this.

EDIT:

To phrase this a different way...is there a reason I should want to limit the number of tasks created/queued/executed simultaneously given that I could have a really large number e.g. 2000. Does the task scheduler optimally schedule tasks, can't seem to find any info on how it actually works...

EDIT:

I'm not using Parallel.Foreach. I've decided to use counters and based on a max number of counters, and the current number of tasks, create tasks or wait until the max number is not exceeded.

回答1:

If you want to send 2000 emails, you don't want to block the current thread and you want to use only a limited number of threads, then I think you should use a single Task that calls Parallel.ForEach(). Something like:

List<Email> emails = …;

var task = Task.Factory.StartNew(() =>
{
    Parallel.ForEach(
        emails,
        new ParallelOptions { MaxDegreeOfParallelism = maxNumberOfThreads },
        email => email.Send());
});

// handle the completion of task here

If you don't know the best number of threads, then you will have to find it out experimentally. TPL is able to guess the best number of threads for CPU-bound computations reasonably well. But your code is not CPU-bound, it's network-bound, which means the optimal number of threads has nothing to do with the number of cores your CPU has, it depends on the bandwidth and latency of your network connection (and possibly also of your mail server).

回答2:

Replying to question in comment:

"I don't know explicitly how many cores..."

You can get the number of cores with Environment.ProcessorCount and use them to set MaxDegreeOfParallelism:

ParallelOptions po = new ParallelOptions
{ 
    MaxDegreeOfParallelism = Environment.ProcessorCount
};  

Yes, one might frequently want to limit or tune the default spawning of new threads provided by parallel tasks (.NET thread pool). There are a lot of such discussions:

• Parallel.ForEach keeps spawning new threads
• Parallel.Foreach spawning way too many threads
• etc.