Given the following code, is it OK to do async/await
inside a Parallel.ForEach
?
eg.
Parallel.ForEach(names, name =>
{
// Do some stuff...
var foo = await GetStuffFrom3rdPartyAsync(name);
// Do some more stuff, with the foo.
});
or is there some gotcha's that I need to be made aware of?
EDIT: No idea if this compiles, btw. Just Pseduo-code .. thinking out loud.
A close alternative might be this:
Ideally, you shouldn't be using a blocking call like
Task.WaitAll
, if you can make the whole chain of methods callsasync
, "all the way down" on the current call stack:Furthermore, if you don't do any CPU-bound work inside
GetStuffFrom3rdPartyAsync
,Task.Run
may be redundant:As pointed out by @Sriram Sakthivel there are some problems with using
Parallel.ForEach
with asynchronous lambdas. Steven Toub'sForEachASync
can do the equivalent. He talks about it here, but here is the code:It uses the
Partitioner
class to create a load balancing partitioner(doco), and allows you to specify how many threads you want to run with thedop
parameter. to see the difference between it andParallel.ForEach
. Try the following code.if you run
GetStuffForEachAsync
the program waits for all work to finish. If you runGetStuffParallelForEach
, the lineFinished
will be printed before the work is finished.From the name, I'm assuming that
GetStuffFrom3rdPartyAsync
is I/O-bound. TheParallel
class is specifically for CPU-bound code.In the asynchronous world, you can start multiple tasks and then (asynchronously) wait for them all to complete using
Task.WhenAll
. Since you're starting with a sequence, it's probably easiest to project each element to an asynchronous operation, and then await all of those operations:No, It doesn't make sense to combine
async
withParalell.Foreach
.Consider the following example:
What output you will expect?
That's not what will happen. It will output :
Why? Because when
ForEach
hits firstawait
the method actually returns,Parallel.ForEach
doesn't know it is asynchronous and it ran to completion!. Code afterawait
runs as continuation on another thread not "Paralell processing thread"Stephen toub addressed this here