Nesting await in Parallel.ForEach
async-await, c#, parallel.foreach, task-parallel-library, wcf
Solution
The whole idea behind `Parallel.ForEach()` is that you have a set of threads and each thread processes part of the collection. As you noticed, this doesn't work with `async`-`await`, where you want to release the thread for the duration of the async call.
You could “fix” that by blocking the `ForEach()` threads, but that defeats the whole point of `async`-`await`.
What you could do is to use TPL Dataflow instead of `Parallel.ForEach()`, which supports asynchronous `Task`s well.
Specifically, your code could be written using a `TransformBlock` that transforms each id into a `Customer` using the `async` lambda. This block can be configured to execute in parallel. You would link that block to an `ActionBlock` that writes each `Customer` to the console. After you set up the block network, you can `Post()` each id to the `TransformBlock`.
In code:
var ids = new List<string> { "1", "2", "3", "4", "5", "6", "7", "8", "9", "10" };
var getCustomerBlock = new TransformBlock<string, Customer>(
async i =>
{
ICustomerRepo repo = new CustomerRepo();
return await repo.GetCustomer(i);
}, new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded
});
var writeCustomerBlock = new ActionBlock<Customer>(c => Console.WriteLine(c.ID));
getCustomerBlock.LinkTo(
writeCustomerBlock, new DataflowLinkOptions
{
PropagateCompletion = true
});
foreach (var id in ids)
getCustomerBlock.Post(id);
getCustomerBlock.Complete();
writeCustomerBlock.Completion.Wait();
Although you probably want to limit the parallelism of the `TransformBlock` to some small constant. Also, you could limit the capacity of the `TransformBlock` and add the items to it asynchronously using `SendAsync()`, for example if the collection is too big.
As an added benefit when compared to your code (if it worked) is that the writing will start as soon as a single item is finished, and not wait until all of the processing is finished.
Problem
In a metro app, I need to execute a number of WCF calls. There are a significant number of calls to be made, so I need to do them in a parallel loop. The problem is that the parallel loop exits before the WCF calls are all complete. How would you refactor this to work as expected? ``` var ids = new List<string>() { "1", "2", "3", "4", "5", "6", "7", "8", "9", "10" }; var customers = new System.Collections.Concurrent.BlockingCollection<Customer>(); Parallel.ForEach(ids, async i => { ICustomerRepo repo = new CustomerRepo(); var cust = await repo.GetCustomer(i); customers.Add(cust); }); foreach ( var customer in customers ) { Console.WriteLine(customer.ID); } Console.ReadKey(); ```