Threading with Parallel.For Adding Lists

问题:

I am having trouble using the Parallel.For method. I am making a GET call to get back a list. Then I want to take that list and add it to the main list. I have tried addRange which is not thread safe, and will return the wrong data in the list. I have also tried to use ConcurrentBag which also does not get the right data. When I say does not get the right data I mean some of the data in the lists is either repeating or gets over written.

Here is my code(updated):

var thisLock = new Object();
var list = new List<Person>();
Parallel.For(1, 10, x =>
{
    request.Page = x;
    response = Get(request); // call to client
    lock(thisLock)
    {
        list.AddRange(response);
    }
}

Any other ideas besides addRange or ConcurrentBag

回答1:

I am making a few assumptions here, but it would appear that your problem is the fact that your request/response variables are not scoped within the Parallel.For call.

The problem is you make a (presumably) synchronous Get call which updates the response variable but given you have X threads all working with that very same response if that gets updated at any given point i.e. whilst another thread is adding it to the list, then that is going to very well lead to duplicate data.

Same goes for the request, you have an obvious race condition meaning that when one thread changes the request.Page and another thread is just about to pull the data then you are effectively pulling the same page across various threads.

The solution is simple, create your request/response objects locally

var list = new ConcurrentBag<T>();
Parallel.For(1, 10, x =>
{
    var request = // create new request;
    request.Page = x;
    var response = Get(request); // call to client
    foreach (var item in response) {
        list.Add(item); // add it main list
    }
}

回答2:

This is a good candidate for PLINQ. You can then use SelectMany for flattening the sequences.

var list = ParallelEnumerable.Range(1, 10).SelectMany(x =>
{
    var request = // create new request;
    request.Page = x;
    response = Get(request); // call to client
    return response;
}).ToList();