Multi-threading with Linq to SQL

Question

I am building an application which requires me to make use of DataContext's inside threads. My application keeps throwing InvalidOperationException's similar to:

There is already an open DataReader associated with this Command which must be closed first

ExecuteReader requires an open and available Connection. The connection's current state is connecting

These exceptions are intermittant.

Here is a snippet of my code:

var repo = new Repository();
var entities = repo.GetAllEntities();
foreach (var entity in entities)
{
    ThreadPool.QueueUserWorkItem(
        delegate
        {
            try
            {
                 ProcessEntity(entity);
            }
            catch (Exception)
            {
                throw;
            }
        });
}

I think it may have something to with passing an entity to a thread from the main thread as the error seems to throw as soon as I try and access a property of entity.

Anyone have any idea's why this is happening and how I can resolve it?

Update

Here is what I decided to go with:

var events = new Dictionary<int, AutoResetEvent>();
var repo = new Repository();
var entities = repo.GetAllEntities();
foreach (var entity in entities)
{
    events.Add(entity.ID, new AutoResetEvent(false));
    ThreadPool.QueueUserWorkItem(
        delegate
        {
            var repo = new Repository();
            try
            {
                ProcessHierarchy(repo.GetEntity(entity.ID), ReportRange.Weekly);
            }
            catch (Exception)
            {
                throw;
            }
            finally
            {
                events[entity.ID].Set();
            }
        });
}

WaitHandle.WaitAll(events.Values.ToArray());

Improvements/Suggestions welcome, but this seems to have done the trick.

Answer 1

The solution for me was LightSpeed Persistence framework, which is free until 8 entities. Per thread create the unitwork.

http://www.mindscapehq.com/products/LightSpeed/default.aspx

Answer 2

Try creating the Repository inside the new thread instead of passing it in.

Answer 3

Be aware that a SqlConnection instance is NOT thread safe. Whether you still have an open reader or not. In general, the access of a SqlConnection instance from multiple threads will cause you unpredictable intermittent problems.

See: http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.aspx

Answer 4

The exception is thrown since some properties of entity executes new query while a previous reader has not been closed yet. You cannot execute more than one query on the data context at the same time.

As a workaround you can "visit" the properties you access in ProcessEntity() and make the SQL run prior to the thread.

For example:

var repo = new Repository();
var entities = repo.GetAllEntities();
foreach (var entity in entities)
{
    var localEntity = entity; // Verify the callback uses the correct instance
    var entityCustomers = localEntity.Customers;
    var entityOrders = localEntity.Orders;
    var invoices = entityOrders.Invoices;
    ThreadPool.QueueUserWorkItem(
        delegate
        {
            try
            {
                 ProcessEntity(localEntity);
            }
            catch (Exception)
            {
                throw;
            }
        });
}

This workaround will execute the SQL only on the main thread and the processing will be done in other threads. You loose here some of the efficiency since all the queries are done in the single thread. This solution is good if you have a lot of logic in ProcessEntity() and the queries are not very heavy.