Parallels.ForEach Taking same Time as Foreach

All,

I am using the Parallels.ForEach as follows

private void fillEventDifferencesParallels(IProducerConsumerCollection<IEvent> events, Dictionary<string, IEvent> originalEvents)
    {
        Parallel.ForEach<IEvent>(events, evt =>
        {
            IEvent originalEventInfo = originalEvents[evt.EventID];
            evt.FillDifferences(originalEventInfo);
        });
    }

Ok, so the problem I'm having is I have a list of 28 of these (a test sample, this should be able to scale to 200+) and the FillDifferences method is quite time consuming (about 4s per call). So the Average time for this to run in a normal ForEach has been around 100-130s. When I run the same thing in Parallel, it takes the same amount of time and Spikes my CPU (Intel I5, 2 Core, 2 Threads per Core) causing the app to become sluggish while this query is running (this is running on a thread that was spawned by the GUI thread).

So my question is, what am I doing wrong that is causing this to take the same amount of time? I read that List wasn't thread safe so I rewrote this to use the IProducerConsumerCollection. Is there any other pitfalls that may be causing this?

The FillDifferences Method calls a static class that uses reflection to find out how many differences there are between the original and the modified object. The static object has no 'global' variables, just ones local to the methods being invoked.

Some of you wanted to see what the FillDifferences() method called. This is where it ends up ultimately:

  public  List<IDifferences> ShallowCompare(object orig, object changed, string currentName)
    {
        List<IDifferences> differences = new List<IDifferences>();
        foreach (MemberInfo m in orig.GetType().GetMembers())
        {
            List<IDifferences> temp = null;

            //Go through all MemberInfos until you find one that is a Property.
            if (m.MemberType == MemberTypes.Property)
            {
                PropertyInfo p = (PropertyInfo)m;
                string newCurrentName = "";
                if (currentName != null && currentName.Length > 0)
                {
                    newCurrentName = currentName + ".";
                }
                newCurrentName += p.Name;
                object propertyOrig = null;
                object propertyChanged = null;

                //Find the property Information from the orig object
                if (orig != null)
                {
                    propertyOrig = p.GetValue(orig, null);
                }

                //Find the property Information from the changed object
                if (changed != null)
                {
                    propertyChanged = p.GetValue(changed, null);
                }

                //Send the property to find the differences, if any. This is a SHALLOW compare.
                temp = objectComparator(p, propertyOrig, propertyChanged, true, newCurrentName);
            }
            if (temp != null && temp.Count > 0)
            {
                foreach (IDifferences difference in temp)
                {
                    addDifferenceToList(differences, difference);
                }
            }
        }
        return differences;
    }

回答1:

I believe you may be running into the cost of thread context switching. Since these tasks are long running I can imagine many threads are being created on the ThreadPool to handle them.

0ms == 1 thread
500ms == 2 threads
1000 ms == 3 threads
1500 ms == 4 threads
2000 ms == 5 threads
2500 ms == 6 threads
3000 ms == 7 threads
3500 ms == 8 threads
4000 ms == 9 threads

By 4000ms only the first task has been completed so this process will continue. A possible solution is as follows.

System.Threading.ThreadPool.SetMaxThreads(4, 4);

回答2:

Looking at what it's doing, the only time your threads aren't doing anything is when the OS switches them out to give another thread a go, so you've got the gain of being able to run on an other core - the cost of all the context switches.

You'd have to chuck some logging in to find out for definite, but I suspect the bottle neck is physical threads, unless you have one somewhere else you' haven't posted.

If that's true, I'd be tempted to rejig the code. Have two threads one for finding properties to compare, and one for comparing them and a common queue. May be another one to throw classes in the list and collate the results.

Could be me old time batch processing head though.