I've been getting two exceptions at random times in my asp.net mvc code running on iis7:
Exception type: InvalidOperationException
Exception message: Collection was modified; enumeration operation may not execute.
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List'1.Enumerator.MoveNextRare()
at System.Collections.Generic.List'1.Enumerator.MoveNext()
at System.Web.Routing.RouteCollection.GetRouteData(HttpContextBase httpContext)
at System.Web.Routing.UrlRoutingModule.PostResolveRequestCache(HttpContextBase context)
at System.Web.Routing.UrlRoutingModule.OnApplicationPostResolveRequestCache(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
and
Exception type: NullReferenceException
Exception message: Object reference not set to an instance of an object.
at System.Web.Routing.RouteCollection.GetRouteData(HttpContextBase httpContext)
at System.Web.Routing.UrlRoutingModule.PostResolveRequestCache(HttpContextBase context)
at System.Web.Routing.UrlRoutingModule.OnApplicationPostResolveRequestCache(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
It's not consistently reproducible, but I assume it's something changing (or corrupting) RouteTable.Routes
. The only place I access RouteTable.Routes
in my project is in Global.asax.cs
and I know that the code there is only being called once, so it's not the problem. Any idea on how to track it down?
In my case, it ended up being a HttpModule: Ext.Direct.Mvc (Ext.Direct for ASP.NET MVC). This module had a bug (Fixed in version 0.8.0) which registered routes again every time Init() was called for the IHttpModule. (which might be called multiple times). If the timing was right, it would corrupt the RouteTable.Routes
collection, and cause one of the two exceptions above.
That error is consistent with a collection not being thread-safe in .Net.
As per the MSDN article on RouteCollection.GetWriteLock(), RouteTable.Routes
is not thread-safe. This issue is because some piece of code tried to modify the RouteTable.Routes
in a manner which was not thread-safe.
The best practice is to modify the RouteTable.Routes
in the Application_Start
. If you have to modify it in a way which can be accessed by multiple threads at the same time, make sure you make it in a thread-safe way. Check the example from MSDN, copied below for ease of reference:
using (RouteTable.Routes.GetWriteLock())
{
Route newRoute = new Route("{action}/{id}", new ReportRouteHandler());
RouteTable.Routes.Add(newRoute);
}
Same applies for when reading from the RouteTable - RouteCollection.GetReadLock().
Probably, if this was fixed by removing the HttpModule, it is because this HttpModule does not implement such locking mechanisms.
Other answers explained WHAT happens but provided little details on how to actually track it down. It may not be your own code causing all this trouble, so Ctrl-F is not enough..
The idea
What makes solving it difficult, is the fact that you usually get errors on a different requests than the ones actually modifying RouteCollection. A generally safe call to UrlHelper.GenerateUrl()
or the like would fail at seemingly random times. To track it down, I chose to move exceptions from victims to the culprit by forbidding changing routes after the initial set up.
Step 1: Implement custom RouteCollection
IF a flag is set then yell hard when someone tries to change routes. It's a crude implementation but you get the idea.
public class RestrictedRouteCollection : RouteCollection
{
public Boolean EnableRestrictions { get; set; }
protected override void InsertItem(Int32 index, RouteBase item)
{
if (EnableRestrictions) { throw new Exception("Unexpected route added".); }
base.InsertItem(index, item);
}
protected override void SetItem(Int32 index, RouteBase item)
{
if (EnableRestrictions) { throw new Exception("Unexpected change of RouteCollection item."); }
base.SetItem(index, item);
}
protected override void RemoveItem(Int32 index)
{
if (EnableRestrictions) { throw new Exception("Unexpected removal from RouteCollection, index: " + index); }
base.RemoveItem(index);
}
protected override void ClearItems()
{
if (EnableRestrictions) { throw new Exception("Unexpected clearing of routecollection."); }
base.ClearItems();
}
}
Step 2: Replace the default collection
In Application_start in Global.asax replace the default RouteCollection with your instance before the routes are set up. As there does not seem to be an API to replace it then we force it with reflection on the private fields:
var routeTable = new LockableRouteCollection();
var field = typeof(RouteTable)
.GetField("_instance", BindingFlags.Static | BindingFlags.NonPublic);
if (field == null) { throw new Exception("Expected field _instance was not found."); }
field.SetValue(null, routeTable);
Debug.Assert(RouteTable.Routes == routeTable);
Step3: Restrict changes once intended routes are set up
RouteConfig.RegisterRoutes(routeTable);
routeTable.EnableRestrictions = true;
All done! Now watch the bad request light up in exception logs.
A possible culprit: ImageResizer
In my case it was a 3-rd party component named ImageResizer
(v4.0.4) whose internal MvcRoutingShimPlugin
took the liberty of adding/removing a route without locking. This specific bug was already reported and fixed (though at this time not yet officially released).