SignalR giving 'The ConnectionId is in the inc

Question

I have a SignalR project that was working without issue in my development environments but once it was moved to production almost every call would fail with the error

The ConnectionId is in the incorrect format.

I saw a few other items on StackOverflow about the same error message but none of them had a solution that helped.

I am using SignalR 2.0.2 and SignalR Scaleout with SQL Server, effectively a load balancer for SignalR that uses SQL Server for coordination.

Event code: 3005 Event message: An unhandled exception has occurred. Event time: 2/11/2014 9:11:27 PM Event time (UTC): 2/12/2014 3:11:27 AM Event ID: cff1fd93fa6d4b83b1848078a905371c Event sequence: 73 Event occurrence: 10 Event detail code: 0 Application information:

Application domain: /LM/W3SVC/2/ROOT-3-130366479327251392 
Trust level: Full 
Application Virtual Path: / 
Application Path: C:\inetpub\wwwroot\<remvoved>
Machine name: BSDUSHC1WW09    Process information: 
Process ID: 7088 
Process name: w3wp.exe 
Account name: <remvoved>    Exception information: 
Exception type: InvalidOperationException 
Exception message: The ConnectionId is in the incorrect format.    at

Microsoft.AspNet.SignalR.PersistentConnection.GetConnectionId(HostContext context, String connectionToken) at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequest(HostContext context) at Microsoft.AspNet.SignalR.Hubs.HubDispatcher.ProcessRequest(HostContext context) at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequest(IDictionary`2 environment) at Microsoft.Owin.Mapping.MapMiddleware.d__0.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.EndFinalWork(IAsyncResult ar) at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Request information: Request URL: http:///signalr/connect?transport=serverSentEvents&connectionToken=GZlhDBCjkD1/bL1rc4Rlq2PVKYRs0B9nN7b71cU/E6x7sCsFvR1DqM/rBnDhg+URwkYyBlGmrczV59XIn/goyt9x0xXOd8Gs3Qswo1oXqSttH2QPO548C0fbdBvvlUupzS4S0Rl+aShoQwnj+qFDpA==&connectionData=[{"Name":""}]

Request path: /signalr/connect 
User host address: 10.240.14.26 
User: <remvoved>
Is authenticated: True 
Authentication Type: Negotiate 
Thread account name: <remvoved>    Thread information: 
Thread ID: 23 
Thread account name: <remvoved>
Is impersonating: False 
Stack trace:    at Microsoft.AspNet.SignalR.PersistentConnection.GetConnectionId(HostContext

context, String connectionToken) at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequest(HostContext context) at Microsoft.AspNet.SignalR.Hubs.HubDispatcher.ProcessRequest(HostContext context) at Microsoft.AspNet.SignalR.PersistentConnection.ProcessRequest(IDictionary`2 environment) at Microsoft.Owin.Mapping.MapMiddleware.d__0.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.EndFinalWork(IAsyncResult ar) at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Answer 1

Although Kevin already answered the question (which helped me and I upvoted), after configuring the same machine key for every server in the farm, the issue was still there (it strangely worked randomly sometimes, but the first try always failed).

You also need to make sure your application runs with the same credentials in each server. In my scenario, the Application Pool was configured to run under Network Service (which presents the own machine credentials over the network). Since each server has its own credentials, they are not the same, therefore the problem wasn't solved yet.

I had to create a user with access to every server in the farm, and make the application in every server to run with this same user. Only after doing this my problem was solved.

Answer 2

Just like when load balancing an ASP.NET website that uses Session, SignalR with SQL Server Scaleout requires that all servers which handle SignalR requests must share a machine key.

There is no obvious mention of this in the documentation for SignalR so I have to assume the connectionToken in the above event log message is encrypted using the machine key. When an application hits server 1 it is assigned a connectionToken generated using that machine's machine key. If the application then hits server 2 with that assigned connectionToken, machine 2 cannot decrypt the token unless is has a matching machine key.

There are countless resources online about how to set a machine key so I'll leave just the easiest one. Click on the website in IIS 8 (maybe 7 too) and open the Machine Key settings. Click Generate Keys in the actions pane on the right. Uncheck the 'Generate a unique key for each application' checkbox under both the Validation Key and Decryption Key sections.