TFS 503 Service Unavailable when retrieving files

Question

We have a TFS 2013 server which I use only for source control. I just got a new desktop PC with Windows 10 and Visual Studio 2017. I am able to connect successfully to TFS, and I can start to pull down code, but after the directory structure has been built locally and files start to come down, several are successful, then the rest fail with a 503 Service Unavailable, and the connection is lost, forcing me to reconnect.

I can connect again a few seconds later, and keep trying, and it happens again.

If I pull down files one at a time, it seems to be ok, but when I try to pull an entire project, it blows up.

This happens on multiple projects on the server.

In general, I see 503 errors in ASP.NET when I overload an application to the point where it crashes the application pool - I don't know if that's what's happening here, but if it is, I'm wondering if maybe VS is pulling down the code too fast, maybe with too many concurrent threads or something, that the older version of TFS can't handle, which crashes it.

If that's the case, is there anything I can do to change that on my machine? Or do you have any other ideas on what could cause this?

As of now, I don't have access to the server to grab the event logs, but I can put in a request if it's not something I can figure out and fix myself.

Answer 1

Last time I encountered this, I was behind a rate-limiting proxy. 503 is a common error code for temporary service interruptions such as rate limiting.

Excessive requests are delayed until their number exceeds the maximum burst size in which case the request is terminated with an error 503 (Service Temporarily Unavailable).

The IIS service that's hosting TFS can also be configured to have request rate limiting.

maxBandwidth Optional uint attribute.

Specifies the maximum network bandwidth, in bytes per second, that is used for a site. Use this setting to help prevent overloading the network with IIS activity.

maxConnections Optional uint attribute.

Specifies the maximum number of connections for a site. Use this setting to limit the number of simultaneous client connections.

Visual studio 2017 has optimized the way it downloads over multiple parallel threads (at least 8 if I remember correctly). This can trigger the rate limiting in some proxies. And lots of small code files can easily trigger a rate limit.

If it's possible to reach the TFS server directly, you could try to add it to the proxy exclusion list to rule out whether the proxy on your end is causing the interruption. But a HTTP server such as nginx could also be implemented as a security measure and may reverse-proxy on the server end, in which case the server admin must change the limits.