Web request fails, but only for localhost and when

I've got a very weird problem with System.Net.HttpWebRequest, that's driving me nuts: When I run it with a localhost address, then it is either extremely slow (around 30 sec) or, in most cases, it times out entirely. This happens only with requests that target localhost, and only from C# code.

Concretely, taking an ASP.NET Web API project as an example, when I have this code:

[Test]
public void TestRequest()
{
    var request = WebRequest.Create("http://localhost:64497/api/values");
    WebResponse response = request.GetResponse();
}

..it will most of the time fail with this exception:

System.Net.WebException : Unable to connect to the remote server
----> System.Net.Sockets.SocketException : A connection attempt failed because the connected party did not properly respond after a period of time or established connection failed because connected host has failed to respond 127.0.0.1:64497

By inspecting the .NET code, I tracked down the error to the WSAConnect API returning a non-null value, but I can't see that this is of much help. (However, this is a 32bit API, whereas I'm running on a 64bit system. Could there be some strange 32/64bit problem?)

When I run the above code with an address other than localhost (say http://www.google.com), then it behaves normally and I get response times around 1 sec. Also, when I submit the above request from a different client (I used Chrome's Advanced REST Client), then I get the normal behaviour: A delay time of about 5-8 sec for the first request due to initial compilation, and very short response times (20-40 msec) for all subsequent requests.

Given these findings, it seems quite obvious to me that the problem must come from something that's specific to the implementation of the System.Net.HttpWebRequest class...

The web application itself is hosted on IIS Express, but I'm quite sure that this is irrelevant, since the above request never reaches the web server (I checked that both by running IIS Express from the command line and then inspecting the requests coming through and also per ProcessMonitor).

Further information: I'm running Win 8.1 64bit and VS 2013 SP4 pro. The system's HOST file is unmodified, i.e. it does not contain any entries.

Things I've tried so far:

Running VS in admin mode
Switching off all firewall and antivirus software
using IIS instead of IIS Express
setting proxysettings explicitly to null, both per code and per configuration
inspecting the IIS Express process per ProcessMonitor - no result (as I said, I'm quite sure that the Web server is not directly involved in this problem)
changing localhost to 127.0.0.1

I don't understand what the problem is, and I can't find any relevant info on the web. I don't even have any more ideas what I could try to track down the error. In other words: I'm totally stuck on this, and it effectively prevents me from writing any integration tests against locally hosted webs. Any help/hint/tip is highly appreciated.

Just to check this part: "By inspecting the .NET code, I tracked down the error to the WSAConnect API returning a non-null value, but I can't see that this is of much help. (However, this is a 32bit API, whereas I'm running on a 64bit system. Could there be some strange 32/64bit problem?)"

Make sure that in IIS, for the application pool, that you have 32-bit enabled (default is disabled).

Also, I assume the binding is set correctly. (that last part [binding] was supposed to be a comment, but I didn't have the rep - it was deleted, apparently-or will be)