Limit of outgoing connections for one process (.Ne

Question

When I download a file in one thread, it took 0.1 second. But when I download the same file in 100 threads - it took 10 seconds for each downloading. Source code:

private static int _threadsCount;
private static string _url;

private static void Main(string[] args)
{
    _url = ConfigurationManager.AppSettings["Url"];

    int threadsLimit = 1;

    if (0 != args.Length)
        threadsLimit = int.Parse(args[0]);

    for (int i = 0; i < threadsLimit; i++)
    {
        var thread = new Thread(Start);
        thread.Start();
    }

    while (_threadsCount < threadsLimit)
    {
        Thread.Sleep(1000);
    }

    Console.WriteLine("Done");
}

static void Start()
{
    var webClient = new WebClient();

    var stopwatch = new Stopwatch();
    stopwatch.Reset();

    stopwatch.Start();

    for (int i = 1; i <= 10; i++)
    {
        webClient.DownloadData(_url);
    }

    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);

    Interlocked.Increment(ref _threadsCount);
}

Thus, if I run a program with 100 threads, speed 10 seconds per file. But if I run the second program at the same time with 1 thread, speed 0.1 second per file. So, problem is not in internet speed.

Why the download speed goes down with increasing number of threads, but it does not affect the other process (the same file)? How to increase the speed in one process?

Answer 1

1) You can adjust this parameter in your configuration file (default value is 2) :

<system.net>
    <connectionManagement>
        <add address="*" maxconnection="2" />
    </connectionManagement>
</system.net>

2) In order to force your program to create several sockets, download from different application domains.

Answer 2

1. There is only one network adapter. This could be your bottleneck
2. Are you downloading from the same server? This could be a bottleneck
3. Threads have a cost - switching between threads on a cpu (context switching) takes times and I doubt you have 100 cpus running.