Is a non-blocking, single-threaded, asynchronous w

Question

I was looking at this question, looking for a way to create a single-threaded, event-based nonblocking asynchronous web server in .NET.

This answer looked promising at first, by claiming that the body of the code runs in a single thread.

However, I tested this in C#:

using System;
using System.IO;
using System.Threading;

class Program
{
    static void Main()
    {
        Console.WriteLine(Thread.CurrentThread.ManagedThreadId);

        var sc = new SynchronizationContext();
        SynchronizationContext.SetSynchronizationContext(sc);
        {
            var path = Environment.ExpandEnvironmentVariables(
                @"%SystemRoot%\Notepad.exe");
            var fs = new FileStream(path, FileMode.Open,
                FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true);
            var bytes = new byte[1024];
            fs.BeginRead(bytes, 0, bytes.Length, ar =>
            {
                sc.Post(dummy =>
                {
                    var res = fs.EndRead(ar);

                    // Are we in the same thread?
                    Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
                }, null);
            }, null);
        }
        Thread.Sleep(100);
    }
}

And the result was:

1
5

So it seems like, contrary to the answer, the thread initiating the read and the thread ending the read are not the same.

So now my question is, how do you to achieve a single-threaded, event-based nonblocking asynchronous web server in .NET?

Answer 1

What you need is a "message loop" which takes the next task on a queue and executes it. Additionally, every task needs to be coded so that it completes as much work as possible without blocking, and then enqueues additional tasks to pick up a task that needs time later. There is nothing magical about this: never using a blocking call and never spawn additional threads.

For example, when processing an HTTP GET, the server can read as much data as is currently available on the socket. If this is not enough data to handle the request, then enqueue a new task to read from the socket again in the future. In the case of a FileStream, you want to set the ReadTimeout on the instance to a low value and be prepared to read fewer bytes than the entire file.

C# 5 actually makes this pattern much more trivial. Many people think that the async functionality implies multithreading, but that is not the case. Using async, you can essentially get the task queue I mentioned earlier without ever explicility managing it.

Answer 2

Here is one more implementation of the event-loop web server called SingleSand. It executes all custom logic inside single-threaded event loop but the web server is hosted in asp.net. Answering the question, it is generally not possible to run a pure single threaded app because of .NET multi-threaded nature. There are some activities that run in separate threads and developer cannot change their behavior.

Answer 3

i am wondering nobody mentioned kayak it's basicly C#s answer to Pythons twisted, JavaScripts node.js or Rubys eventmachine

Answer 4

LibuvSharp is a wrapper for libuv, which is used in the node.js project for async IO. BUt it only contains only low level TCP/UDP/Pipe/Timer functionality. And it will stay like that, writing a webserver on top of it is an entire different story. It doesn't even support dns resolving, since this is just a protocol on top of udp.

Answer 5

Some kind of the support from operating system is essential here. For example, Mono uses epoll on Linux with asynchronous I/O, so it should scale really well (still thread pool). If you are looking and performance and scalability, definitely try it.

On the other hand, the example of C# (with native libs) webserver which is based around idea you have mentioned can be Manos de Mono. Project has not been active lately; however, idea and code is generally available. Read this (especially the "A closer look at Manos" part).

Edit:

If you just want to have callback fired on your main thread, you can do a little abuse of existing synchronization contexts like the WPF dispatcher. Your code, translated to this approach:

using System;
using System.IO;
using System.Threading;
using System.Windows;

namespace Node
{
    class Program
    {
        public static void Main()
        {
            var app = new Application();
            app.Startup += ServerStart;
            app.Run();
        }

        private static void ServerStart(object sender, StartupEventArgs e)
        {
            var dispatcher = ((Application) sender).Dispatcher;
            Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
            var path = Environment.ExpandEnvironmentVariables(
                @"%SystemRoot%\Notepad.exe");
            var fs = new FileStream(path, FileMode.Open,
                FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true);
            var bytes = new byte[1024];
            fs.BeginRead(bytes, 0, bytes.Length, ar =>
            {
                dispatcher.BeginInvoke(new Action(() =>
                {
                    var res = fs.EndRead(ar);

                    // Are we in the same thread?
                    Console.WriteLine(Thread.CurrentThread.ManagedThreadId);
                }));
            }, null);
        }
    }
}

prints what you wish. Plus you can set priorities with dispatcher. But agree, this is ugly, hacky and I do not know why I would do it that way for another reason than answer your demo request ;)

Answer 6

The whole SetSynchronizationContext is a red herring, this is just a mechanism for marshalling, the work still happens in the IO Thread Pool.

What you are asking for is a way to queue and harvest Asynchronous Procedure Calls for all your IO work from the main thread. Many higher level frameworks wrap this kind functionality, the most famous one being libevent.

There is a great recap on the various options here: Whats the difference between epoll, poll, threadpool?.

.NET already takes care of scaling for you by have a special "IO Thread Pool" that handles IO access when you call the BeginXYZ methods. This IO Thread Pool must have at least 1 thread per processor on the box. see: ThreadPool.SetMaxThreads.

If single threaded app is a critical requirement (for some crazy reason) you could, of course, interop all of this stuff in using DllImport (see an example here)

However it would be a very complex and risky task:

Why don't we support APCs as a completion mechanism? APCs are really not a good general-purpose completion mechanism for user code. Managing the reentrancy introduced by APCs is nearly impossible; any time you block on a lock, for example, some arbitrary I/O completion might take over your thread. It might try to acquire locks of its own, which may introduce lock ordering problems and thus deadlock. Preventing this requires meticulous design, and the ability to make sure that someone else's code will never run during your alertable wait, and vice-versa. This greatly limits the usefulness of APCs.

So, to recap. If you want a single threaded managed process that does all its work using APC and completion ports, you are going to have to hand code it. Building it would be risky and tricky.

If you simply want high scale networking, you can keep using BeginXYZ and family and rest assured that it will perform well, since it uses APC. You pay a minor price marshalling stuff between threads and the .NET particular implementation.

From: http://msdn.microsoft.com/en-us/magazine/cc300760.aspx

The next step in scaling up the server is to use asynchronous I/O. Asynchronous I/O alleviates the need to create and manage threads. This leads to much simpler code and also is a more efficient I/O model. Asynchronous I/O utilizes callbacks to handle incoming data and connections, which means there are no lists to set up and scan and there is no need to create new worker threads to deal with the pending I/O.

An interesting, side fact, is that single threaded is not the fastest way to do async sockets on Windows using completion ports see: http://doc.sch130.nsc.ru/www.sysinternals.com/ntw2k/info/comport.shtml

The goal of a server is to incur as few context switches as possible by having its threads avoid unnecessary blocking, while at the same time maximizing parallelism by using multiple threads. The ideal is for there to be a thread actively servicing a client request on every processor and for those threads not to block if there are additional requests waiting when they complete a request. For this to work correctly however, there must be a way for the application to activate another thread when one processing a client request blocks on I/O (like when it reads from a file as part of the processing).