NUnit test with Application loop in it hangs when

I have a few tests with WebBrowser control wrapped with MessageLoopWorker as described here: WebBrowser Control in a new thread

But when another test creates user control or form, the test freezes and never completes:

    [Test]
    public async Task WorksFine()
    {
        await MessageLoopWorker.Run(async () => new {});
    }

    [Test]
    public async Task NeverCompletes()
    {
        using (new Form()) ;
        await MessageLoopWorker.Run(async () => new {});
    }

    // a helper class to start the message loop and execute an asynchronous task
    public static class MessageLoopWorker
    {
        public static async Task<object> Run(Func<object[], Task<object>> worker, params object[] args)
        {
            var tcs = new TaskCompletionSource<object>();

            var thread = new Thread(() =>
            {
                EventHandler idleHandler = null;

                idleHandler = async (s, e) =>
                {
                    // handle Application.Idle just once
                    Application.Idle -= idleHandler;

                    // return to the message loop
                    await Task.Yield();

                    // and continue asynchronously
                    // propogate the result or exception
                    try
                    {
                        var result = await worker(args);
                        tcs.SetResult(result);
                    }
                    catch (Exception ex)
                    {
                        tcs.SetException(ex);
                    }

                    // signal to exit the message loop
                    // Application.Run will exit at this point
                    Application.ExitThread();
                };

                // handle Application.Idle just once
                // to make sure we're inside the message loop
                // and SynchronizationContext has been correctly installed
                Application.Idle += idleHandler;
                Application.Run();
            });

            // set STA model for the new thread
            thread.SetApartmentState(ApartmentState.STA);

            // start the thread and await for the task
            thread.Start();
            try
            {
                return await tcs.Task;
            }
            finally
            {
                thread.Join();
            }
        }
    }

The code steps-in well except return await tcs.Task; never returns.

Wrapping new Form into the MessageLoopWorker.Run(...) seems to make it better, but it does not work with more complicated code, unfortunately. And I have quite a lot of other tests with forms and user controls that I would like to avoid wrapping into messageloopworker.

Maybe MessageLoopWorker can be fixed to avoid the interference with other tests?

Update: following the amazing answer by @Noseratio I've reset the synchronisation context before the MessageLoopWorker.Run call and it now works well.

More meaningful code:

[Test]
public async Task BasicControlTests()
{
  var form = new CustomForm();
  form.Method1();
  Assert....
}

[Test]
public async Task BasicControlTests()
{
    var form = new CustomForm();
    form.Method1();
    Assert....
}

[Test]
public async Task WebBrowserExtensionTest()
{
    SynchronizationContext.SetSynchronizationContext(null);

    await MessageLoopWorker.Run(async () => {
        var browser = new WebBrowser();
        // subscribe on browser's events
        // do something with browser
        // assert the event order
    });
}

When running the tests without nulling the sync context WebBrowserExtensionTest blocks when it follows BasicControlTests. With nulling it pass well.

Is it ok to keep it like this?

I repro'ed this under MSTest, but I believe all of the below applies to NUnit equally well.

First off all, I understand this code might have been taken out of context, but as is, it doesn't seem to be very useful. Why would you want to create a form inside NeverCompletes, which runs on an random MSTest/NUnit thread, different from the thread spawned by MessageLoopWorker?

Anyhow, you're having a deadlock because using (new Form()) installs an instance of WindowsFormsSynchronizationContext on that original unit test thread. Check SynchronizationContext.Current after the using statement. Then, you facing a classic deadlock well explained by Stephen Cleary in his "Don't Block on Async Code".

Right, you don't block yourself but MSTest/NUnit does, because it is smart enough to recognize async Task signature of NeverCompletes method and then execute something like Task.Wait on the Task returned by it. Because the original unit test thread doesn't have a message loop and doesn't pump messages (unlike is expected by WindowsFormsSynchronizationContext), the await continuation inside NeverCompletes never gets a chance to execute and Task.Wait is just hanging waiting.

That said, MessageLoopWorker was only designed to create and run WinForms object inside the scope of the async method you pass to MessageLoopWorker.Run, and then be done. E.g., the following wouldn't block:

[TestMethod]
public async Task NeverCompletes()
{
    await MessageLoopWorker.Run(async (args) =>
    {
        using (new Form()) ;
        return Type.Missing;
    });
}

It was not designed to work with WinForms objects across multiple MessageLoopWorker.Run calls. If that's what you need, you may want to look at my MessageLoopApartment from here, e.g.:

[TestMethod]
public async Task NeverCompletes()
{
    using (var apartment = new MessageLoopApartment())
    {
        // create a form inside MessageLoopApartment
        var form = apartment.Invoke(() => new Form {
            Width = 400, Height = 300, Left = 10, Top = 10, Visible = true });

        try
        {
            // await outside MessageLoopApartment's thread
            await Task.Delay(2000);

            await apartment.Run(async () =>
            {
                // this runs on MessageLoopApartment's STA thread 
                // which stays the same for the life time of 
                // this MessageLoopApartment instance

                form.Show();
                await Task.Delay(1000);
                form.BackColor = System.Drawing.Color.Green;
                await Task.Delay(2000);
                form.BackColor = System.Drawing.Color.Red;
                await Task.Delay(3000);

            }, CancellationToken.None);
        }
        finally
        {
            // dispose of WebBrowser inside MessageLoopApartment
            apartment.Invoke(() => form.Dispose());
        }
    }
}

Or, you can even use it across multiple unit test methods, if you're not concerned about potential coupling of tests, e.g. (MSTest):

[TestClass]
public class MyTestClass
{
    static MessageLoopApartment s_apartment;

    [ClassInitialize]
    public static void TestClassSetup()
    {
        s_apartment = new MessageLoopApartment();
    }

    [ClassCleanup]
    public void TestClassCleanup()
    {
        s_apartment.Dispose();
    }

    // ...
}

Finally, neither MessageLoopWorker nor MessageLoopApartment were designed to work with WinForms object created on different threads (which is almost never a good idea anyway). You can have as many MessageLoopWorker/MessageLoopApartment instances as you like, but once a WinForm object has been created on the thread of a particular MessageLoopWorker/MessageLoopApartment instance, it should further be accessed and properly destroyed on the same thread only.