Task continuation was scheduled to non thread-pool

Question

In my console application I do create my own Thread to implement working queue. Besides I have implemented my own SynchronizationContext for this only thread.

When I await a Task from the Main thread suddenly continuation (the remaining part of my routine) is scheduled on to my working thread what is wrong because I do not expect my thread will be used as a ThreadPool thread for random tasks.

I am experiencing this behaviour only when running the code with Mono.

Here is a code which reproduces the problem on mono (tested at mac os x and linux system):

using System;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;

class Program
{
    static void Main( string[] args )
    {
        Foo();
        Console.ReadLine();
    }


    async static void Foo()
    {
        Console.WriteLine( "{0}: current thread ID={1}; scheduler={2}; context={3};",
            "   Main BEFORE awaiting",
            Thread.CurrentThread.ManagedThreadId, 
            TaskScheduler.Current.Id,
            SynchronizationContext.Current != null );
            // MONO Output: Main BEFORE awaiting: current thread ID=1; scheduler=1; context=False;

        WorkQueue queue = new WorkQueue();

        // !!! 
        // I do expect that current context which is null will be captured for continuation.
        // !!!
        await queue.Enqueue();

        // !!!
        // As we can see our custom context was captured to continue with this part of code.
        // 
        Console.WriteLine( "{0}: current thread ID={1}; scheduler={2}; context={3};",
            "   Main AFTER awaiting",
            Thread.CurrentThread.ManagedThreadId,
            TaskScheduler.Current.Id,
            SynchronizationContext.Current != null );
        // MONO Output: Main AFTER awaiting: current thread ID=4; scheduler=1; context=True;
    }
}

// Custom context which does nothing but enqueues fake tasks to the queue.
//
class WorkQueueSyncContext : SynchronizationContext
{
    readonly WorkQueue queue;

    public WorkQueueSyncContext( WorkQueue queue )
    {
        this.queue = queue;
    }

    public override void Post( SendOrPostCallback d, object state )
    {
    }

    public override void Send( SendOrPostCallback d, object state )
    {
        queue.Enqueue().Wait();
    }
}

// The queue
//
class WorkQueue
{
    readonly Thread thread;

    class WorkQueueItem
    {
        public TaskCompletionSource<object> Completion
        {
            get;
            set;
        }
    }

    BlockingCollection<WorkQueueItem> queue = new BlockingCollection<WorkQueueItem>();


    public WorkQueue()
    {
        thread = new Thread( new ThreadStart( Run ) );
        thread.Start();
    }

    private void Run()
    {
        // Set ower own SynchronizationContext.
        //
        SynchronizationContext.SetSynchronizationContext( new WorkQueueSyncContext( this ) );

        Console.WriteLine( "{0}: current thread ID={1}; scheduler={2}; context={3};",
            "   WorkQueue START",
            Thread.CurrentThread.ManagedThreadId,
            TaskScheduler.Current.Id,
            SynchronizationContext.Current != null );
        // MONO Output: current thread ID=4; scheduler=1; context=True;

        // Working loop.
        //
        while ( true )
        {
            WorkQueueItem item = queue.Take();

            Console.WriteLine( "{0}: current thread ID={1}; scheduler={2}; context={3};",
                "   WorkQueue DOING TASK",
                Thread.CurrentThread.ManagedThreadId,
                TaskScheduler.Current.Id,
                SynchronizationContext.Current != null );
            // MONO Output: current thread ID=4; scheduler=1; context=True;

            // Completed the task :)
            //
            item.Completion.SetResult( true );
        }
    }

    public Task<object> Enqueue()
    {
        TaskCompletionSource<object> completion = new TaskCompletionSource<object>();
        queue.Add( new WorkQueueItem() { Completion = completion } );
        return completion.Task;
    }
}

So, here is MONO output:

   Main BEFORE awaiting: current thread ID=1; scheduler=1; context=False;
   WorkQueue START: current thread ID=3; scheduler=1; context=True;
   WorkQueue DOING TASK: current thread ID=3; scheduler=1; context=True;
   Main AFTER awaiting: current thread ID=3; scheduler=1; context=True;

And this is Windows output:

   Main BEFORE awaiting: current thread ID=10; scheduler=1; context=False;
   WorkQueue START: current thread ID=11; scheduler=1; context=True;
   WorkQueue DOING TASK: current thread ID=11; scheduler=1; context=True;
   Main AFTER awaiting: current thread ID=6; scheduler=1; context=False;

Please note (the last line) how context capturing differs.

EDIT:

Is not reproducible with Mono 3.4.0, so seems to be a bug in older version (at least 3.2.6);

Answer 1

I think you've found a bug in Mono runtime. The continuation after await should not be happening on a thread with a different synchronization context from that captured by TaskAwaiter, at the point of await.

The following scenarios are possible:

1. Both the original thread and the completion thread have the same synchronization context. The continuation may be inlined (executed synchronously on the completion thread).
2. Both the original thread and the completion thread have no synchronization context (SynchronizationContext.Current == null). The continuation still may be inlined.
3. In any other combination, the continuation must not be inlined.

By "may be inlined" I mean it is not required or guaranteed to be so (it still might be scheduled using TaskScheduler.Current or TaskScheduler.FromCurrentSynchronizationContext for asynchronous execution). Nevertheless, under the current Microsoft's implementation of TPL, it does get inlined for conditions #1 and #2.

However, for #3 it must not be inlined, this is dictated by common sense. So feel free to report a bug to Xamarin. Try the most recent Mono build first to see if the problem is still there.