Using Await in BackgroundWorker causes WorkerCompl

I have this weird problem. I need to invoke a process from within a background worker

Private Shared _process As Process
Private Shared _StartInfo As ProcessStartInfo
Private WithEvents _bwConvertMedia As New BackgroundWorker

Here is the work in DoWorkAsync

Private Async Sub _bwConvertMedia_DoWorkAsync(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles _bwConvertMedia.DoWork
  For AI = 1 To 100
    _StartInfo = New ProcessStartInfo(".\mycmd.exe", "-1")
    _StartInfo.RedirectStandardOutput = True
    _StartInfo.UseShellExecute = False
    _StartInfo.CreateNoWindow = True
    _StartInfo.RedirectStandardError = True

    _process = New Process() With {.EnableRaisingEvents = True, .StartInfo = _StartInfo}
    AddHandler _process.OutputDataReceived, AddressOf OutputHandler
    AddHandler _process.ErrorDataReceived, AddressOf ErrorHandler
    AddHandler _process.Exited, AddressOf Exited
    Try
      aSuccess = Await AwaitProcess()
    Catch ex As Exception
    End Try
    _bwConvertMedia.ReportProgress(ai)
  Next

And here the

Private Shared Async Function AwaitProcess() As Task(Of Integer)
  _tcs = New TaskCompletionSource(Of Integer)
  _status.Converting = True
  _Error.Clear()
  _process.Start()
  _process.BeginErrorReadLine()
  _process.BeginOutputReadLine()    
  Return Await _tcs.Task
End Function

The issue is that when the Await _tcs.Task is executed the _bwConvertMedia RunWorkerCompleted procedure is executed so when I do call the _bwConvertMedia.ReportProgress(ai)

I get an error that the worker is already finished.

Why is that? can you help me?

What happens is

DoWork - iteration 1
at wait process 1
RunWorkerComplete
DoWork iteration 2-100

The correct behavior is that the background worker invokes 100 times the process and THEN it finishes the execution and calls the RunWorkerCompleted

I made some modifications to the code I previously linked and here are two examples of a sequential non-blocking Async/Await procedure and a non-blocking Parallel procedure using Task.Factory.

Since I can't test your program, I simply used Tracert.exe to simulate a stdout result to update the User Interface.

To synchronize the running tasks/threads with the UI, I used in the first case the .SynchronizingObject of the processes and in the second the TaskScheduler method TaskScheduler.FromCurrentSynchronizationContext().

The output from Tracert.exe is passed to two TextBoxes. In the parallel example, I inserted a delay of 1 second between Tasks, to see how the two TextBoxes are updated.

The Async/Await example can be modified to work differently, since you don't need to wait for a task to complete to start another.

The ProcessStartInfo and Process objects are added to a Pool using a List(Of ProcessStartInfo) and a List(Of Process).

These are used in both examples. Define a correct scope.

Public psInfoPool As List(Of ProcessStartInfo)
Public ProcessPool As List(Of Process)

Sequential Async/Await

The delegate is used with SynchronizingObject.BeginInvoke if InvokeRequired = true

Public Delegate Sub UpdUI(_object As TextBox, _value As String)

Public Sub UpdateUIDelegate(control As TextBox, _input As String)
    control.AppendText(_input)
End Sub

    Dim NumberOfProcesses As Integer
    For x = 0 To 1
        Dim OutCtl As TextBox = If(x = 0, Me.TextBox1, Me.TextBox2)
        Dim _result As Integer = Await Task.Run(Async Function() As Task(Of Integer)
                                     Return Await Test_SequentialAsync("192.168.1.1", OutCtl)
                                 End Function)
        NumberOfProcesses += _result
    Next

The MediaToConvert parameter would be the name of the file to convert when you adapt the examples to your needs. The OutCtl parameter is just the TextBox used for the output

Public Async Function Test_SequentialAsync(ByVal MediaToConvert As String, OutCtl As TextBox) As Task(Of Integer)
    Dim _CurrentProcessInfo As Integer
    Dim _CurrentProcess As Integer

    Dim ExitCode As Integer = Await Task.Run(Function() As Integer

        Dim _processexitcode As Integer

        psInfoPool.Add(New ProcessStartInfo)
        _CurrentProcessInfo = psInfoPool.Count - 1
        psInfoPool(_CurrentProcessInfo).RedirectStandardOutput = True
        psInfoPool(_CurrentProcessInfo).CreateNoWindow = True
        psInfoPool(_CurrentProcessInfo).UseShellExecute = False
        'Name of the executable to start
        psInfoPool(_CurrentProcessInfo).FileName = "Tracert"    'psInfo.FileName = ".\mycmd.exe"""
        'Parameter(s) to pass to the executable
        psInfoPool(_CurrentProcessInfo).Arguments = MediaToConvert
        psInfoPool(_CurrentProcessInfo).WindowStyle = ProcessWindowStyle.Hidden

        ProcessPool.Add(New Process)
        _CurrentProcess = ProcessPool.Count - 1

        ProcessPool(_CurrentProcess) = New Process() With {.StartInfo = psInfoPool(_CurrentProcessInfo),
                                                           .EnableRaisingEvents = True,
                                                           .SynchronizingObject = Me}

        ProcessPool(_CurrentProcess).Start()
        ProcessPool(_CurrentProcess).BeginOutputReadLine()
        AddHandler ProcessPool(_CurrentProcess).OutputDataReceived,
            Sub(sender As Object, e As DataReceivedEventArgs)
                    If e.Data IsNot Nothing Then
                        If ProcessPool(_CurrentProcess).SynchronizingObject.InvokeRequired Then
                            ProcessPool(_CurrentProcess).SynchronizingObject.BeginInvoke(
                                                         New UpdUI(AddressOf UpdateUIDelegate),
                                                         New Object() {OutCtl,
                                                         e.Data + Environment.NewLine})
                        Else
                            OutCtl.AppendText(e.Data + Environment.NewLine)
                        End If
                    End If
            End Sub

        'Add an event handler for the Exited event
        AddHandler ProcessPool(_CurrentProcess).Exited,
                Sub(source As Object, ev As EventArgs)
                    _processexitcode = ProcessPool(_CurrentProcess).ExitCode
                    Console.WriteLine("The process has exited. Code: {0}  Time: {1}",
                    _processexitcode,
                    ProcessPool(_CurrentProcess).ExitTime)
                End Sub

        ProcessPool(_CurrentProcess).WaitForExit()
        ProcessPool(_CurrentProcess).Close()
        Return _processexitcode
    End Function)

    Return If(ExitCode = 0, 1, 0)

End Function

Parallel processes using Task.Fatory

Define a Scheduler and associate it with the current context

Public _Scheduler As TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext()

To use Await Task.Delay(1000) you must be in an Async method, but it's just for testing the output, it's not needed.

For x = 0 To 1
    Dim OutCtl As TextBox = If(x = 0, Me.TextBox1, Me.TextBox2)
    Dim _result As Integer = Test_ParallelTasks("192.168.1.1", OutCtl)
    Await Task.Delay(1000)
    NumberOfProcesses += _result
Next

Note that a new Task is created when the OutputDataReceived event handler reports that new data has been received. The UI is updated accordingly using DataReceivedEventArgs e.Data.

Private Function Test_ParallelTasks(ByVal MediaToConvert As String, OutCtl As TextBox) As Integer
    Dim _processexitcode As Integer
    Dim _CurrentProcessInfo As Integer
    Dim _CurrentProcess As Integer

    Task.Factory.StartNew(Function()
        psInfoPool.Add(New ProcessStartInfo)
        _CurrentProcessInfo = psInfoPool.Count - 1
        psInfoPool(_CurrentProcessInfo).RedirectStandardOutput = True
        psInfoPool(_CurrentProcessInfo).CreateNoWindow = True
        psInfoPool(_CurrentProcessInfo).UseShellExecute = False
        psInfoPool(_CurrentProcessInfo).FileName = "Tracert"  'psInfo.FileName = ".\mycmd.exe"
        psInfoPool(_CurrentProcessInfo).Arguments = MediaToConvert
        psInfoPool(_CurrentProcessInfo).WindowStyle = ProcessWindowStyle.Hidden

        ProcessPool.Add(New Process)
        _CurrentProcess = ProcessPool.Count - 1
        ProcessPool(_CurrentProcess) = New Process() With {.StartInfo = psInfoPool(_CurrentProcessInfo),
                                                           .EnableRaisingEvents = True,
                                                           .SynchronizingObject = Me}

        ProcessPool(_CurrentProcess).Start()
        ProcessPool(_CurrentProcess).BeginOutputReadLine()

        AddHandler ProcessPool(_CurrentProcess).OutputDataReceived,
            Sub(sender As Object, e As DataReceivedEventArgs)
                If e.Data IsNot Nothing Then
                    Try
                        'Update the UI or report progress 
                        Dim UpdateUI As Task = Task.Factory.StartNew(Sub()
                        Try
                            OutCtl.AppendText(e.Data + Environment.NewLine)
                        Catch exp As Exception
                              'An exception may raise if the form is closed
                        End Try

                        End Sub, CancellationToken.None, TaskCreationOptions.PreferFairness, _Scheduler)
                        UpdateUI.Wait()

                    Catch exp As Exception
                       'Do something here
                    End Try
                End If
            End Sub

        'Add an event handler for the Exited event
        AddHandler ProcessPool(_CurrentProcess).Exited,
                Sub(source As Object, ev As EventArgs)
                    _processexitcode = ProcessPool(_CurrentProcess).ExitCode
                    Console.WriteLine("The process has exited. Code: {0}  Time: {1}",
                    _processexitcode,
                    ProcessPool(_CurrentProcess).ExitTime)
                End Sub

        ProcessPool(_CurrentProcess).WaitForExit()
        ProcessPool(_CurrentProcess).Close()
        Return _processexitcode
    End Function, TaskCreationOptions.LongRunning, CancellationToken.None)

    Return If(_processexitcode = 0, 1, 0)
End Function