using Parallel.ForEach with/or async/await

Question

I try to verify my image URLs to see whether or not they are valid. I have so many of them that it take hours to complete this task. Therefore, I decided to do it asynchronously. I would like to know if there are any big differences or advantage of doing my code as below.

My main functions are:

Async Function testUrl_async(ByVal myImageurl As String) As Task(Of Boolean)

   myHttpResponse = Await myHttpClient.GetAsync(myImageurl)
    If myHttpResponse.IsSuccessStatusCode Then
        mySuccess = True
    Else
        mySuccess = False
    End If

    Return mySuccess
End Function

 Function testUrl(ByVal myImageurl As String) As  Boolean

   myHttpResponse = myHttpClient.GetAsync(myImageurl)
    If myHttpResponse.IsSuccessStatusCode Then
        mySuccess = True
    Else
        mySuccess = False
    End If

    Return mySuccess
End Function

1) using async await.

For Each myImage In myImages
    Dim result=await testUrl_async(myImageUrl).Result 
    'some code                  
Next

2) using parallel foreach

Parallel.ForEach(myImages, 
    Sub(myImage)
        testUrl(pictureComponent.websiteShop.hqpatronen, myImageUrl) 
        'some code
    End Sub)

3) using parallel foreach and asnyc/await

Parallel.ForEach(myImages, 
    Sub(myImage)
        await testUrl_async(pictureComponent.websiteShop.hqpatronen, myImageUrl) 
    'some code
    End Sub)

The third one could be the best solution, but it will not allow me to call Await/Async within the ForEach.

If I use the second one, the testurl function has the async http call, but not with Await, thereofore it crashes with the exception message:

[TaskCanceledException: A task was canceled.]

on the line that calls myHttpClient.GetAsync. I am guessing that it throws this exception because ForEach has ended and cancellation was requested but httpclient didn't finish its job yet. How can i handle this if this could be the best solution?

Alternatively any other solution which makes my job faster.

Answer 1

You certainly don't want to use Parallel.ForEach. Parallel is for spreading CPU-bound algorithms over multiple cores, which would provide you no benefit (in your scenario, your algorithm is not CPU-bound).

What you actually want is concurrency, not parallelism. Asynchronous concurrency can be done using Task.WhenAll:

Dim tasks = myImages.Select(Function(x) testUrl_async(x))
Dim results = Await Task.WhenAll(tasks)