httpclient.GetStringAsync blocking

Question

I must be doing something wrong since this code is blocking and runs synchronously, inspite of calling the async method of GetStringAsync. Any help would really help me understand the reasons:

Private Async Sub btnTest_Click(sender As Object, e As EventArgs) Handles btnTest.Click
    Dim urls As List(Of String) = SetUpURLList()
    Dim baseAddress = New Uri("http://www.amazon.com")
    ServicePointManager.DefaultConnectionLimit = 10
    Dim requestNumber As Integer = 0
    For Each url In urls
        requestNumber += 1
        Console.WriteLine("Request #:{0}", requestNumber)
        Dim cookies As New CookieContainer()
        Dim handler As New HttpClientHandler With {.CookieContainer = cookies, _
                                                   .UseCookies = True}
        Dim httpClient = New HttpClient(handler) With {.BaseAddress = baseAddress}
        Dim response As String = Await HttpClient.GetStringAsync(url).ConfigureAwait(False)
        For Each cook As Cookie In cookies.GetCookies(baseAddress)
            Console.WriteLine(cook.Name & "=" & cook.Value)
        Next
        httpClient.Dispose()
    Next

    Console.WriteLine("Done")
End Sub

Answer 1

Here is the complete working code now - thx to @l3arnon for fire all and wait up on all completion.

Dim concurrencySemaphore As New SemaphoreSlim(10)
Private Async Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
    Dim urls As List(Of String) = SetUpURLList()
    ServicePointManager.DefaultConnectionLimit = 10 'Not working strangely
    Dim tasks As List(Of Task) = New List(Of Task)()
    For Each url In urls
        tasks.Add(GetUrl(url))
    Next
    Await Task.WhenAll(tasks)
    Console.WriteLine("Done")
End Sub
Async Function GetUrl(url As String) As Task
    concurrencySemaphore.Wait()
    Dim baseAddress = New Uri("http://www.amazon.com")
    Dim cookies As New CookieContainer()
    Dim handler As New HttpClientHandler With {.CookieContainer = cookies, _
                                               .UseCookies = True}
    Dim httpClient = New HttpClient(handler) With {.BaseAddress = baseAddress}
    Dim response As String = Await httpClient.GetStringAsync(url).ConfigureAwait(False)
    For Each cook As Cookie In cookies.GetCookies(baseAddress)
        Console.WriteLine(cook.Name & "=" & cook.Value)
    Next
    httpClient.Dispose()
    concurrencySemaphore.Release()
End Function

Answer 2

Your code isn't blocking, it's just sequential. You are firing each Async operation and asynchronously waiting for it to complete with Await before starting the next one.

If you want to fire all these operations concurrently first create a task for each url and then Await all these tasks at once using Task.WhenAll:

Dim semaphore As New SemaphoreSlim(10)
Async Sub btnTest_Click(sender As Object, e As EventArgs) Handles btnTest.Click
    Dim urls As List(Of String) = SetUpURLList()
    ServicePointManager.DefaultConnectionLimit = 10
    Dim tasks As List(Of Task) = new List(Of Task)()
    For Each url In urls
        tasks.Add(GetUrlAsync(url))
    Next
    Await Task.WhenAll(tasks)
    Console.WriteLine("Done")
End Sub

Async Function GetUrlAsync(url As String) As Task
    Await semaphore.WaitAsync()
    Dim baseAddress = New Uri("http://www.amazon.com")
    Dim cookies As New CookieContainer()
    Dim handler As New HttpClientHandler With {.CookieContainer = cookies, _
                                               .UseCookies = True}
    Dim httpClient = New HttpClient(handler) With {.BaseAddress = baseAddress}
    Dim response As String = Await HttpClient.GetStringAsync(url).ConfigureAwait(False)
    For Each cook As Cookie In cookies.GetCookies(baseAddress)
        Console.WriteLine(cook.Name & "=" & cook.Value)
    Next
    httpClient.Dispose()
    semaphore.Release()
End Sub

^{*I hope this code makes sense, since I'm not really familiar with VB.Net.}