Parallel Blob Upload throwing 404 Bad Request inte

Question

I have a very simple service,

public class AzureService : IAzureService
{
    private readonly CloudBlobContainer _container;
    public AzureService(ISettings settings)
    {
        var storageAccount = CloudStorageAccount.Parse(settings.BlobConnectionString);            
        var blobClient = storageAccount.CreateCloudBlobClient();
        _container = blobClient.GetContainerReference(settings.BlobContainerName);
    }

    public Task UploadBlobAsync(string fileName, Stream stream)
    {
        var blob = _container.GetBlockBlobReference(fileName);
        return blob.UploadFromStreamAsync(stream);
    }

    public Task DeleteBlobAsync(string fileName)
    {
        var blob = _container.GetBlockBlobReference(fileName);
        return blob.DeleteAsync();
    }
}

This method is called from,

    public Task SaveAllAsync(Dictionary<string, Stream> images)
    {
        var tasks = new List<Task>();
        foreach (var image in images)
        {
            var fileName = image.Key;
            var stream = image.Value;
            var task = _azureService.UploadBlobAsync(fileName, stream);
            tasks.Add(task);
        }
        return Task.WhenAll(tasks);
    }

My stream is HttpPostedFileBase.InputStream. Sometimes it works and sometimes I get The remote server returned an error: (400) Bad Request.. If I put a break-point it works as well.

Answer 1

The behaviour you are describing sounds very much like a threading problem (i.e. if you breakpoint the code it works fine as it's effectively single-threaded at that time) resulting in incomplete or invalid data being sent to the Azure Storage API.

Your "var image" definition may behave unexpectedly in a multi-threaded environment (If you utilised ReSharper it will highlight this variable and advise to change the code because it is potentially unsafe).

Have a read of this SO post to understand a little more and how you might better implement your code.

The foreach identifier and closures

Answer 2

I had the same problem , I tried to upload 20 + images in 1 strike , single threaded works , multi threaded using await Task.WhenAll failed with "The remote server returned an error: (400) Bad Request."

• see RequestInformation inside Microsoft.WindowsAzure.Storage.StorageException which is thrown from Upload[xxx]Async methods for more detailed information.
  

• At first , RequestInformation said something about a MD5 problem with error code of "Md5Mismatch" , buy my intuition said otherwise because single thread works like a charm , and then .. I found it... DefaultRequestOptions.ParallelOperationThreadCount on CloudBlobClient object and problem sovled.

• BlobRequestOptions Members MSDN

    private CloudBlobContainer ConnectToImageContainer()
    {
        var credentials = new StorageCredentials(AccountName, ImagesContainerKey);
        var account = new CloudStorageAccount(credentials, useHttps: true);
        var client = account.CreateCloudBlobClient();
        client.DefaultRequestOptions.ParallelOperationThreadCount = 64; // max value
        client.DefaultRequestOptions.SingleBlobUploadThresholdInBytes = 67108864; // max value
        var container = client.GetContainerReference(ImagesContainerName);
        return container;
    }