Copying storage data from one Azure account to ano

Question

I would like to copy a very large storage container from one Azure storage account into another (which also happens to be in another subscription).

I would like an opinion on the following options:

1. Write a tool that would connect to both storage accounts and copy blobs one at a time using CloudBlob's DownloadToStream() and UploadFromStream(). This seems to be the worst option because it will incur costs when transferring the data and also be quite slow because data will have to come down to the machine running the tool and then get re-uploaded back to Azure.

2. Write a worker role to do the same - this should theoretically be faster and not incur any cost. However, this is more work.

3. Upload the tool to a running instance bypassing the worker role deployment and pray the tool finishes before the instance gets recycled/reset.

4. Use an existing tool - have not found anything interesting.

Any suggestions on the approach?

Update: I just found out that this functionality has finally been introduced (REST APIs only for now) for all storage accounts created on July 7th, 2012 or later:

http://msdn.microsoft.com/en-us/library/windowsazure/dd894037.aspx

Answer 1

You can also use AzCopy that is part of the Azure SDK.

Just click the download button for Windows Azure SDK and choose WindowsAzureStorageTools.msi from the list to download AzCopy.

After installing, you'll find AzCopy.exe here: %PROGRAMFILES(X86)%\Microsoft SDKs\Windows Azure\AzCopy

You can get more information on using AzCopy in this blog post: AzCopy – Using Cross Account Copy Blob

As well, you could remote desktop into an instance and use this utility for the transfer.

Update:

You can also copy blob data between storage accounts using Microsoft Azure Storage Explorer as well. Reference link

Answer 2

Try CloudBerry Explorer. It copies blob within and between subscriptions.

For copying between subscriptions, edit the storage account container's access from Private to Public Blob.

The copying process took few hours to complete. If you choose to reboot your machine, the process will continue. Check status by refreshing the target storage account container in Azure management UI by checking the timestamp, the value gets updated until the copy process completes.

Answer 3

I'm a Microsoft Technical Evangelist and I have developed a sample and free tool (no support/no guarantee) to help in these scenarios.

The binaries and source-code are available here: https://blobtransferutility.codeplex.com/

The Blob Transfer Utility is a GUI tool to upload and download thousands of small/large files to/from Windows Azure Blob Storage.

Features:

• Create batches to upload/download
• Set the Content-Type
• Transfer files in parallel
• Split large files in smaller parts that are transferred in parallel

The 1st and 3rd feature is the answer to your problem.

You can learn from the sample code how I did it, or you can simply run the tool and do what you need to do.