Migrating Data Between Blob Storage Accounts in Azure

by

Over the last couple of posts I’ve been talking about working with different configurations and in my previous post I noted that one of the things we had to do was to migrate some data that had been entered into the Test environment into Production environment (again I stress that I’m not recommending it but occasionally you have to bend the process a little). One of the challenges we encountered was that we not only had to migrate the database, which was easy using the database copy capability in the Azure portal, we also needed to migrate the related blob storage data from one account into another. Here’s some quick code that makes use of the Azure Storage client library (WindowsAzure.Storage package via NuGet and more information here).

Firstly in the app.config we have two connection strings:

<connectionStrings>
    <add name=”BlobMigrator.Properties.Settings.SourceStorage”
      connectionString=”DefaultEndpointsProtocol=https;AccountName=sourceaccount;AccountKey=YYYYYYYYYYY” />
    <add name=”BlobMigrator.Properties.Settings.TargetStorage”
      connectionString=”DefaultEndpointsProtocol=https;AccountName=targetaccount;AccountKey=XXXXXXXXXXXXXXX” />

</connectionStrings>

Next, some straight forward code to iterate through containers in one storage account and copy content across to the target account:

var source= CloudStorageAccount.Parse(Settings.Default.SourceStorage);
var target= CloudStorageAccount.Parse(Settings.Default.TargetStorage);

var sourceClient = source.CreateCloudBlobClient();
var targetClient = target.CreateCloudBlobClient();

var containers= sourceClient.ListContainers(“searchprefix”).ToArray();
Debug.WriteLine(“Source containers: ” + containers.Length);
var idx = 0;
foreach (var cnt in containers)
{
    var tcnt =targetClient.GetContainerReference(cnt.Name);
    await tcnt.CreateIfNotExistsAsync();

    var sblobs = cnt.ListBlobs();
    foreach (var sblob in sblobs)
    {
        var b = await sourceClient.GetBlobReferenceFromServerAsync(sblob.Uri);
        var tb = tcnt.GetBlockBlobReference(b.Name);
        var ok = await tb.StartCopyFromBlobAsync(b.Uri);
        Debug.WriteLine(ok);
    }
    idx++;
    Debug.WriteLine(“Migrated {0} of {1} – {2}”,idx,containers.Length,cnt.Name);
}

In this case it’s limiting the containers that are copied to those that start with the prefix “searchprefix” but this is optional if you want to copy all containers.

Leave a comment