Running native code on Azure

Question

I am trying to run a C executable on Azure. I have many workerRoles and they continuously check a Job Queue. If there is a job in the queue, a worker role runs an instance of the C executable as a process according to the command line arguments stored in a job class. The C executable creates some log files normally. I do not know how to access those created files. What is the logic behind it? Where are the created files stored? Can anyone explain me? I am new to azure and C#.

One other problem is that all of the working instances of the C executable need to read a data file. How can I distribute that required file?

Answer 1

In Azure land you shouldn't write to the file system. You should write to SQL Azure, Table Storage or most likely in this case Blob storage (basically, I think you should think of blob storage as the old file system)

This is because:

1. You could have multiple instances running and you will end up having different files on different instances (which are just virtual machines)

2. Your instance could potentially be moved at any moment and you would lose the info on the file system as it's not part of your deployment package.

Using one of the three storage options will provide a central repository for all of your instances to access and it will be persisted over a redeployment.

Answer 2

First, realize that in Windows Azure, your worker role is simply running inside a Windows 2008 Server environment (either SP2 or R2). When you deploy your app, you would deploy your C executable as well (or grab it from blob storage, but that's a bit more advanced). To find out where your app lives on disk, call Environment.GetEnvironmentVariable("RoleRoot") - that returns a path. You'd typically have your app sitting in a folder called AppRoot under the role root. You'd find your C executable there.

Next, you'll want your app to write its files to an output directory you specify on the command line. You can set up storage in your local VM with your role's properties. Look at the Local Storage tab, and configure a named local storage area:

Now you can get the path to that storage area, in code, and pass it as a command line argument:

var outputStorage = RoleEnvironment.GetLocalResource("MyLocalStorage");
var outputFile = Path.Combine(outputStorage.RootPath, "myoutput.txt");
var cmdline = String.Format("--output {0}", outputFile);

Here's an example of launching your myapp.exe process, with command line arguments:

var appRoot = Path.Combine(Environment.GetEnvironmentVariable("RoleRoot")
            + @"\", @"approot");

var myProcess = new Process()
{
   StartInfo = new ProcessStartInfo(Path.Combine(appRoot, @"myapp.exe"), cmdline)
   {
      CreateNoWindow = false,
      UseShellExecute = false,
      WorkingDirectory = appRoot
   }
};
myProcess.WaitForExit();

Normally you'd set CreateNoWindow to true, but it's easier to debug if you can see the command shell window.

Last thing: Once your app is done creating the file, you'll want to either:

• Process it and delete it (it's not in a durable place so eventually it'll disappear)
• Change your storage to use a Cloud Drive (durable storage)
• Copy your file to a blob (durable storage)

In production, you'll want to add exception-handling, and you can re-route stdout and stderr to be captured. But this sample code should be enough to get you started.

OOPS - one more 'one more thing': When adding your 'myapp.exe' to your project, be SURE to go to its Properties, and set 'Copy to Output Directory' to 'Copy Always' - otherwise your myapp.exe file won't end up in Windows Azure and you'll wonder why things don't work.

EDIT: Pushing results to a blob - a quick example

First get set up a storage account and add to your role's Settings. Say you called it 'AzureStorage' - now set it up in code, get a reference to a blob container, get a reference to a blob within that container, and then perform a file upload to the blob:

        CloudStorageAccount storageAccount = CloudStorageAccount.FromConfigurationSetting("AzureStorage");
        CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
        CloudBlobContainer outputfiles = blobClient.GetContainerReference("outputfiles");
        outputfiles.CreateIfNotExist();

        var blobname =  "myoutput.txt";
        var blob = outputfiles.GetBlobReference(blobname);
        blob.UploadFile(outputFile);