Set-AzureVMDscExtension with large package leads t

Question

I finally managed to automate our release process using Desired State Configuration with the Azure PowerShell SDK methods, in particular the Publish-AzureVMDscConfiguration -> Set-AzureVMDscExtension -> Update-AzureVM combo.

After thinking for a while in a way to send my build outputs somewhere accessible by the VM, I ended up with the strategy of appending my build drops in the configuration package that gets uploaded to Azure Storage.

My problem now is that as soon as the PowerShell DSC Extension in the VM starts downloading that package, it's memory consumption goes through the roof. When I open task manager, I can see the newly created PowerShell process going from 30 or so megabytes, to 300, and then to 1.3GB, completely ruining my VM.

Yesterday afternoon, I left work and let it processing, but when I logged into the VM today, the inner zip file, containing my build outputs, had 0 bytes in the DSCWork folder. My problem is that even if it worked in the end, it is taking a very long time and making my VM useless... I can't even change between windows in remote access, since the machine is completely stuck at 100% RAM usage.

Why is PowerShell taking so much memory and time to download my configuration package? It only has 60MB zipped, and roughly 200MB unzipped. Is there something I can do to prevent that from happening?

UPDATE:

I tested it just now and it finally finished correctly. Took more than a full hour, but the files are there... This is unacceptable though.

Answer 1

This issue should be resolved in the next iteration of the extension. In the meanwhile you may want to consider uploading your build content to a blob separate from your configuration ZIP package (you can use Set-AzureStorageBlobContent for this).

Then you can use either the remote file or script resources in your original configuration to download the blob. Be sure to add the appropriate dependencies in your configuration so that the blob gets downloaded before you use it.

configuration DownloadSample
{
    Import-DscResource -Module xPSDesiredStateConfiguration

    xRemoteFile Download
    { 
       Uri = 'https://....blob.core.windows.net/windows-powershell-dsc/foo.zip?sv=...'
       DestinationPath = 'd:\tmp\download.zip'
   }
}