Kirk Evans Blog

.NET From a Markup Perspective

Creating Dev and Test Environments with Windows PowerShell

This post will discuss creating application environments with Windows PowerShell.  We will use these environments in subsequent posts.

Background

I have participated in a series of readiness workshops with our top GSI partners.  Part of the workshop includes case studies where we have the participants review requirements and propose solutions within a few constraints.  One of the case studies involves Visual Studio Release Management.  It’s a very interesting case study, but I felt it could have been so much better with a demo.  You know me, as I come up with ideas like this I try to get some time at the keyboard to build the demo and then share it here on my blog.  This time is no exception.

The case study shows off Visual Studio Release Management to several environments.  I’ll get to that part later, for now we will focus on designing the environment to suit our needs.

Requirements

We will have three environments: dev, stage, and prod.  Each environment must have an uptime SLA of 99.5 or higher while minimizing cost.  These virtual machines must be accessible from on-premises using HTTP over port 8080, but not accessible externally (no HTTP endpoint exposed).  A CPU spike in one environment should not affect the other environments. 

If we break the requirements down, we can see that there are three environments, each requiring an SLA of 99.95% or higher while minimizing cost.  We implement two virtual machines in each environment and place them within an availability set to meet the 99.95% SLA requirement.  To minimize our cost during the POC phase, we will use the Small VM size (for more information on VM sizes, see Virtual Machine and Cloud Service Sizes for Azure).  The part about HTTP over port 8080, we’ll take care of that in a future post as it requires some internal configuration of the VM, but we can take care of the requirement not to expose port 8080 simply by not adding an endpoint for the environment.  We’ll partially address the on-premises connectivity in this post, but that will also need a little more work to complete later.  Finally, we address the customer’s concerns about CPU by placing each environment within its own cloud service.

Our proposed architecture looks like the following.

image

I created this environment using my MSDN subscription in about an hour just using the new Azure portal (https://portal.azure.com).  However, it’s been awhile since I did anything with PowerShell, so let’s show you how to do this using the service management PowerShell SDK.

The Virtual Network

The first task is to create a virtual network XML configuration file.  To be honest, I created this using the old Azure portal (https://manage.windowsazure.com), then exported the VNet to a file.  In the file you can see the 3 subnets with the incredibly clever names of Subnet-1, Subnet-2, and Subnet-3.  The virtual network itself has a name, in this file it’s “kirketestvnet-southcentral”.  You probably want to change that Smile  The value “kirketest” is part of a naming scheme that I use, I prefix all of the services (virtual network, storage, and cloud service) with the same value helping to avoid name collisions as the storage and cloud service names must be globally unique.  In this example, I’ve also added a gateway.  We aren’t going to create the gateway just yet, but we will leave the gateway subnet in the definition for now.

NetworkConfig.xml
  1. <NetworkConfiguration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration">
  2.   <VirtualNetworkConfiguration>
  3.     <VirtualNetworkSites>
  4.       <VirtualNetworkSite name="kirketestvnet-southcentral" Location="South Central US">
  5.         <AddressSpace>
  6.           <AddressPrefix>10.0.1.0/24</AddressPrefix>
  7.         </AddressSpace>
  8.         <Subnets>
  9.           <Subnet name="Subnet-1">
  10.             <AddressPrefix>10.0.1.0/27</AddressPrefix>
  11.           </Subnet>
  12.           <Subnet name="Subnet-2">
  13.             <AddressPrefix>10.0.1.32/27</AddressPrefix>
  14.           </Subnet>
  15.           <Subnet name="Subnet-3">
  16.             <AddressPrefix>10.0.1.64/26</AddressPrefix>
  17.           </Subnet>
  18.           <Subnet name="GatewaySubnet">
  19.             <AddressPrefix>10.0.1.128/29</AddressPrefix>
  20.           </Subnet>
  21.         </Subnets>
  22.         <Gateway>
  23.           <VPNClientAddressPool>
  24.             <AddressPrefix>10.0.0.0/24</AddressPrefix>
  25.           </VPNClientAddressPool>
  26.           <ConnectionsToLocalNetwork />
  27.         </Gateway>
  28.       </VirtualNetworkSite>      
  29.     </VirtualNetworkSites>
  30.   </VirtualNetworkConfiguration>
  31. </NetworkConfiguration>

To set the virtual network configuration, you use the following PowerShell script.

Set-AzureVNetConfig
  1. $vnetConfigFilePath = "C:\temp\NetworkConfig.xml"
  2.  
  3. Set-AzureVNetConfig -ConfigurationPath $vnetConfigFilePath

A word of caution here… this will update ALL of the virtual networks for your subscription.  If you use the XML as-is above, that is the same as telling Azure to delete existing virtual networks that you may have and then add the new network named “kirketest-southcentral”.  Luckily, if those virtual networks are in use, the operation will fail.  I find it much less risky to simply export the existing virtual network, make whatever changes I need to, then use the Set-AzureVNetConfig to apply all of the changes.

Creating Virtual Machines

When you create a virtual machine in Azure, you choose a base image.  Using Get-AzureVMImage, you can get a list of all the available images and their locations.  The image name will be something family-friendly like this:

a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201505.01-en.us-127GB.vhd

Notice the date and version number on that image.  This image will go away at some point, replaced by a newer image.  Hardcoding that image name in your script will cause you problems later, but it’s easy to just obtain the latest image version.  Michael Collier provides a great post on The Case of the Latest Windows Azure VM Image with a simple solution:  get the image sorted by the date it is published, descending, and get the first image.  He also explains in that post how not all images are available in all locations, so you should include the location as part of your filter.

Get-LatestVMImage
  1. function Get-LatestVMImage([string]$imageFamily, [string]$location)
  2. {
  3.     #From https://michaelcollier.wordpress.com/2013/07/30/the-case-of-the-latest-windows-azure-vm-image/
  4.     $images = Get-AzureVMImage `
  5.     | where { $_.ImageFamily -eq $imageFamily } `
  6.         | where { $_.Location.Split(";") -contains $location} `
  7.         | Sort-Object -Descending -Property PublishedDate
  8.         return $images[0].ImageName;
  9. }

Calling this function is really simple, just provide the location name (obtained by Get-AzureLocation).

Code Snippet
  1. #$imageName = "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201505.01-en.us-127GB.vhd"
  2. $imageName = Get-LatestVMImage -imageFamily "Windows Server 2012 R2 Datacenter" -location $location

Now that we have an image, we create a new AzureVMConfig with the settings for our VM.  We then create an AzureProvisioningConfig to tell the provisioning engine that this is a Windows machine with a username and password.  We set a virtual network, and we are left with the configuration object.  We haven’t yet told Azure to create a VM.  This lets us create the configuration for multiple VMs at once before we finally start the provisioning process.  Putting the VMs in an availability set is as easy as providing the –AvailabilitySetName parameter (for more information, see Michael Washam’s post, Understanding and Configuring Availability Sets).

Create Multiple VMs
  1.   New-AzureService -ServiceName $serviceName -Location $location
  2.                          
  3. $vm1 = New-AzureVMConfig -Name "DEV1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  4. Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
  5. Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
  6.  
  7. $vm2 = New-AzureVMConfig -Name "DEV2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  8. Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
  9. Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
  10.  
  11. New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName

We now have the basic building blocks out of the way.  The rest of the script is simply putting everything together. 

The Script

The rest of the script is fairly straightforward.  We create a cloud service for each environment, two virtual machines in an availability set in each cloud service, each VM is placed within a virtual network subnet.  The full script is shown here:

Full Script
  1. function Get-LatestVMImage([string]$imageFamily, [string]$location)
  2. {
  3.     #From https://michaelcollier.wordpress.com/2013/07/30/the-case-of-the-latest-windows-azure-vm-image/
  4.     $images = Get-AzureVMImage `
  5.     | where { $_.ImageFamily -eq $imageFamily } `
  6.         | where { $_.Location.Split(";") -contains $location} `
  7.         | Sort-Object -Descending -Property PublishedDate
  8.         return $images[0].ImageName;
  9. }
  10.  
  11.   $prefix = "mydemo"
  12. $storageAccountName = ($prefix + "storage")
  13.                $location = "South Central US"
  14. $vnetConfigFilePath = "C:\temp\NetworkConfig.xml"
  15.  
  16. #$imageName = "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201505.01-en.us-127GB.vhd"
  17. $imageName = Get-LatestVMImage -imageFamily "Windows Server 2012 R2 Datacenter" -location $location
  18.  
  19. $size = "Small"
  20.   $adminUsername = "YOUR_USERNAME_HERE"
  21.   $adminPassword = "YOUR_PASSWORD_HERE"
  22. $vnetName = ($prefix + "vnet-southcentral")
  23. #Use Get-AzureSubscription to find your subscription ID
  24. $subscriptionID = "YOUR_SUBSCRIPTION_ID_HERE"
  25.  
  26. #Set the current subscription
  27. Select-AzureSubscription -SubscriptionId $subscriptionID -Current
  28.  
  29. #Create storage account
  30. New-AzureStorageAccount -StorageAccountName $storageAccountName -Location $location
  31.  
  32. #Set the current storage account
  33. Set-AzureSubscription -SubscriptionId $subscriptionID -CurrentStorageAccountName $storageAccountName
  34.  
  35. #Create virtual network
  36. Set-AzureVNetConfig -ConfigurationPath $vnetConfigFilePath
  37.  
  38.  
  39. #Development environment
  40. $avSetName = "AVSET-DEV"
  41. $serviceName = ($prefix + "DEV")
  42. $subnetName = "Subnet-1"
  43.  
  44.   New-AzureService -ServiceName $serviceName -Location $location
  45.                          
  46. $vm1 = New-AzureVMConfig -Name "DEV1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  47. Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
  48. Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
  49.  
  50. $vm2 = New-AzureVMConfig -Name "DEV2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  51. Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
  52. Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
  53.  
  54. New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName
  55.  
  56.  
  57. #Staging environment
  58. $avSetName = "AVSET-STAGE"
  59. $serviceName = ($prefix + "STAGE")
  60. $subnetName = "Subnet-2"
  61.  
  62.   New-AzureService -ServiceName $serviceName -Location $location
  63.                          
  64. $vm1 = New-AzureVMConfig -Name "STAGE1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  65. Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
  66. Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
  67.  
  68. $vm2 = New-AzureVMConfig -Name "STAGE2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  69. Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
  70. Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
  71.  
  72. New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName
  73.  
  74.  
  75. #Production environment
  76. $avSetName = "AVSET-PROD"
  77. $serviceName = ($prefix + "PROD")
  78. $subnetName = "Subnet-3"
  79.  
  80.   New-AzureService -ServiceName $serviceName -Location $location
  81.                          
  82. $vm1 = New-AzureVMConfig -Name "PROD1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  83. Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
  84. Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
  85.  
  86. $vm2 = New-AzureVMConfig -Name "PROD2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
  87. Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
  88. Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
  89.  
  90. New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName

 

The Result

I ran the script using my MSDN subscription.  In about 10 minutes I had 6 virtual machines grouped within 3 cloud services and 3 virtual networks.  I could then change the subscription and prefix variables then run the script in a different subscription.  I could have done all of this using the portal (the first time, I did), but once I had the script completed it became an asset to create the environments in a repeatable and consistent manner.

The virtual network shows the resources that are deployed into the correct subnets.

image

Looking at the Configure tab of a single virtual machine lets us see that the virtual machine is part of an availability set.

image

Clean Up

This is my MSDN subscription, so I don’t want to leave resources lying around if I am not using them.  Clean up is simple, I run another script to delete everything I just created.  Again, a word of caution:  this deletes the virtual machines, the associated VHD files, the cloud services, the storage account, and the virtual network configuration.  This is not a reversible operation. 

Clean Up
  1. $prefix = "mydemo"
  2. $storageAccountName = ($prefix + "storage")
  3. $subscriptionID = "YOUR_SUBSCRIPTION_ID_HERE"
  4.  
  5. #Set up credentials
  6. Add-AzureAccount
  7.  
  8. #Set the current subscription
  9. Select-AzureSubscription -SubscriptionId $subscriptionID -Current
  10.  
  11. $serviceName = ($prefix + "DEV")
  12. #The following command deletes the associated VHD, but takes awhile
  13. Get-AzureVM -ServiceName $serviceName | %{Remove-AzureVM -Name $_.Name -ServiceName $serviceName -DeleteVHD}
  14. Remove-AzureService -ServiceName $serviceName -Force
  15.  
  16. $serviceName = ($prefix + "STAGE")
  17. #The following command deletes the associated VHD, but takes awhile
  18. Get-AzureVM -ServiceName $serviceName | %{Remove-AzureVM -Name $_.Name -ServiceName $serviceName -DeleteVHD}
  19. Remove-AzureService -ServiceName $serviceName -Force
  20.  
  21. $serviceName = ($prefix + "PROD")
  22. #The following command deletes the associated VHD, but takes awhile
  23. Get-AzureVM -ServiceName $serviceName | %{Remove-AzureVM -Name $_.Name -ServiceName $serviceName -DeleteVHD}
  24. Remove-AzureService -ServiceName $serviceName -Force
  25.  
  26. #Remove storage account.  This will fail if the
  27. #disks haven't finished deleting yet.
  28. Remove-AzureStorageAccount -StorageAccountName $storageAccountName
  29.  
  30. #Remove all unused VNets
  31. Remove-AzureVNetConfig

Coming up next we’ll look at how we can establish on-premises connectivity for just a few devices, and then we’ll turn our attention to deploying some code and services to these virtual machines using Visual Studio Release Management.

For More Information

Virtual Machine and Cloud Service Sizes for Azure

Manage the availability of virtual machines

The Case of the Latest Windows Azure VM Image

Understanding and Configuring Availability Sets