Cross server DSC dependency options with Azure Resource Manager templates
One of the topics I have been discussing a little lately around the use of the DSC extension for Azure virtual machines is how to create dependencies for DSC configurations that work between servers.If you're not familiar with how to use the DSC extension in Azure ARM templates have a read of my previous blog post on the topic.
Let me give you a basic example of what I'm talking about here though, lets say you are trying to provision an AD server and a SQL server that I want to configure both with DSC. The AD server needs to install features and then create the domain, and then do some service accounts, and the SQL server then needs to join the domain and assign one of those service accounts sysadmin rights on the server. There are a couple of ways we can do this, including the option to do it entirely within ARM, or we can take advantage of some PowerShell 5 features to help streamline this. So if we were to look at the list of actions here we have
- Create the AD server
- Apply the AD configuration
- Create the SQL server
- Apply the SQL configuration
So from the point of view of the dependencies we have here we can say that item 2, depends on 1, and also that 4 depends on 3. We also know that the 4 depends on 2 as well, because it will fail if the AD doesn't exist. So we want to tell the ARM engine that it should do things in a specific order and wait for completion of the resources before moving on to the next thing. The first thing to call out here though is that we are able to provision the VMs at the same time, it's just the DSC extension that we run after the VM provisioning that we need to make sure is sequential - this is great because it can speed up our overall provisioning time.
Creating the dependencies within ARM
The first option is to do our dependencies entirely within ARM, that is to specify that our VMs get provisioned first and then we can chain together the DSC extensions. That would give us something that looks a little like my abridged example:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
[...]
},
"variables": {
[...]
},
"resources": [
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('VNetName')]",
"location": "[resourceGroup().location]",
"properties": {
[...]
}
},
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('DCDomainName')]",
"location": "[resourceGroup().location]",
"properties": {
[...]
}
},
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('SQLDomainName')]",
"location": "[resourceGroup().location]",
"properties": {
[...]
}
},
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/networkInterfaces",
"name": "[variables('DCNicName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('DCDomainName'))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('VNetName'))]"
],
"properties": {
[...]
}
},
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Compute/virtualMachines",
"name": "[variables('DCName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', variables('DCNicName'))]"
],
"properties": {
[...]
}
},
{
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat(variables('DCName'),'/Microsoft.Powershell.DSC')]",
"apiVersion": "2015-05-01-preview",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', variables('DCName'))]"
],
"properties": {
[...]
}
},
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Network/networkInterfaces",
"name": "[variables('SQL1NicName')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('SQLDomainName'))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('VNetName'))]"
],
"properties": {
[...]
}
},
{
"apiVersion": "2015-05-01-preview",
"type": "Microsoft.Compute/virtualMachines",
"name": "[variables('SQL1Name')]",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkInterfaces',variables('SQL1NicName'))]"
],
"properties": {
[...]
}
},
{
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat(variables('SQL1Name'),'/Microsoft.Powershell.DSC')]",
"apiVersion": "2015-05-01-preview",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', variables('SQL1Name'))]",
"[concat('Microsoft.Compute/virtualMachines/', variables('DCName'),'/extensions/Microsoft.Powershell.DSC')]"
],
"properties": {
[...]
}
]
}
I have shortened this one up by hiding the properties on each node just to demonstrate how we are managing the dependencies here. So based on what the dependencies are that I described earlier you can see each of them in the above example:
1. Apply configuration for AD depends on the provisioning of the AD server
You can see the the first "dependsOn" value for the AD DSC extension has one value:
"[resourceId('Microsoft.Compute/virtualMachines', variables('DCName'))]"
So here we are saying that the DSC extension should not begin until the success of the VM provisioning has been completed.
2. Apply configuration for SQL depends on the provisioning of the SQL server
The same as the above example, you see the SQL DSC extension has its first dependency listed as the virtual machine with the name of the SQL server
3. Apply the SQL configuration only after the AD configuration has completed
In the same place on the SQL DSC extension that we tell it to wait for the SQL VM, you also see a second dependency:
"[concat('Microsoft.Compute/virtualMachines/', variables('DCName'),'/extensions/Microsoft.Powershell.DSC')]"
This is where we are telling ARM to not begin this DSC extension until the DSC extension on the AD machine has been completed. The ARM engine will begin the configuration of the AD box and once it completes successfully, the configuration extension will begin working to configure the SQL server, so I know that the domain and service accounts will exist. If something happens that causes the DSC on the AD server to fail, then the ARM for that object will be treated as failing also, which will mean you will see an error and the template will not even attempt the SQL server configuration as one of its dependencies did not complete successfully. So this approach allows us to meet the goal of making sure that the dependencies are all in place before we continue to the next machine.
Improving the process with PowerShell 5
The above approach will work fine for what we are trying to do, but we have the option to streamline this a little further and make things run a bit faster by taking advantage of some new DSC features in the PowerShell 5 preview. If you are using any of the 2.x versions of the DSC extension you should be seeing the WMF 5 preview installed so you can use these techniques in your approach as well. One of the reasons the above approach does take some time is that the DSC extension will install WMF5 and reboot your server before applying the configuration. This will slow each server down by a few minutes which can start to add up when you do it over a few servers - but what if we could do it all at the same time? Well with PowerShell 5 we can manage dependencies across machines within the configurations themselves with the addition of the WaitForAll, WaitForSome and WaitForAny resources.
While WMF5 is still in preview the documentation is still a little sketchy, but the description in the details of "whats new in WMF 5" describes these as:
Cross-computer synchronization is new in DSC in Windows PowerShell 5.0. By using the built-in WaitFor* resources (WaitForAll, WaitForAny, and WaitForSome), you can now specify dependencies across computers during configuration runs, without external orchestrations. These resources provide node-to-node synchronization by using CIM connections over the WS-Man protocol. A configuration can wait for another computer’s specific resource state to change.
So if we take this concept we can be more specific with our dependencies - instead of saying that the configuration for SQL depends on the AD configuration, we can say that the domain join activity on the SQL depends on the create domain activity on the AD server, and that the activity to add our sysadmin account on the SQL server depends on the activity on the AD server that will provision our service account. The WaitFor activities have built in loops to poll for the completion of these activities that we can set our own timeouts for, so we can define the rules that they will play by when checking for completion. This is great for us though as it means we can remove the dependency on the AD server configuration from the SQL one, and the DSC extension on each VM can be free to begin as soon as each VM is provisioned. This will result in the WMF installation happening in parallel on the servers, and they will then pause right at the point of needing the resource on the remote server to be complete. This gives us a more efficient process overall that should execute faster than if we did the dependencies within just the ARM template. There is a nice article on PowerShell Magazine that describes how the WaitForAll action looks in use and is worth a read as well
But there you have it - that's how we can go about chaining together dependencies on configuration across servers in Azure with the DSC extension. You can get quite complex with this pretty quickly so make sure you keep a good level of understanding and documentation on your approaches to ensure that your team/organisation understand your approaches and the dependencies between systems as you go as well. Happy DSC'ing!