DevOps - Using Azure MSI with VSTS - step by step

  • Article

Hi,

[Update]: I have released a custom extension doing all of this on the marketplace. https://marketplace.visualstudio.com/items?itemName=stephane-eyskens.aadv1appprovisioning

Microsoft recently announced Azure Managed Service Identity (MSI) which in a nutshell, is a way to avoid storing credentials in code or in locations such as the web.config, the app service settings etc...thanks to an automatically provisioned Service Principal (bootstrap identity) that you can leverage using the App Service (or other components supporting MSI).

As Microsoft highlights in the above article, even Azure Key Vault didn't really solve the problem of disclosing credentials since your code needed credentials to get access to the Vault. Therefore, any developer could have written a console app, connect  & retrieve the actual secret values from the Vault.

Ok, that was still better than storing the credentials in the previously mentioned locations but that wasn't the panacea. So, let's see how to leverage Azure MSI from VSTS and thus, not disclosing any credentials to anyone.

In this walkthrough, I will build upon my previous blog post that explains how to provision Azure AD Apps in a highly controlled way because when provisioning AAD Apps, you also generate app secrets and these guys are typically good candidates to be stored in the Vault and retrieved via MSI.

So, I will assume you already followed what I explained there but here is a short recap of the actions to be taken upfront:

  • Creation of a VSTS Azure AD App that is granted the Manage apps that this app creates or own application permission + admin consent of the VSTS App
  • Granting RBAC contributor/owner role access to the subscription
  • Creating a Service Endpoint in VSTS that makes use of the vsts app credentials
  • Creating a Variable Group (or env variable on on-premises build servers) that contains a hidden variable holding the secret value of the vsts app

The above list of steps is a pre-requisite before you try out what comes next.

Creating a Key Vault

Of course, if you don't have a key vault yet, just create one. You can go to the portal and add a new service of type Key Vault. I created one called devopstore which I'll mention further in this article.

Grant the VSTS app access to this newly created Key Vault

If you followed the steps mentioned in my previous blog post, your VSTS app is already contributor/owner of the subscription. However, that doesn't make it a Key Vault contributor. Key Vault comes with its own access policies, so you have to add an access policy to your vault as shown below.

to access this screen, click on your Vault and click on Access Policies.

Note that I only took Secret Management as for what I want to demo here, it's enough, however, feel free to grant more permissions should you require them.

With that in hand, our VSTS app should now be able to create new secrets and that's what we'll do when provisioning new AAD Apps.

Deploying a Web App with MSI enabled

If you're already using ARM templates to deploy MSI, this step should be crystal clear to you. In this blog post, Microsoft explains how to enable MSI using ARM templates. In a nutshell, you only have to add the following JSON property to your existing templates:

This property will trigger the creation of the bootstrap identity. On top of this, since our ultimate goal is to read a secret value from the Vault with the bootstrap identity, we need to push a key/value pair to the App Service's app settings:

So, in the template file we bind an app setting parameter to a release variable defined in the release definition, this release variable holds the value of the secret name to be stored in Key Vault. I recommend using an unpredictable secret name, you'll understand later why.

You might know that Key Vault secrets keep an history of their values so, when getting secrets from the Vault, if you don't specify the history ID, you'll get the last one back by default. In my release variable, I can only give a secret name since there is no history yet.

Of course, we also need to define this parameter in the parameters.json file:

Release definition - Deploying the webapp with our ARM template

So, similarly to what we did in my previous post, let's create a release definition directly (just for the test) and add an Azure Deployment task that is using our VSTS service endpoint.

Note that for sake of simplicity, I uploaded both the template.json & parameters.json files to one of my websites (anonymous access). By the way, feel free to download these files as they contain the necessary bits to deploy the webapp with MSI enabled. Note that if you replay them, you should change the webapp's name as of course, there can only be one devopswebappazblog.azurewebsites.net at a time. Also, I'm overriding the template parameter webapisecretname with the value of the release variable.

Release definition - Granting Key Vault access to the bootstrap identity

That's where things become a little bit more complicated. The bootstrap identity gets created and is usable from the web app but by default, it's not granted access to vault. However, your code needs to retrieve the sensitive information so we need to make sure it can do so. Remember that what we want to achieve is to retrieve AAD Apps secrets from the Vault in order to not disclose this information by storing it in inappropriate locations.

Now, you can add an Azure PowerShell task that is using our VSTS service endpoint and add this piece of code:

 
Install-Module -Name AzureADPreview -Force -Scope CurrentUser
$body="resource=https%3A%2F%2Fgraph.windows.net&client_id=vstsappclientid&client_secret=$(vstsappsecret)&grant_type=client_credentials"
$resp=Invoke-WebRequest -UseBasicParsing -Uri https://login.microsoftonline.com/yourtenant/oauth2/token -Method POST -Body $body| ConvertFrom-Json
Connect-AzureAD -TenantId yourtenant -AadAccessToken $resp.access_token -AccountId user@yourtenant

Now, I'm going to hard code the web app's name (devopswebappazblog) for sake of simplicity and I'm going to grant the bootstrap identity access to the vault:

 

$NewApp=New-AzureADApplication -DisplayName "devopswebappadapp" -IdentifierUris "https://devopswebappadapp"
$NewAppPrincipal = New-AzureADServicePrincipal -AppId $NewApp.AppId
$secret=New-AzureADApplicationPasswordCredential -ObjectId $NewApp.ObjectId
$principal = Get-AzureADServicePrincipal -Filter "DisplayName eq 'devopswebappazblog'"
Set-AzureRmKeyVaultAccessPolicy -VaultName devopstore -ObjectId $principal.ObjectId -PermissionsToSecrets get
$ss = ConvertTo-SecureString -String $secret.Value -AsPlainText -Force
Set-AzureKeyVaultSecret -VaultName devopstore -Name $(webapisecretname) -SecretValue $ss

So, let's explain a little bit the above snippets:

  • In the first snippet, I get an access token to leverage the VSTS AzureAD app and its application permission. Feel free to read my previous blog post if you want to know more.
  • First line of the second snippet, I provision an Azure AD App for the webapp (that this webapp could use). In this example, I don't give any permission to it because I want to make it as simple as possible and focus on MSI but if you read my previous blog post, you'll also see a more realistic example
  • I then create a new service principal for this AAD App (not needed for MSI but it's a typical step when provisioning AAD apps)
  • Then I create a new PasswordCredential and I store the result in the $secret variable which holds the secret value that is to be stored in Key Vault
  • Then I retrieve the SPN of the bootstrap identity. I'm filtering SPNs on the displayName which will be equal to the name of the webapp. It's kind of a weak way to filter but I have no other information at my disposal.
  • Then I setup a new Access Policy with a GET permission on Secrets. Note that with GET you can get the value of any secret but you need to know its path first. That's why I recommended using unpredictable secret names to avoid having apps trying to read non-owned secrets.
  • Then I convert the secret value into a secure string.
  • I finish with the creation of the secret. Note that the secret name comes from release variables as depicted earlier.

Testing MSI

Now that we ran a minimal release definition with a task that deployed our MSI-enabled webapp via an ARM template and an Azure PowerShell script that provisioned an AAD App to be used by the webapp and an App Secret to keyvault, yet to be used by the webapp; we can deploy the actual code.

So here, to make it short, I haven't used any build definition so I will just deploy a Visual Studio ASP.NET WebAPI project manually using the webapp 's publishing profile.

So, using these three Nuget Packages:

 
<package id="Microsoft.Azure.KeyVault" version="2.3.2" targetFramework="net452" />
<package id="Microsoft.WindowsAzure.ConfigurationManager" version="3.2.3" targetFramework="net452" />
<package id="Microsoft.Azure.Services.AppAuthentication" version="1.0.0-preview" targetFramework="net452" />

I can simply write these bits in my home controller:

 

KeyVaultClient keyVaultClient = new KeyVaultClient(
new KeyVaultClient.AuthenticationCallback(
  async (string authority, string resource, string scope) =&gt;
  {
    var azureServiceTokenProvider = new AzureServiceTokenProvider();
    return await azureServiceTokenProvider.GetAccessTokenAsync(resource);
  }));
SecretBundle theBundle = keyVaultClient.GetSecretAsync(
  CloudConfigurationManager.GetSetting("webapisecretname")).GetAwaiter().GetResult();
ViewBag.Message = theBundle.Value;

Sure enough, with the above code I can extract the value from Key Vault thanks to the AzureServiceTokenProvider class being made available by the AppAuthentication Nuget package and I didn't have to use any credentials to connect to the vault.

Ideally, developers should keep the extracted values in memory to avoid unnecessary rountrips to Key Vault whenever they need to work with these values. On top of this, using MSI not only prevents from disclosing sensitive information, it also unleashes the power of the Vault which is a very good way of maintaining this type of information (certificates renewal, secrets renewal etc.) seamlessly for business applications.

 

Happy Deployments!