Connect(); 2016 News for Azure DevTest Labs: Azure Resource Manager template-based environments, VM auto-shutdown and more


Update: Both of the features are now available in all the regions where DevTest Labs is available!

Below is the original post:

Today, at Connect(); 2016, we are very excited to announce that two more new features are available in Azure DevTest Labs: the capability to create your environments using Azure Resource Manager (ARM) templates, and set auto-shutdown for any ARM-based Azure VMs.

Note: Both of the features are available at West Central US region today, and are scheduled to be rolled out to all the regions by the end of this week. We will update this post and publish a Service Update once it’s available everywhere.

In case you haven’t heard about Azure DevTest Labs, this May, we announced the general availability (GA) of Azure DevTest Labs: your self-service sandbox environment in Azure to quickly create Dev/Test environments while minimizing waste and controlling costs. The goal for this service is to solve the problems that IT and development teams have been facing: delays in getting a working environment, time-consuming environment configuration, production fidelity issues, and high maintenance cost. It has been helping our customers to quickly get “ready to test” with a worry-free self-service environment. The reusable templates in the DevTest Labs can be used everywhere once created. The public APIs, PowerShell cmdlets and VSTS extensions make it super easy to integrate you Dev/Test environments from labs to your release pipeline. In addition to the Dev/Test scenario, Azure DevTest Labs can also be used in other scenarios like training and hackathon. For more information about its value propositions, please check out our GA announcement blog post. If you are interested in how DevTest Labs can help for training, check out this article to use Azure DevTest Labs for training.

In the remaining of this article, we will walk you through more details about the new features we’ve just released. You can also see a live demo with more features we’ve shipped recently at our Connect(); 2016 session, What’s New in Azure DevTest Labs.

Create environments with ARM templates

In the past months, we’ve been talking with a lot of customers and listening to their stories and feedback. It turns out that Azure DevTest Labs has provided a very decent experience for users to create a single VM. When it comes to multi-VM environments (e.g. multi-tier web apps or a SharePoint farm), customers look for more productive way to provision and manage those environments. In addition, Azure offers Azure Resource Manager templates, which has been widely used by customers, for you to define the infrastructure/configuration of your Azure solution and repeatedly deploy in a consistent state. In order to address these needs, we implemented this feature that allows you to create multi-VM environments from your ARM templates. Here are a couple of highlights of the feature:

  • ARM templates are loaded directly from your source control repository (GitHub or VSTS Git).
  • Once set up, DevTest Labs users can create an environment by simply picking an ARM template from the Azure portal as what they can do with other types of bases.
  • Azure PaaS resources can be provisioned in an environment from ARM template in addition to IaasS VMs.
  • Cost of environments can be tracked in the lab in addition to individual VMs created by other types of bases.

"TestFullEnv" environment created from ARM templates

Figure 1: “TestFullEnv” environment created from ARM templates

Let’s go through these one by one.

Set up ARM template repositories

As one of the best practices in infrastructure-as-code and configuration-as-code, environment templates should be managed in source control. Azure DevTest Labs follows this best practice and loads all the ARM templates directly from your GitHub or VSTS Git repositories. There are a couple of rules to organize your ARM templates in the repo:

  • ARM templates used for the same environment should be placed in the same folder. DevTest Labs assumes each folder that contains ARM templates represents an environment.

    An example how ARM templates should be organized in folders

    Figure 2: An example how ARM templates should be organized in folders

  • The entry file of ARM templates for DevTest Labs to start with should be named as azuredeploy.json. If you want the Labs to use parameter values defined in the parameter file, the parameter file is required to be named as azuredeploy.parameters.json.

    An example of ARM template files inside the folder

    Figure 3: An example of ARM template files inside the folder

  • You can define a metadata.json file to specify the template name and description:
{

"itemDisplayName": "<your template name>",

"description": "<description of the template>"

}

Adding the repository to your lab is very straightforward using Azure portal. Below are the steps:

  • Go to your lab in Azure portal.
  • Click Configuration in the resource menu on the left side.
  • Click Repositories in the Configuration settings list.
    This will show you the Repositories blade with a list of repositories that have been added into the lab. A repository named “Public Repo” goes with all the labs, which connects to the DevTest Labs GitHub repo with a bunch of DevTest Labs’ VM artifacts out-of-box.

    A list of repositories configured in the lab - Public Repo, your custom repos (FabirkamCentralRepo and FabrikamPrivateRepo) with one disabled

    Figure 4: A list of repositories configured in the lab – Public Repo, your custom repos (FabirkamCentralRepo and FabrikamPrivateRepo) with one disabled

  • Click + Add to add your ARM template repository.
    You will see a blade expanded with several fields that require your input, including the repo name that will be used in the lab, the Git clone URI, the branch name, the personal access token to access the repo (hover over the small info icon next to the label to get instruction if you don’t know how to get the token), and the folder path starting with “/” to specify which folder the lab should use to load the artifacts (the 1st input box in the Folder paths section) or ARM templates (the 2nd input box). If the folders that contain ARM templates are at the root path, input “/” in the 2nd input box.

    Adding a repo to load ARM templates from the master branch

    Figure 5: Adding a repo to load ARM templates from the master branch

  • Once all the required fields are filled in and pass the validation, click Save button.

Now your lab is ready for your lab users to create environments from the ARM templates in that repository.

Create environments

Once the ARM template repo is set up in the lab, your lab users can easily create an environment using Azure portal with the following steps:

  • Go to the lab that is set up with an ARM template repository.
  • Click + Add on top of the lab Overview blade.
    You will see all the valid ARM templates are loaded and listed on top of the Choose a base list.

    Clicking + Add in the lab's Overview blade to create an environment from ARM templates

    Figure 6: Clicking + Add in the lab’s Overview blade to create an environment from ARM templates

  • Select an template from the Choose a base list.
    Once a template is selected, you will see an Add blade expanded with some input boxes. The first input box asks for an environment name that will be used to display the environment in the lab, followed by the input fields for parameters defined in the ARM template. If default values are defined in the template or azuredeploy.parameter.json file is presented, you will see default values in those input fields. For parameters of which the type is secure string, you can use the secrets stored in the lab’s personal secret store.

    Creating an environment using "IIS VMs & SQL Server 2014 VM" ARM template

    Figure 7: Creating an environment using “IIS VMs & SQL Server 2014 VM” ARM template

  • Click Add button to create the environment.
    The environment will start provisioning immediately and show in My virtual machine list. A new resource group will be automatically created by the lab to provision all the resources defined in the ARM template.
  • Once the environment is created, click the environment in My virtual machine list to open the resource group blade and browse all the resources provisioned in the environment.

    Browse resources provisioned in an environment

    Figure 8: Browse resources provisioned in an environment

What about PaaS, policies and more manageability?

There are still several user stories for us to complete a reasonable end-to-end experience for ARM template-based environments. It’s been on the top of our backlog. We will keep working on these user stories and try to deliver them as soon as possible, including but not limited to:

  • Manage VMs in an environment all at once (e.g. start, stop, etc.).
  • Quota limits for environments.
  • Guideline on release pipeline integration.

The current release allows you to provision environments with Azure PaaS resources and track its cost. We are also proactively talking with our customers to identify more policies that will be useful to control the cost for various PaaS resources.

Azure VM auto-shutdown (without creating a lab)

In addition to the ARM template-based environments support, we are also pleased to announce that auto-shutdown is now available to all the ARM-based Azure VMs!

In the past months, we’ve been very happy to see that auto-shutdown is the #1 policy used by our customers. On the other hand, we also learned from quite a few customers that they have their centrally managed Dev/Test workloads already running in Azure and simply want to set auto-shutdown for those VMs. Since those workloads have already been provisioned and managed centrally, self-service is not really needed. It’s a little bit overkill for them to create a DevTest lab in this case just for the auto-shutdown settings. That’s why we make this popular feature, VM auto-shutdown, available to all the ARM-based Azure VMs.

With this feature, setting auto-shutdown can’t be easier:

  • Go to your VM blade in Azure portal.
  • Click Auto-shutdown in the resource menu on the left-side.

    Note: If you can’t see this option, it’s likely the feature deployment hasn’t been rolled out to your region yet. Please try it in a couple of days or watch for an update from this post.

    You will see an auto-shutdown settings page expanded, where you can specify the auto-shutdown time and time zone. You can also configure to send notification to your webhook URL 15 minutes before auto-shutdown. This post illustrates how you can set up an Azure logic app to send auto-shutdown notification.

    Set auto-shutdown for any ARM-based Azure VMs

    Figure 9: Set auto-shutdown for any ARM-based Azure VMs

We hope you find this feature helpful. Meanwhile, the team is also wrapping up the implementation for another functionality about auto-shutdown that has been highly requested by our customers: the capability to set different auto-shutdown time for different VMs running in DevTest Labs, which will be available to you in a few days. If you are interested in trying it, keep an eye on this blog or our Service Updates for its release.

Summary

In addition to these two new features we just released, there are many more since our GA early this year, including attaching additional data disks to lab VMs, and setting target costs and thresholds in a lab, etc. Please check out our Connect(); 2016 session, What’s New in Azure DevTest Labs, to see a live demo how these features work for you.

To get latest information on the service releases or our thoughts on the DevTest Labs, please subscribe the RSS feed of this team blog and our Service Updates.

As mentioned earlier, there are still a lot of things in our roadmap that we can’t wait to build and ship to our customers. Your opinions are valuable for us to deliver the right solutions for your problems. We welcome ideas and suggestions on what DevTest Labs should support, so please do not hesitate to create an idea at the DevTest Labs feedback forum, or vote on others’ ideas.

If you run into any problems when using the DevTest Labs or have any questions, we are ready at the MSDN forum to help you.

Enjoy!

Comments (14)

  1. durga says:

    Hi when created lab in West Central US and vm laso trying to create logicapp with the same region as getting error as Configure the built-in template
    The Logic Apps comes with a Logic Apps Designer. This is where you create an integration with your email client. Below are the steps how you set it up:
    Open Logic Apps Designer. This is supposed to open automatically the first time when you open the app. If it’s not the case, click Edit from the top of the app blade to open it.
    Choose HTTP Request-Response template in the Logic Apps Designer
    added json data to request
    {
    “$schema”: “http://json-schema.org/draft-04/schema#”,
    “properties”: {
    “delayUrl120”: {
    “type”: “string”
    },
    “delayUrl60”: {
    “type”: “string”
        },
        “eventType”: {
            “type”: “string”
        },
        “guid”: {
            “type”: “string”
        },
        “labName”: {
            “type”: “string”
        },
        “owner”: {
            “type”: “string”
        },
        “resourceGroupName”: {
            “type”: “string”
        },
        “skipUrl”: {
            “type”: “string”
        },
        “subscriptionId”: {
            “type”: “string”
        },
        “text”: {
            “type”: “string”
        },
        “vmName”: {
            “type”: “string”
        }
    },
    “required”: [
        “skipUrl”,
        “delayUrl60”,
        “delayUrl120”,
        “vmName”,
        “guid”,
        “owner”,
        “eventType”,
        “text”,
        “subscriptionId”,
        “resourceGroupName”,
        “labName”
    ],
    “type”: “object”
    }
    }
    Click HEADERS in the Response step.
    Select Headers from the listed outputs that are populated from the previous step.
    Click BODY in the same step.
    Select outputs with the info that you want to send through emails, e.g. owner, vmName, labName, skipUrl.
    and tried to integrate with emailclient by selecting office-365 outlook-send an email
    so logic app is saved successfully

    so when i run the trigger while running theapp  in response tab CodeInvalidTemplate
    MessageUnable to process template language expressions in action ‘Response’ inputs at line ‘1’ and column ‘1362’: ‘The template language expression ‘triggerBody()[‘resourceGroupName’]’ cannot be evaluated because property ‘resourceGroupName’ cannot be selected. Please  for usage details.’.
    getting the above error for vmname and labname resourcegroupname

  2. durga says:

    while adding additional data disks to lab it is adding but not visible when try to add existing datadisks to other vm in the same lab

    1. Xiaoying Guo says:

      @durga, thanks for reporting the issue. To help you with better diagnosis, could you please report it to our MSDN forum? https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureDevTestLabs. Thanks!

  3. Hi Xiaoying,

    Great news! Do you have a estimated date to implement auto-shutdown in Brazil?

    Thanks

    1. Xiaoying Guo says:

      @Felipe, it should be available in all the regions where you can create a lab. Please try again. 🙂

  4. GS says:

    What is the purpouse of having Auto-Shutdown without Auto-PowerOn?

    1. Xiaoying Guo says:

      @GS, we ship features at the minimum unit and ship frequently. Auto-start is another feature in our backlog. On the other hand, it depends on your scenario, there’s user case where you auto-shut down VMs that are not regularly used.

      1. Anh says:

        Auto Shutdown but have to manually turn back on.
        what if we have 20 servers. I don’t think MS is not clever to have Auto-ShutDown. It is too much of paint

  5. Todd Bowman says:

    When are RBAC controls coming to the auto-shutdown feature for VM’s outside of dev-test labs? I want to be able to provision and enforce auto shutdown on a VM but let a user control the time and notification options.

    1. Xiaoying Guo says:

      @Todd, we don’t have plan yet to enable that for VMs outside of DevTest Labs. But it’s supported for the VMs within DevTest Labs. Would you considering trying it using DevTest Labs?

  6. Gary says:

    This is “limited” tutorial. Most of the time, we don’t need to create an complex environment as your samples. In my scenarios, we only need to an environment with multiple VMs from custom template in an existing DevTest lab. I don’t need to create virtual network, domain, SQL, web service, etc… like your sample

    Do you have such a vanilla ARM template that spinup VMs as the start up?
    Also, since we’re still TFS on premise, we want to use either from Powershell or “Azure Resource Group Deployment” but not DevTest Lab in TFVS.

  7. Gary Nguyen says:

    Most of the time, we don’t need to create an complex environment as your samples. In my scenarios, we only need to provision an environment with multiple VMs from custom template in an existing DevTest lab. I don’t need to create virtual network, domain, SQL, web service, etc…

    Do you have such a vanilla ARM template that as the start up for these? I think users out there will need to perform this scenarios alot.

    1. Xiaoying Guo says:

      @Gary, you can find quick-start template of creating lab VMs using lab resources (custom images in the lab, etc.) here: https://github.com/Azure/azure-devtestlab/tree/master/ARMTemplates/101-dtl-create-vm-username-pwd-customimage. All the other sample templates of provisioning lab resources (i.e. with resource type starts with Microsoft.DevTestLabs/labs) are here: https://github.com/Azure/azure-devtestlab/tree/master/ARMTemplates. However, the multi-VM environment experience introduced in this blog post doesn’t work with those templates, because the feature is designed and targetted for using the same ARM template across your release cycle from testing to prod, while those quick-start templates provision resources that can only work in the lab. So in order to deploy those template, you need to run PowerShell or CLI instead of using the flow described in this post. If you do want to leverage the flow/experience described in this post, you need to use an ARM template like this one: https://github.com/Azure/azure-quickstart-templates/blob/master/101-vm-from-user-image/azuredeploy.json, but you can point the image VHD location and VNET resource to the ones that have been added in the lab.

Skip to main content