DevOps Lab for Microsoft Services

Friday I am delivering a DevOps lab to Services and thought I would place it on my personal blog so they can refer to it later.


  1. Creating Test Environment with Dev/Test Lab
  2. Running a Load Test with Azure Performance Test
  3. Creating the Continuous Integration Build
  4. Creating and adding a Build Agent
  5. Creating the Continuous Deployment Release Pipeline


Lab 1: Creating Test Environments with Azure Dev/Test Lab

You can provision environments within allocated quotas and virtual machine (VM) size policies, and apply automatic shutdown rules for idle VMs and environments. The service also lets you track projected spending against budgets and analyze Lab usage in detail. Please see this page for more information: 

0.  Get an account from Chuck. The accounts we will be using are all AAD backed and will have the names. Happy to discuss why we are doing it this way



1. Start the new Azure Portal with the feature flag switch. 

Note this only works with subscriptions that have been enabled!  Access to Azure Dev/Test can be requested here:

Do not use the feature flags on subscriptions that have not been enabled. or much easier you can use the shortcut which takes you directly to the Dev/Test Blade.



2. Create a new Lab for your environment by hitting the green + sign


3. Set the parameters for the lab. Please use:

  • Your email alias for the lab name
  • The MSService Resource Group
  • image

    4. Create a lab VM Please use:

  • Visual Studio 2015 Enterprise Base
  • Change the size to A2
  • Add the Chocolatey artifact
  • image

    Use Chocolatey to install Azure Power Shell…The package name is “Azurepowershell”


    In many workflows you would also add a VSO build in the artifacts section (see image below) but in this case we have a chicken and the egg issue in that you haven’t created build yet.

    In future versions there will be build tasks to create Lab VMs as part of a Build or Release.   Future versions will also have facilities for controlling spend and resources like Jason Zander showed at AzureCon.


    Congrats you have completed the Dev/Test Section of this Lab

    Lab 2: Running a Load Test with Azure Performance Test

    Enabling your operations team to easily reuse the performance tests created by the developers to ensure SLA is a key benefit of Azure Performance Tests(coming in next couple of sprints).

    Note In the Release Pipeline lab we will also show how to reference a Load Test as part of the automated Build and Deployment labs. 

    Steps to use the feature:

    1. Navigate to the Web Performance Blade

    Performance testing tool is found from the Tools menu of any Web App eg Browse > Web Apps > PartsUnlimitedWeb> Tools > Performance Test

    2. Select your Visual Studio Online account. 

    If you don’t have one, creating a Visual Studio Online account is free of cost and takes less than a minute.

    The Visual Studio Online Account maintains the history of the past performance tests you have done.

    3. Set the properties a new Performance test,

    Note while the URL is prepopulated you can type in any address.


    4. Verify the Web App is meeting the SLA.

    Check the errors and response times of the application in question and verify it is not experiencing any performance issues.




    Lab 3. Creating a Continuous Integration Build

    For setting up your own Visual Studio Client to build the PartsUnlimited Project please see:  Getting started with the standard steps for getting started with GIT

    Directions Creating the CI Build

    Step 1.  Create the Continuous Build. Log into a Visual Studio Online Account with the user account given to you at the start of the lab i.e.  and select the MSService team project.


    Step 2. Navigate to the MSService team project by clicking the “Browse” link.


    Step 3. Select the “Build” Menu.

    NOTE Build.Vnext and Build has been merged you will NO longer see the build Build.Vnext as a separate option.

    The legacy builds are now called XAML builds and new builds are simply called just “Builds”.

    Step 4. Create a new Build. Click the Action button (Green Plus) to create new build definition. 

    Please name this with your email address so you can find it later


    Step 5.  As you can see, you can now do Xamarin Android/IOS and Builds as well as Xcode builds. Click on Empty Template under the Build tab on the Definition Templates dialog.



    Step 7. Set the repo to build

    As we may have multiple repos and branches, so we need to select the correct Repo and Branch before we can select which Solution to build.  If you had created a branch Click on the Repository tab and Select the repository you created above. If you didn’t create a branch just select Master.


    Step 6. Add the task to Build the Visual Studio solution

    Click the add build step > select Build > and choose the “Visual Studio Build Task

    While we could add all kinds of tasks at this stage we will wait and do that in the Release Management Lab.


    Step 7. Select the Solution from Version Control.

    Now we’re going to select the solution we’re going to build.  Click on the Build tab and Click the ellipsis in the Build Solution pane and select the PartsUnlimited.sln file.


    Step 8. Configure the build definition to build the Publish package. This is done by specifying an msbuild argument

    /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.stagingDirectory)"



    Step 9.  So that the Build fires off every time there’s a check in, enable the Continuous integration trigger. You can select which branch you wish to monitor, as well. On the trigger tab of the build definition select “Continuous Integration”.


    Step 10. Save the build with a name containing your email address that you will remember


    Step 11. Test the CI Trigger.

    Open one of the files in the PartsUnlimited Project and edit it…in this case to ensure we don’t break the build we modifying the

    (Obviously this can also be done in Visual Studio!)


    Step 12.   Check in the change

    From Visual Studio Online simply click save…

    Step 13. Verify the build was started.

    You can see the Build steps and the details in this panel. Double click on the queued build.

    Monitor the build process and confirm that it completes successfully.  If you don’t see you Build being initiated



    Step 14. Why don’t I see my build or why is my build failing?

    Extra credit for the first person to tell me why didn’t Build start.



    Lab 3: Adding a Build Agent

    While sharing a build agent certainly makes it easy to get started but for performance and flexibility reasons you will more than likely want to run your own build agent

    (and the time of this lab it was required for Release Management)

    Step 1. Open the connection to the VM

    In the first lab we created a virtual machine.  We are going to use this machine to host our build agent.



    Step 2. Log into the Visual Studio Online Account. 

    From within the virtual machine; log into a Visual Studio Online Account


    Step 3.  Open the Visual Studio Online Control Panel

    Click on the gear in the MServices Project




    Step 4. Navigate to the Build Pool administration

    Navigate to the root of the control panel and select “New Pool”



    Step 5. Create new Build Pool

    Please give it your email address



    Step 6. Download Build Agent into the Virtual Machine.

    Download the agent and extract it into the root of the Virtual Machine C:\ Drive


    Step 7. Install The Build Agent

    Navigate to c:\agent and run

    Make sure you set the server name to:

    Change the build Agent pool to the one you created in the previous step


    Update the CI build to be used from Release Management

    Step 1. Edit the CI Build to Use Pool just created


    Step 2.  Edit the build Task to create the web package




    /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.stagingDirectory)"

    Step 3. Add the publish Build Task

    Release Management expects to interact with builds via Build Artifacts


    Step 4. Set up the publish build task to be used with RMO

    Set the artifact name to “drop”

    Set the type to Server


    Step 5. Manually start a build

    As we will be referencing the output of this build in the next lab start a new build.


    Lab 4: Creating the Continuous Deployment Release Pipeline with Visual Studio (online) Release Management

    Now that we have our CI build working and creating the Web Deploy package. It is time to get set up continual deployment in a release pipeline.  Note: at the time of authoring this document Release Management was not available publically in VSO….you will have to use  shared account

    Step 3. In Visual Studio Online click on the Release Hub. Note: if you don't see you "Release" in your Visual Studio Online account it is due to the fact it is still hidden with a feature flag and haven’t logged into the PartsUnlimited VSO Account.


    Step 2.  Create a new release definition

    Click the Action button (Green Plus) to create new release definition. Select the empty template for this lab.

    Please name this with your email address as it is likely we will be creating lots of releases!


    Step 3. Create the Release Pipeline Structure

    The stereotypical dev team will have three environments…lets create those.  Click the Action button (Green Plus) for environments twice…giving us a total of three environments.

    Name the environments:

    a. Dev

    b. QA

    c. Prod



    Step 4. Set the environments to use the Agent Pool you created in the previous lab


    Step 5. Associate your release with your build definition created above




    Step 6.  Add Build Tasks

    You will note there are lots of options for deploying your build, copying files, changing configurations running Tests  and giving you insight into your build process as part of the build.  Lets set that up:

    1. Select the "Dev" Environment

    2. Click the Action button (Green Plus) for tasks (see image above) select the "Test" category and add a task that runs all unit tests in the solution on the build controller.


    Step 7. Set up up the Test task

    …while there is no specific actions here it is worth reviewing the options for running tests…

    Step 8. Add the Web App Task for the Dev Environment

    To Set up the Dev environment as part of the release deployment.  Click the Action button (Green Plus) for tasks (see image above) select the "Deployment" category and add a task that deploys an Azure Website


    Step 9.  Set Azure Web Site Deployment task.  For Azure subscription set it to use the parts unlimited sponsorship, set the web site location to be West US, set the website name to PartsunlimitedwebDev and the package to be the package we are created in the CI Lab





    Step 10. Add Approvers to the QA Environment step of the build release.  So that bad builds or content doesn’t show up unexpectedly you can set up approvers.  To do this right click on the environment and choose Assign Approvers.


    Step 11. Add Approvers to the Pre-deployment stage of the QA –Environment.

    NOTE: Make and sure and the account you are logged in with or you will have to reassign the approver to complete this build


    Step 12.  Add task and go to the "Test" category


    Step 13. Add Load Testing to the QA Environment

    Note the Testing settings file is in the root and the



    Step 14. Setup the Test environment as part of the release deployment.  Select the QA environment. Click the Action button (Green Plus) for tasks (see image above) select the "Deployment" category and add a task that deploys an Azure Website


    Step 15.  Set Azure Web Site Deployment task.  For Azure subscription set it to use the parts unlimited sponsorship, set the web site location to be West US, set the website name to PartsunlimitedwebTest and the package to be the package we are created in the CI Lab:


    Step 16. Setup the Prod environment as part of the release deployment.  Select the Prod environment and Click the Action button (Green Plus) for tasks (see image above) select the "Deployment" category and add a task that deploys an Azure Website


    Step 17.  Set Azure Web Site Deployment task.  For Azure subscription set it to use the parts unlimited sponsorship, set the web site location to be West US, set the website name to Partsunlimitedweb (or some other name) and the package to be the package we are created in the CI Lab:


    Step 18. Save the Release Definition

    Step 19. If you cannot save the release make sure the account is in the release managers group.  To add your account to the Release managers group click on the gear.  Click on Security tab.


    Step 20. Select Release Managers > Members > Add and add the account you are logged in with


    Step 21.  Now that all the components of this release pipeline are completed it is time to set it to run on the completion of a build. Click on Triggers


    Step 22. Check the check box for continuous deployment and select the build you created above


    Step 23 As a test it is probably worth verifying your release pipeline will work when started manually.  Click the Action button (Green Plus) to create new release.


    Step 24. This will prompt you which build do you want to run the release on…Choose a build and select "Create"


    Step 25. Check the status.  Open the Release by selecting the hyperlink for that Release.



    Step 26. If you remember earlier in the lab, the release has an approval step in it.  Select the release hyper link you just created above and select "logs"


    Step 19.  Approve the release to the QA environment


    Step 27.  Review the Release


    Step 28. Viewing the test results. 

    Opening the build run you will see the results of the run such as the commits, tags, issues and in data we are interested in –Test Results.  Clicking on this Test results will take you the Test>Runs and opens that run ID.


    Step 29: Viewing the Test run Data.  By selecting the “Test results” menu you can drill down into the test data.  In this case it looks like we may have issues with specific environments/browsers.

    Comments (3)

    1. Hi Chuck!  I noticed that the Azure Performance Test is only accessible from an existing web application's Tools menu.  Is there any other way in which a user could generate a new Azure Performance Test for a given URL from the portal?  If not, are there any plans to open that up?

    2. Hello Josh!

      Yes we are going to add Performance Test to the "New" workflow in the Azure portal.  For now you can actually pin the Performance Tile to the home page which decouples it from Azure Web Apps.

    Skip to main content