Lab Management in the overall ALM picture

  • Article

I (Shay Mandel) have recently made a series of customer visits in Israel. I have met R&D managers, QA managers and CTOs from 3 ISVs/Start-Ups, 2 IT organizations (one large insurance company and one governmental department), and 2 MVPs - Guy Kolbis and Leon Langleyben from SRL which is a Microsoft VSTS implementation partner. Guy has also posted about Lab in his blog after our meeting.

It was very good to see that most of the development organizations today either use virtualization as part of their development and testing process, or that they intend to use it in the coming year. This shows that we have a great timing for the product.

I have also presented to the Israel ALM (Application Lifecycle Management) User Group. I have shown how lab can be woven into the overall quality process of ALM. While preparing to it, I saw that we have a lot of places that we can help:

  • By getting developers an environment that emulates complexity of a multi-layer, production-like deployment in a matter of minutes, so that their testing will be much more realistic.
  • Providing an easy to use Library of VM and Environment templates that they can use
  • Removing the hassle of finding the hardware resources to run a VM on, as we introduce the notion of Machine Pools that contain several physical hosts, on which we automatically allocate the best machine to run on
  • By making the Gated Check-in build to run on a clean environment, using our built-in workflow activity for reverting an environment (in this case the build environment) to a checkpoint. If you are not familiar with Gated Check-in yet, you can see a demo of it (and many more cool features of VSTS 2010) on Cameron's Skinner's PDC session: Lap Around VSTS 2010
  • Similarly, we can help to make the Continuous Integration environment more controlled and more similar to the complexity you will find your real life scenario. We can also revert this environment to a clean state in a matter of minutes before deploying the new build and running the integration tests on it
  • Testers do get a lot of attention in our stories. We allow to deploy an environment for every tester in a matter of minutes, be it a single machine or a complex, clustered set of machines that simulate the real production set.
  • Testers are encouraged to take checkpoints when they find a bug. By simply clicking on a button in the Test Runner, a test engineer can capture the state of all the machines which comprise a system. This includes the memory state of the processes, the active windows, the logs and events in the system and more. All this information is captured as snapshots on the hosts, and we provide a simple link to all this setup. When a developer clicks on the link, he can revert to the exact same state, attach a remote debugger and pinpoint the bug, as if it was revealed just a second ago! This is what call Actionable Bugs!
  • Another good use of lab in the testing arena is for testing complex installations and upgrades. The problem with such installation is that after you test once, your system is not 'clean' anymore and you need to reinstall the OS or face the risk that the bugs you will file after the next time you run the install test will not be because of a real malfunction of the installer/upgrader, but rather because your environment is not real. Using checkpoints, you can overcome this. Simply take a checkpoint before the installer runs for the first time, and make it as 'pre-install state'. Then you can always revert back to this state, and it will be a real pre-install state, as it is a copy of the machine as it was at the time the checkpoint was taken.

I got a lot of great feedback which I will try to share in the next posts, after I share it with the team and we decide on what will get into the product. Feel free to add your own feedback and questions by commenting on this post.

If you are interested in hearing more about ALM and Lab Management, especially for Agile development process, I am going to present on the subject: The Next Generation of Agile Software Development - Virtualization for faster development cycles in Virtualization Congress which will be held on May in Las Vegas. Please vote for it and come hear it.
And if you are interested in that, you should also vote for Vijay Machiraju's topic - Technology choices for virtual lab automation. Vijay will share with the audience the choices we had and the decisions we took for how to use the right technologies for implementing ALM processes in a development and testing organization.
Aseem Bansal from our development team will talk about Testing & debugging made easy using virtual lab automation , and will share from his dogfooding experience how bugs can be more easily reproduced and fixed faster.
Please do vote for those sessions if you think they are interesting (just click on the up arrow near the subject name when following the links above).

Shalom!