VS2010 Tutorial: Testing Tutorial (Step 2)


Last week I posted the Step 1 tutorial for creating a WPF application from scratch using Visual Studio 2010.  In this post, I’m going to create a test plan and test case for the application using the new Visual Studio Test and Lab Manager project.

For this tutorial, I’m using Visual Studio Team Suite 2010 (which includes all of the roles and TFS access).  I’ve already added the demo to TFS so I have full source control.  For the sake of demonstration, I’ve commented out the final fix from the walk through so the label does not update:

image

When I run the application, the label is not updated:

image

The Tester

To create my test, I’m going to run the Test and Lab Manager tool from the start menu:

image

The main page has tabs for test plans, tests, and for tracking work items:

image

First I need to connect to my TFS server by click Add.  My server is VLM13267036 (auto generated name by our internal Hyper-V testing tools):

image

I’ve already got a collection with my code stored in the Projects folder:

image

Next I’ll select Projects and choose the Connect option. This prompts me to set a context:

image

I’ll choose “Set context” which brings up the editor for my new context:

image

I’ll select New:

image

I can now edit all of the properties:

image

After all data has been entered, click Save and Close.  The new item is now in our list:

image

Double clicking the item allows me to add a new test case to this test suite:

image

Here you can fill out all details for the test case:

image

Steps can now be added by clicking on the “Click here to add a step”:

image

I’ve added a few steps including launching the application, hitting the buttons, etc:

image

Now hit Save and Close to go back to the test case list.  The Plan is now complete.  We can run it any time a new build is produced, for each flavor of build, for different configurations, etc.  To execute the test, change focus to the Test tab:

image

Our test plan and test case are already in the list.  Right click the test case and select Run:

image

This will launch the test case.  The manual test runner window docks itself to the left side of my desktop so I can see both the steps I want to run and the full Windows desktop:

image

The “Start test and record” option allows me to not only do the testing steps, but it will also allow recording a WMV of all the steps I do as well as recording my steps to help me author coded UI tests (big helper with automation).  This is really handy if you want someone to see exactly what you did to produce a bug and automate testing in the future. 

In this case I will select “Start test”.  Notice the Test Runner now shows the steps I created above:

image

The first step is to “Launch the PicViewer application” which I’ll do by running the application.  Since that worked, I’ll press the combo box status item behind the step and select ‘Pass’ from the drop down:

image

The item is now marked as passing:

image

I’ll repeat the process for the next two steps, so far so good:

image

When I get to my last step, I discover the file path isn’t actually set.  That makes this item a failure.  Select the drop down box and choose ‘Fail’ from the list.  I’m automatically asked for comments on the failure:

image

Since I didn’t record a video of my steps, I would like to give the developer a screen shot of what went wrong.  Select the camera tool bar button:

image

This will bring up a capture tool turning the cursor into a cross hair that allows me to select a region of the screen.  I’ll select the top of the application to demonstrate the busted label:

image

Notice that the failed test now has a .png file added with the image:

image

I’ve got enough supporting data now so I’ll create a new bug using the toolbar:

image

I’m now prompted to create my bug with a description, etc:

image

Notice the detailed test steps I’ve taken have already been added to the bug:

image

As has my screen shot (the .png file):

image

I’ll now do a Save and Close which will commit the bug to TFS as a Work Item.  Finally I’ll do End Test then Save and Close the test runner.  This will return us to Test and Lab Manager.

image

as a tester, I could now double click the test case and see all of the same data I just saved for the failure:

image

I can also select My Bugs and see the bug filed for this issue (since I conveniently assigned it to me):

image

And just to show how everything is wired together, I can open Visual Studio, Team Explorer and look for bugs assigned to me there as well:

image

At this point my job as a tester is now done. 

The Developer

As a developer, I can now open the bug and read through the issue.  If I select Other Links I’ll find the .png which I can open to see the issue:

image

Sure enough, the label is not updated.  If a WMV had been recorded, I could have actually watched the testing steps in action.  Because the bug is quite simple to find and fix (some idiot commented out the update line!) I can simply make my fix and test it.

Now that things are fixed, I want to check in the bug fix and resolve the work item at the same time.  Click on the Pending Changes tab in VS and select the correct work item:

image

Now we can Check In the fix:

image

I can now verify the bug has been Resolved:

image

In Summary

A key goal for Visual Studio Team System 2010 was to reduce the number of times a tester and developer wind up in a ‘no-repro’ situation.  There are several things I’ve demonstrated in this tutorial which help:

  • Test case steps are documented and set up for repeatable execution
  • Pass/Fail steps are outlined and stored in bugs automatically
  • Video capture is allowed to see all steps taken, and screen snapshots are easy to acquire and file with a bug
  • System information including build number, OS, etc are recorded for you (System Info tab)
  • Although not shown, I could also have collected all of the historical debugging traces from the run as well
  • All data from test cases, results, work items, and source code are kept in TFS and can be shared by test and dev

I hope you’ll pick up Beta 1 and try this set of tutorials for yourself.  Let us know how well it works for you and if you have any suggestions.  I should also point out the work item tracking, auto resolve, etc are all part of VS 2008 so a great way to get prepared for the new version is to get TFS deployed today and get your projects into the system.

Enjoy!

Comments (11)

  1. Thank you for submitting this cool story – Trackback from DotNetShoutout

  2. SSP says:

    When adding the Team Foundation Server, why does the url get "/tfs" automatically added to it?

    This essentially prevents me from adding the TFS server. I’ve even tried adding the server by typing the full url (http://servername:8080), but with no luck.

    I keep getting the "Unable to connect…" message.

  3. Isaac says:

    Really excited about this – using TFS 2008 now but seems like VS and TFS 2010 are going to kill more of our ad-hoc spreadsheets and make our processes so much more consistent and easier to manage!

  4. jk says:

    @SSP: I had tried the same with our TFS 2008 and it resulted in the same error. So now I’m installing TFS 2010 in the hope of these will go together.

  5. Jason Barile says:

    @SSP – Are you trying to connect from VSTS 2010 to a TFS 2008 or TFS 2005 server?  In TFS 2010, we have introduced the concept of a website path for the server, so that’s why you’re seeing the default "tfs" getting added.  In the "Add Team Foundation Server" dialog, you should be able to just delete the text in the "Path:" textbox (it says "tfs" by default), and you should be able to connect just fine.  If you’re still having problems, could you please post in our forums here?  http://social.msdn.microsoft.com/Forums/en-US/tfsprerelease/threads

  6. jk says:

    JasonBa@ – Connection from VSTS 2010 to TFS 2008 is working fine just after typing url to TFS directly, like: http://tfsserver:8080/

    TFS 2008 is then seen as "(legacy server)".

    Problem occurs when I try to connect from Microsoft Lab And Test Manager (Camano) to TFS 2008.

  7. SSP says:

    @JasonBa – I echo what jk said in his last post. The problem only occurs when connecting the Lab and Test Manager.

    @jk – I downloaded TFS2010 last weekend so it’s now onto locating a virtual machine to install on 😉

  8. Cameron Skinner on 10-4 Video on Beta 1 Installation and Visual Studio 2010 and DSL Toolkit SDK Beta

  9. Last month (May 2009) Microsoft has released its first beta for Visual Studio Team System 2010 and Team

  10. Last month (May 2009) Microsoft has released its first beta for Visual Studio Team System 2010 and Team