Getting started with Unit Test Generation

Welcome

Hi I’m Michael Brisset a Quality Assurance Engineer for Visual Studio Team System. Welcome to CodeGen University! While we don’t have diplomas or faculty (or exams) we do have a pretty good football team. Over the next few months I am going to blog about the basics of generating unit tests, discuss some features to help maximize the value of these tests, and eventually discuss some of the advanced technologies used in this feature like the code model and reflection.

CGEN 101: Creating & Verifying simple unit tests

To begin we need an application. We have several options as Unit Test Generation supports C#, Visual Basic, and C++ /CLR:safe projects. Since my boss loves VB, let’s get started with a simple class written in Visual Basic:

1. Create a new Visual Basic Class Library.

2. Add a Reference to System.Drawing.

3. Paste the following code into Class1.vb:

Imports System.Drawing

Public Class MyFavoriteTeam

    Dim Result As String

    Public Function NextGame() As String

        Return "at Pittsburgh"

    End Function

    Public Function Colors() As System.Drawing.Color()

        Dim SchoolColors(2) As System.Drawing.Color

        SchoolColors(1) = Color.Blue

        SchoolColors(2) = Color.Gold

        Return SchoolColors

    End Function

    Public Function Ranking() As Integer

        Return 1

    End Function

    Public Sub LastGame(ByVal bowlGame As Boolean)

        ' Set to true if the caller wants the last bowl game reported.

        If bowlGame = True Then

            Result = "Loss in the Insight Bowl"

        Else

            Result = "Loss to USC"

        End If

        Console.WriteLine(Result)

    End Sub

End Class

Our application has four important methods and we need to validate their correctness. Rather than write these tests by hand we will use Unit Test Generation to help us get started.

4. Place the cursor on the class declaration.

5. Right-click and choose Create Unit Tests...

6. A tree view will be presented with all of our classes and methods. By default items are checked relative to the position of the cursor in the editor. In this case, since the cursor was at the class level, all methods are checked. Hit the + box next to the Members node to display all of the methods in our class.

 

 

7. There is lots of functionality packed into this dialog. We’re going to leave all of that goodness alone for now and focus on the Output Project control. From here we can choose to generate tests into any language supported by test projects (C#, Visual Basic, and C++). If your solution already contains tests projects they will be listed here as well. For now let’s continue to use VB, select new Visual Basic test project if it isn’t already and hit OK.

 8. Keep the default test project name and hit Create.

 

9. Look over the generated file, Class1Test.vb. We now have four new unit tests and we are well on our way to validating our application! Let’s walk-through each test method, examine what was generated and change them slightly to produce more meaningful tests.

10. Let’s start with the RankingTest. Our class is instantiated, variables representing our expected and actual values are declared and two Assert methods are generated. The first Assert method validates that the expected value matches the actual value. Essentially verifying that the function did its job. The final Assert method simply forces an Inconclusive result, this is a safety-valve that can be removed after the editing of the test method is complete.

We understand this function should return MyFavoriteTeam’s ranking. We had better check espn.com to see what the correct value is.

Hmm… there are three polls listed here. Which one should this function reference? Perhaps the spec is incomplete (or non-existent). I don’t see MyFavoriteTeam listed here. What is the proper behavior in this case? Should the function return 0 or -1? Or should it throw an exception?

Early unit testing will help flush out assumptions like these and can contribute to the definition of feature specifications early in the product cycle. We’ll expect a value of 0 in this case and remove the Inconclusive method. This unit test is now complete.

 

11. Next we’ll take a look at ColorsTest. Since this function returns an array, a simple Assert method is no longer sufficient. We need to validate the entire array, not just a single value! To support this, a different Assert method, a CollectionAssert was generated for us.

 

 

Everyone knows MyFavoriteTeam’s colors are blue and gold. We’ll modify the test method to expect these values:

 

12. Let’s move now to the LastGameTest. This method takes a parameter, which unit test generation will declare for us. There is also a TODO comment generated, reminding us to declare the variable appropriate for the test we want to author.

Notice the new text in the generated Inconclusive call. Unit Test Generation is reminding us that correctness for this method cannot be verified through a return value. Methods with no return value will still fail if an exception is thrown.

Only two small changes are necessary for this test. We’ll initialize bowlGame to True and modify the TODO comment to remind us to test the opposite condition in the future as well.

We will also keep the generated Inconclusive statement in this case since we are validating the function didn’t fail rather than explicitly validating correctness. In a later course we’ll validate the correctness of this method using unit test generation’s accessor model and remove the Inconclusive statement.


 

13. The final method to update is the NextGameTest.

 

Unit Test generation gave us a very good start. All we need to do is modify the expected value and remove the Inconclusive statement.

 

14. Now to the really fun stuff, let’s run our tests and see how our application (and unit tests) are performing.

            Test -> Window -> Test View.

Select all the tests and hit the run button.

15. The Test Results window will come to focus and report the results of the tests. 1 Passed and 1 ended with Inconclusive as expected. But 2 tests have failed. Looking closer at the error message column gives a pretty good indication of what went wrong.

 

16. Troubleshoot. The Ranking function is returning 1 in cases when MyFavoriteTeam is unranked. That looks like a bug, and is confirmed by looking back at our implementation. Perhaps it was too optimistic to always assume MyFavoriteTeam is ranked #1? J

The results from the NextGameTest is more subtle. The data looks correct, but is simply in the wrong format. Perhaps we can change the Assert method to fix this…

Summary

Wow, we've covered a lot! To recap we have:

  • created our application.

  • generated unit tests for all of our application’s methods.

  • created a new test project.

  • discovered a design issue and application behavior that needs to be better defined.

  • used different Assert methods to validate application correctness.

  • modified parameters to functions and expected values to create meaningful tests.

  • executed tests through Test View.

  • analyzed results in the Test Results window.