Unit Test Generator under the bonnet (hood) … aka vsarUnitTestFx

Once you’ve moved to Visual Studio 2012, what’s a dev to do without the Create Unit Tests feature? Based on the high demand on User Voice for this feature to be restored, the Visual Studio ALM Rangers have introduced the vsarUnitTestFx Visual Studio Extension.

It is time that we talk about this exciting adventure focused on automating test project creation, adding references, generating stubs, and extensibility to add and target other test frameworks.

We embrace the following project principles for the extension:

• Achieves a moderate amount of restoration of Visual Studio 2010 feature functionality, but does not resurrect the old command.
• Replaces and does not resurrect the old command.
• Supports .NET MS-Test and generation of VB/C# test code.
• Presents a "reference implementation" of how to do this for a particular test framework.
• Provides a basic infrastructure and an extension point.
• It focuses on project and reference management and not on code generation.

We envision the following benefits for the Visual Studio community:

• Quickly get to writing test code, not having to worry about setting up projects and files.
• Customize the unit test code generation using a context menu, ensuring that generated tests better match your naming schemes.
• Generate unit test stubs against multiple frameworks so that you can use any framework you like.

As always we continuously dog-food our evolving infrastructure, Ruck process and tooling. We defined and shared our code quality learnings for sprint objectives, the definition of done, code reviews and project principles. These learnings are undergoing constant revision and are included in the latest Ruck Guide on CodePlex for possible re-use by other project solution teams. See ALM Rangers Dogfooding – Code Review and Build process for more information.

Who’s who?

When we refer to “we” and the “team” we refer to a team of competent and passionate ALM Rangers, as shown in the slide I extracted from the recent show-what-we-have presentation.

Visit https://aka.ms/vsarindex for more information on these ALM Rangers.

What is the status?

One down, two milestones to go!

We shipped an ALPHA release to all ALM Rangers and the product group in April as planned, which addressed the first of three main Epics . In May we are planning to release a BETA to an as yet undefined early-adopter community. The BETA  introduces the functionality allowing user to customise the naming of generated code and prepares the foundation for the last Epic as described below.

We have a competent team of developers, testers, pedantic quality guardians and a champion product owner, which means we can all look forward to a phenomenal product. Once we have achieved our definition of DONE we will release version 1.0 on the Visual Studio Gallery.

Epics … our menu

Another slide extract from the recent show-what-we-have presentation show the list of Epics and their status:

Peeking under the bonnet (hood)

Seeing is believing, so let’s switch to the world of Visual Studio and have a look at the extension. The main components of the solution (ALPHA release) are visible in the context diagram below. It is important to re-iterate that this is based on the ALPHA code and will be changing when we address the extensibility Epic.

Generating a dependency diagram of the solution (BETA candidate) gives us a different view of what is hiding within the VS extension:

Walking through some code scenarios

Once installed, you will find the extension under Extensions and Updates in the Tools menu. For this blog post I am using a code quality test project developed by Vladimir Gusarov, which we are using as an evolving playground for our manual test cases.

• As shown in the illustration below we are working in the WeirdFormatting class.
• First we right-click on the class definition.

• We select Generate Unit Test from the context menu.
  
• This generates a test project and a test class—if needed—then adds the references, the namespace, and the test methods.
• You can create additional test methods by repeating the same process within one or more method bodies, which would create additional instances of test methods.

Let’s investigate the resultant Unit Test dish:

1. A test project was created, named CSharpTestProjectTests.
2. References to the project being tested and the quality tools were added to the References.
3. A test method was created in CSharpTestProjectTests for each public method in the WeirdFormatting class in the CSharpTestProjectTests project.
4. The namespace containing the WeirdFormattingTests class is based on the namespace used in the WeirdFormatting class.
5. The test method body contains one statement: Assert.Fail(), which differs from the ALPHA release where the body was empty.

NOTE: There are no plans to support the rare scenario of generating tests against test methods or operators. REF: WI#5815 and #6166.

Mock-ups … what’s next?

To understand what we are planning to include in the BETA, let us look at some of the mock-ups from our product owner. As before it is important to re-emphasise that these are visions, mock-ups and subject to change.

We start with the Create Unit Test Options item which will appear in the Test menu as shown …

The configuration option will allow us to change the behaviour of the Unit Test Generator, for example controlling the class and method name formatting.

Give the mock-up to Anisha Pindoria and the following proposed dialog starts evolving:

To summarise, the BETA will introduce the ability to influence the unit test generator and the last Epic, which will bring us to version 1.0 will allow you to generate unit test stubs against multiple frameworks so that you can utilize potentially any framework.

Thoughts?

We would appreciate your candid feedback, either by adding a comment to this post, or contacting me here.