Test planning using Camano

The concept of a test plan

Just like architects plan and document the design and requirements of software products, and just like developers plan and document how they will be implementing it, testers also plan and document their strategy to verify that such products meet those specifications and requirements. This documented strategy created by testers is called a test plan. Great test plans are those that are able to effectively verify the quality of a product (given the requirements and specifications) through the use of effective manual and/or automated test cases.

Test plans in Camano

Camano is the internal codename for the test case management and test runner solution that we have been working on as part of our next major release (Visual Studio Team System 2010 Test Edition).

Test plans in Camano fit the definition that we already talked about, by providing testers the tools they need to document their strategy. But it also go beyond that - letting them organize, execute and track the progress of the different artifacts on their plans (test cases, configurations, runs, etc). This is how a test plan looks like:

I will throw in some questions for you along the way. Are these test plan properties (name, description, iteration, etc) shown here enough? Leave a comment if you think there are key attributes missing?

When and why does it make sense to use a test plan?

A test plan makes sense whenever there is an application, specific feature, or specific goal (such as testing localization) within a context of an application. Test plans in Camano will help test managers track testing from its initial design to its completion, and will help testers track their test cases, manual and/or automated runs, analysis, and defects.

Test design: adding/organizing test cases to/on a test plan

The testing design starts whenever testers start thinking and documenting test cases that will help them uncover defects throughout the whole lifecycle of a product. Camano enables this design process in a very simple way.

A tester can add existing test cases (maybe there are test cases that can be reused from other projects or features) or just create new ones on any test plan. These test cases can also be grouped in what we call static or query-based suites. This is what an organized test plan with test cases could look like:

As you can see in the left pane, suites are used to organize tests (which appear in the right pane), and progress on the test plan can be seen right there as well. Does this seem easy and straightforward to use?

Test cases are ready to be run either manually or in an automated fashion right away, and the progress of plans can be tracked in a straightforward way as well. A usually complex task turned into a simple and fun effort!

Test organization
Test organizations can be as small as one tester doing the whole job, and up to tens or hundreds of people working on a project. Roles go from test managers, to test leads, and individual testers. Camano makes it easy for the whole team to work together, and at the same time enables each person on the team to focus on stuff that he/she needs to get done.

Test managers or leads will be interested in knowing how the testing effort is going. They might want to know, for example, how many test cases exist for a given feature, and how many have been completed so far. They might also want to know the owner for each of these test cases. They can easily view all this information from any given test plan:

Testers may want to track planned test cases, and may want to filter based on those that are assigned to them. They may also want to specify different configurations for them. Camano will let them do all these and more…

How to know when testing is done?
Saying that we’re done testing is never an easy task. In order to have the right answer, the correct test design will need to be figured out, and the actual implementation of that test design will need to be completed. Camano will not test for you, but it will give you the right and simple-to-use tools to accomplish this goal: Easy test design, easy tracking of the effort, and easy reporting on the status of test cases; all these things right out of the test plan. If the right design is in place, the answer will be a straightforward one: “We’re done testing”.

What else is there in Camano?
This post has focused on test planning; but there is much more in Camano. As soon as test planning is done, execution is what makes a test plan look good. Camano makes running tests a simple and efficient task. Afterwards, there is a whole integrated story that includes analyzing runs, linking and tracking bugs, viewing reports, looking at recommended tests to run on new builds, running tests remotely and collecting data such as code coverage, and even creating test environments that testers can share with their developers. Are we missing any features that you as a tester think we have to ship with this version?

When we go out to the field and demo the product to customers, we’ve heard that they love the whole integrated story that lets testers easily test a product and create bugs against it; also the fact that developers get all the information they need in order to be able to repro the bugs quickly and fix them accordingly.

Stay tuned for further blog posts describing all these features in Camano that integrate the tester and developer stories together!

Can’t wait for future blog posts?

Then go ahead and try the latest bits! You can get them here:
https://connect.microsoft.com/VisualStudio/content/content.aspx?ContentID=9790

Please let us know if you love our product, and if you think we’re missing anything.