Hello everyone – Please find some answers to the most common questions that we have been getting asked below. We will try to keep this updated over time.
Q: What is the difference between what Microsoft Test Manager offers and Record & Playback?
A: Test Manager’s record/playback is intelligent and robust. We try to figure user intention based on actual captured input to reduce dependency on specific location of controls, etc… Focus is on helping manual tester accelerate through some already recorded steps.
Q. Can testers view and modify code created by the recorder?
A: While the action log created by the recorder cannot be modified, you can create a Coded UI test (either C# or VB.Net code) which can be viewed or modified in the Visual Studio IDE. We made the rerecording of the actions easy enough that we don’t expect you to edit the recording.
Q. How does the “assisted manual testing” react to UI whose properties are dynamically generated?
A: This should work fine as long as the generated properties are deterministic. Here is a blog that enables you to control search properties.
Q. How easy would it be for a developer to re-use the rich bug to test the fix?
A: This is fairly straight forward. If the test has been automated, the developer can open the associated test case, choose to run the test case and playback the recorded actions against the fixed application.
Q: Would you recommend leveraging the UI automation created in Test Manager in a performance test?
A: While UI automation can be used in a performance test, that would not be our recommendation since you won’t be able to drive significant load using a UI tool that will bottleneck at the UI layer. Instead you might be better off using Web Performance Tests to drive the server load without the UI dependency.
Q: When you convert test actions to code, what languages do you support?
A: We generate either C# or VB.Net code from the recorded actions.
Q: Can you parameterize your automated tests to run for multiple sets of data?
A: Yes, you can parameterize your tests to be run for multiple sets of data. If you like, the data stored on a parameterized Manual Test can be bound to and leveraged by it’s automated counterpart.
Q: Can you drive your automated tests from data in a database?
A: You can bind your automated tests to a database to run them for multiple iterations of data. Please refer to this article for more details.
Q: What level of support do you provide for code coverage?
A: Users of Visual Studio are able to see the code covered by their automated tests in Visual Studio. Unfortunately Code Coverage information is not currently collected when Manually Testing.
Q: How does this tool compare to Selenium and other open-source testing tools?
A: Our offering engages testers in the overall ALM effort through integration with lab management and TFS for work-item tracking, source control, build management, and reporting. Many of the open-source testing tools are stand alone and tend to focus on a particular platform or type of testing (e.g., Selenium is specifically for browser-based testing of web applications).
Q: What are the key benefit this tool provides over HP’s QC/QTP?
A: We feel that Visual Studio Test Professional 2010 is optimized for the way that testing is performed today where more than 70% of testing is done manually through ad-hoc tools such as Microsoft Excel and Word. It is an integrated testing toolset that delivers a complete plan-test-track workflow while allowing testers to collaborate with developers effectively and test efficiently.
Q: Does this tool integrate with Office Project?
A: Yes, via TFS. Office Project can be used to manage data stored in TFS (such as dependent tasks) to facilitate planning projects, scheduling tasks, assigning resources, and tracking changes. By using Office Project, you gain access to features such as a project calendar, Gantt charts, and resource views.
Q: Does this tool integrate with Office SharePoint?
A: Yes, via TFS, which enables creation of a team project portal associated with a SharePoint site. The project portal includes a customizable set of dashboards (SharePoint Web parts) that enable team members to monitor project data in the form of PivotChart reports and lists or counts of work items. The testing tools are also integrated with SharePoint in the sense that they can be used to test SharePoint sites.
Q: Does this tool have the capacity to integrate with other testing systems such as Rally, QTP, or QC?
A: Not directly; but if you can export test cases, bugs, and other artifacts from the other system into Office Excel, you can then import them to TFS as work items. You can use the Test Case Migrator Tool (available at http://tcmimport.codeplex.com/) to import test case steps and parameterized data along with your test cases.
Q: Does seeing the line of code that failed require a special type of build or deployment?
A: No, no special build or deployment is required. A tester must however configure to capture intellitrace data on the machine which is running the code while they are testing, this is done via the Test Settings.
Q: Will the developer be able to see the line of code for web-based development where the code sits on the server and the test is executed via client browser?
A: Yes the developer will be able to see the line of code that failed, so long as the tester configures the webserver to capture intellitrace logs.
Internal Use of Test Professional
Q: Were these test tools used during the development of Visual Studio 2010 or any other MSFT products?
A: Absolutely. The entire Visual Studio team has used this toolset to build Visual Studio 2010. There are many other teams at Microsoft which are using them as well.
Q: Has lab management released with VS2010 on April 12th?
A: Visual Studio Lab Management 2010 is currently available as a release candidate. You can try it out right now, it has yet to be officially released
Q: Are you planning to have host a webinar on lab management?
A: We plan to have a webinar around our Visual Studio Lab Management 2010 product around its release. Please stay tune for announcements from SQE later this Fall (Q3CY2010).
Q: How much does Test Professional Cost?
A: Visual Studio Test Professional 2010 with a one year MSDN subscription retails for $2169. Your organization may already have volume licensing with us. For details of licensing, please refer to this licensing white paper.
Q: What are the licensing details of Test Professional?
A: The ins and outs of licensing for Visual Studio family of products are spelled out in this licensing paper.
Q: Do you have anything to assist with Load or Performance Testing?
A: Yes, since 2005 we have shipped tools that allow users to load test their application. This blog post is a good overview.
Q: Can you load test using Test Professional?
A: Load testing must be performed through use of Visual Studio 2010 Ultimate.
Q: Can I migrate existing test cases authored in Excel to Test Manager?
A: You can use the Test Case Migrator Tool to convert test cases authored in Excel to Team Foundation Server work items that can be used in Test Manager. The Test Case Migrator Tool can be found at http://tcmimport.codeplex.com/
Q: How do I migrate my manual test cases from previous versions of Visual Studio to be run in the current version of Visual Studio?
A: Please use the Test Case Migrator Tool found at http://tcmimport.codeplex.com/ to migrate old Test Cases to the current version used in Visual Studio 2010.
Q: Can I migrate existing test artifacts present in Quality Center to Test Manager?
A: You can migrate your test cases from Quality Center by exporting the test cases in Quality Center to Excel, and then using the Test Case Migrator Tool to import the Test Cases to Team Foundation Server. The Test Case Migrator Tool can be found at http://tcmimport.codeplex.com/
Q: Can you provide some information about the platform support for Microsoft Test Manager? What class of applications can I test with the tool?
A: You can use the tool to manually test almost any platform, though your experience around the creation of rich bugs and support for automation will be better if you are targeting IE7/IE8, Windows Forms, WPF, Win32, MFC or SharePoint. This blog post has an updated enumeration of the automation platform support and will be updated over time as we add additional support.
Q: Does Test Professional have requirement tracing built in?
A: Requirements are treated as first class citizens in Test Professional. One can see full traceability of Requirement -> Test -> Code. One can also base a test suite off of a requirement. This blog post speaks very extensively on the topic.
Q: Can a user produce a Requirement traceability report using Test Professional?
A: Test Professional provides full requirement traceability. Out of the box, users can see a requirements traceability report on their team dashboard.
Q: Can I email a defect in Visual Studio 2010?
A: Yes, you can email a link to a defect from Visual Studio.
Q: How can I link Bugs to Test Cases? How can I link bugs to Product Backlog items?
A: Defects, test cases, backlog items are all work items on Team Foundation Server. Test Professional automatically links the work items together when appropriate (for example, when a test creates a bug, the test and the bug are linked together). Users can also manually link work items together in Test Manager or inside Visual Studio.
Q: Can a tester send a video of their testing to a developer?
A: Yes a tester can send a video of their testing to a developer. When a tester creates a bug, with background video recording on, the video of the testing which caused the bug is referenced on the bug form which the testers assigns to the developer. If you wish, you can save the video which is stored on the test result to a network share as well. Demo 2 of the webinar demonstrated the rich bug which is accessible to the developer.
Q: Is there any way to track testing across a variety of operating systems?
A: By setting up a configuration for each operating system, you can break down your testing for each operating system. To read more about configurations, please refer to this blog post.
Q: What does Analyze Test Runs do?
A: Analyze Test Runs allows you to review the results of your automated and manual test runs. For more details, please refer to this blog post.
Q: Can we access Test Manager via the web?
A: Test Manager 2010 installs as a client tool. When it launches, you are presented with the various projects on TFS which you can connect to.
Q: We have a large number of testers, can a test manager assign test cases to individuals
A: Test managers can assign test cases to individuals. Testers can also pull from a pool of test cases. Test case assignment is done in the Test Plan Contents, it can be done by clicking on the assign button.
Q: Is Test Manager available as a plugin for Visual Studio 2008?
A: No, Test Manager is only available via Visual Studio Test Professional 2010 or Visual Studio Ultimate 2010
Q: Does Test Professional support the branching of Test Cases?
A: Test Manager does not support branching of work items. However you can import test cases from one plan to another so you would be able to use the same test case across code branches.
Q: Is it possible to work with components of tests being shared between test cases?
A: Yes, shared steps can be created once and shared between test cases. They are really powerful. Please refer to this blog post to learn more about them.
Q: Are test steps and expected results which span multiple lines supported?
A: Yes, You can enter multiple lines for the test step and the expected result. The text wraps in both the test runner and while authoring the steps.
Q: Is Test Professional compatible with Agile Methods?
A: Yes it is compatible with Agile Methods. We would suggest that you use 1 test plan for a sprint of work and 1 requirement based suite for each user story. This blog post describes our suggested approach.
Q: What information does recommended tests provide?
A: Recommended tests provides a recommendation of the tests to re-run based upon code which a developer has recently checked in. This blog post describes the feature thoroughly. This one does as well.
Q: When I’m editing a test case in MTM, the steps / parameters section of my work item is changing height, or is very small. This is driving me crazy!
A: This is a known bug, and we’re looking at addressing it in a future release. For the moment, there is a work around: Adjust the height of the window. This should make the issue go away.
Q: How can I find all the test cases that I have action recordings for, so I can create recordings for them?
A: Today, there is not an easy way to do this. The recording is stored on the Test Result, and it is not possibly to query between Work Items, and test results. It’s important to note that with Fast Forward for Navigation, the expectation is that people will record, and update, their action recordings as they are performing their normal manual testing.
Q: How do I view the results of more than one query?
A: Just click the ‘queries’ link at the top of the application again. This will open a new Queries view, and allow you to have two queries open. You can find the other Queries you have open in the ‘Open Items’ drop down.
Q: How does Recommended Tests work?
A: When the Test Impact Data Collector is turned on, Test Runner records the code paths taken by a Manual or Automated Test Case when it is executed. Recommended Tests recommends test for execution when a developer makes a check in that touches the code covered by the test case.
Q: Do applications need to be Debug builds to track the lines of code hit by the test?
A: No applications do not need to be debug builds to capture Test Impact Data. The Test Impact Data Collector must just be turned on while testing.
Q: What is the difference between Test Impact Analysis and Code Coverage?
A: Code Coverage is a report which can be generated which speaks to the lines of your code under test which have been covered by your automated testing. Test Impact analysis collects the lines of code which your Tests cover to report which Test Cases are suggested for you to re-run based upon code churn.
Q: Can I use Test Professional without a TFS 2010 Server?
A: No, Test Manager requires a connection to a TFS 2010 Server.
Q: Can I use Test Professional 2010 with a TFS 2008 Server?
A: No, Test Manager requires a connection to a TFS 2010 Server.
Q: Is there a Microsoft online virtual environment to test the product without having to download a trial version?
A: Unfortunately, there is not currently an online virtual environment to test the product.
Q: Is TFS 2010 available as a trial?
A: Yes it is available as a trial. Please use this link to download it.
Q: Who should I contact for a thorough training on Test Professional?
A: We recommend that you contact our partners for hands-on product training. For example, RTTS is a New York based (NYC) partner who specializes on test training as well as services on the Visual Studio platform and tools. Notion Solution and Northwest Cadence, based in the Dallas area and Seattle area respectively, are partners who deliver training as well as services around the country focusing exclusively on Visual Studio Application Lifecycle Management.
RTTS’s courses can be found here.
Notion Solutions’ courses can be found here.
Northwest Cadence’ courses can be found here.
Q: Where can I see video of the Webinar entitled Visual Studio Test Professional: The Tool for the Modern Tester?
A: A video recording of the presentation can be viewed at this link.