QFEs for Microsoft Test Manager 2010
Over the course of 2010-2011, we have released several hotfixes to improve performance and fix bugs in Microsoft Test Manager 2010.
Here are some of the QFEs that deal specifically with Test Case Management in MTM.
1. Hotfix Rollup Package for Microsoft Test Manager 2010
In October 2010, 6 months after Visual Studio 2010 was shipped, we released a rollup Hotfix package (available here) to address reported issues and to improve performance.
This package included fixes for the following issues:
1.1 Crashes and exceptions
Issue 1 - When you perform operations to work items in Test Manager, Test Manager may crash, and then you receive the following error message:
The specified Visual is not an ancestor of this Visual.
When this issue occurs, the stack trace resembles the following:
at System.Windows.Media.Visual.TrySimpleTransformToAncestor()
at System.Windows.Media.Visual.InternalTransformToAncestor()
at Microsoft.TeamFoundation.TestManagement.Client.WpfControls.WindowsFormsHostEx.GetViewport()
at Microsoft.TeamFoundation.TestManagement.Client.WpfControls.WindowsFormsHostEx.ClipContentToScrollViewer()
at Microsoft.TeamFoundation.TestManagement.Client.WpfControls.WindowsFormsHostEx.Invalidate()
at Microsoft.TeamFoundation.TestManagement.Controls.WpfWorkItemControls.WpfWorkItemTabControl.<>c__DisplayClass2.<InvalidateLegacyControls>b__0()
Issue 2 - When you try to edit an invalid query by using Test Manager, a Microsoft.TeamFoundation.TestManagement.Client.Runtime.VisualizationCannotBeCreatedException exception occurs, and then you receive the following error message:
Failed to create the UI element of following type:
Microsoft.TeamFoundation.TestManagement.Activities.SelectWorkItemsPartView.
Additionally, you have to restart Test Manager to recover from this issue.
Issue 3 - When you try to close a test case dialog box, Test Manager may crash after you select Don't Save in the test case dialog box.
Issue 4 - After you update the Properties settings of the test plan, the data becomes corrupted. For example, the information in the Owners combo box might be incorrect.
Issue 5 - Consider the following scenario:
- You delete a configuration variable. The configuration variable has no value for the Allowed Values field.
- You try to open the configuration in the Test Configuration Manager page.
In this scenario, a Microsoft.TeamFoundation.TestManagement.Client.TestManagementValidationException exception occurs, and then you receive the following error message:
The test variable contains a value that is not valid.
Issue 6 - When you try to delete an iteration of a test case by using the shortcut menu in Test Manager, Test Manager may crash.
Note: If you debug the problem, a System.Data.DeletedRowInaccessibleException exception occurs, and then you receive the following error message:
Deleted row information cannot be accessed through the row.
The following is the stack trace when this issue occurs:
at System.Data.DataRow.GetDefaultRecord()
at System.Data.DataRow.get_Item()
at Microsoft.TeamFoundation.TestManagement.Client.TestCase.HasDataSetChanged()
at Microsoft.TeamFoundation.TestManagement.Client.TestCase.Flush()
at Microsoft.TeamFoundation.TestManagement.Controls.TestStepsPart.set_Dirty()
at Microsoft.TeamFoundation.TestManagement.Controls.TestStepsPart.SetDirtyHandler()
at Microsoft.TeamFoundation.TestManagement.Controls.BaseActionItem.OnSetDirty()
at Microsoft.TeamFoundation.TestManagement.Controls.DeleteIterationActon.Perform
at Microsoft.TeamFoundation.TestManagement.Controls.TestStepsCustomControl.ContextMenuClickHandler()
at System.Windows.RoutedEventHandlerInfo.InvokeHandler()
at System.Windows.EventRoute.InvokeHandlersImpl()
Issue 7 - When you try to assign a build that has an unassigned drop folder to a test plan in Test Manager, Test Manager may crash.
Issue 8 - When you perform an operation in Test Manager, a Dr. Watson error may occur, and then you receive the following error message:
Application: mtm.exe
Event type: Clr20r3
Exception code: system.argumentexception
FAULT SYMBOL:
MICROSOFT.TEAMFOUNDATION.TESTMANAGEMENT.COMMON.NI.DLL!Microsoft.TeamFoundation.TestManagement.Common.KeyedObservableCollection_2[[System.Int32,_mscorlib]]
Issue 9 - When you perform an operation in Test Manager, a Dr. Watson error may occur, and then you receive the following error message:
Application: mtm.exe
Event type: Clr20r3
Exception code: system.exception
FAULT SYMBOL:
MICROSOFT.VISUALSTUDIO.TESTTOOLS.MANUALTEST.COMMON.DLL!Microsoft.VisualStudio.TestTools.ManualTest.Common.TestPointCollectionViewModel.set_ActiveIteration
Issue 10 - When you perform an operation in Test Manager, a Dr. Watson error may occur, and then you receive the following error message:
Application: mtm.exe
Event type: Clr20r3
Exception code: System.ComponentModel.Win32
FAULT SYMBOL:
MICROSOFT.VISUALSTUDIO.QUALITYTOOLS.COMMON.NI.DLL!Microsoft.VisualStudio.TestTools.Common.SingleInstance.RunningObjectTable.GetObject
Issue 11 - When you insert a new step in a test case that contains some parameters, data corruption occurs. For example, an iteration of a parameter disappears.
1.2 Memory leaks
Issue 1- When you repeatedly open and close a work item that includes a webpage control in the template, the Test Manager process leaks memory.
Issue 2 - Consider the following scenario:
- You have a manual test process that is in several steps in Test Manager.
- You run the manual test process for a long time while you open an instance of an application. For example, you run the manual test process for a long time while you open an instance of Microsoft Visual Studio.
In this scenario, the process leaks memory, and the process may grow by hundreds of megabytes (MB).
Issue 3 - When you run a test case in Test Manager while an instance of an application is running, the MTM.exe process leaks memory and may grow by hundreds of megabytes. For example, you run a test case in Test Manager while an instance of a Visual Studio 2010 is running.
Issue 4 - When you run concurrent operations for several hours in Test Manager, you may receive an Out-of-Memory exception, and then Test Manager crashes.
Issue 5 - When you add many test cases to a test suite, a memory leak occurs.
1.3 Other Issues
Issue 1- After you close the Test Case Manager page, the test cases continue to be downloaded from Microsoft Team Foundation Server (TFS).
Issue 2 - When you run a test case, and when the Assigned To field in the test case is empty, you receive the following error message:
Object reference not set to instance of object.
Additionally, the test case returns no data.
Issue 3 - Test Manager does not have a feature to let you type a fractional number in a Double data type field of a work item.
Issue 4 - An automated test case from a build-drop path requires a long time to load.
Issue 5 - When a test result is published on a test suite that contains thousands of test cases in Test Manager, the publishing operation is slow.
Issue 6 - When you connect Test Manager to a Team Foundation Server (TFS) server, you may receive the following error message:
Automatic connection to the following team project failed: <project_name> on server <tfs server>.
2. Visual Studio 2010 Service Pack 1
Visual Studio 2010 SP1 was released in March 2011 to enhance features made available in Visual Studio 2010 and fix reported issues. This Service Pack can be downloaded here.
This Service Pack addresses the following bugs in Microsoft Test Manager:
- The "Configure Diagnostic Data Adapter-Event Log" page is truncated in different language versions of Microsoft Test Manager.
- Analyzing test results causes the result to become the most recent result in the test point trend.
- The Right-To-Left language settings do not persist across MTM sessions.
- Query-based test suite repopulation takes longer than expected for large suites.
- A crash may occur you select the browser type for Firefox playback in the Microsoft Test Runner.
- You cannot collect Intellitrace data if the application that is being tested is started as a part of action recording for the test case.
- The attachment count for test results returns 0 when iterating through large amounts of test results.
- You cannot bind the "parameter to password" field when you create action recording.
Additionally, Visual Studio 2010 SP1 contains the following improvements:
- The test case description can be shown in Microsoft Test Manager.
- Performance improvements for Microsoft Test Manager are provided.
- Support for Expression Encoder 4.0 in manual testing is added. For more information, click the following article number to view the article in the Microsoft Knowledge Base: 2160831 (https://support.microsoft.com/kb/2160831/) Update for Microsoft Visual Studio 2010 Video Data Adapter
Performance has also been improved, specifically in cases where the Team Foundation Server and Microsoft Test Manager client are geographically separated, with a high latency network.
More details on fixes available as part of Service Pack 1 are available here.
3. Multi-line support for editing test cases in Microsoft Test Manager
An often reported complaint in Visual Studio 2010 is that MTM does not support multi-line editing of test cases.
This issue has been addressed by the Visual Studio 2010 TFS Compatibility GDR (available here). Multi - lines in test steps are supported – use Alt + Enter while editing test cases to go to the next line.
4. Reduce the size of test data saved to the TFS Database
Scenario:
While publishing the results of automated test runs to TFS, MTM would upload all the binaries and the deployment items of the run. These uploaded binaries would be used later if the user wants to re-run tests and analyze failures.
Issue:
- The TFS Database becomes bloated after a while, typically in cases where users had large deployment items.
- Also when a tester tries to open the published test run results, MTM would download all the deployment items locally, which resulted in a performance hit on the end user experience.
Fix:
1. MTM does not upload the deployment items or test binaries of a test run, while publishing results to TFS, by default. MTM would only upload the trx and other data collector attachments. Similarly on the client side (VS), the option to “Re-run” tests would be disabled for such runs.
2. We have exposed a new Boolean attribute in test settings - “UploadDeploymentItems”. The tester can explicitly set it to true in order to override the default behavior and upload deployments items.
3. If code coverage is enabled for the test run, test binaries would always be uploaded, since they are required for doing code coverage analysis.
This QFE is available here.