Troubleshooting Record and Playback issues in Coded UI Test


We have been receiving numerous queries related to record and playback in Coded UI Test. In most of the scenarios, some additional information is sought after to debug and nail the issue. In this post, I will enumerate some basic validations that you (as a Coded UI Test user) can attempt to troubleshoot the problem. I’ll also mention a few workarounds that you can try out for some common failure scenarios. If all of these approaches do not solve the issue, you can very well post your query with a minimal repro step and associated data to the Coded UI Test forum and we’ll respond to it.

In an ideal world, you would be able to playback whatever script was generated out of your recorded scenario. The recorder though has some of its limitations; it could be some optimizations brought into the recorder to keep a balance between performance and resilience, or it could be some accessibility issues with the application under test itself. I will start to explain the various playback failures you might encounter and then relate it back to what might have gone wrong with the recording.

First of all, do go through the post here to understand how the Coded UI Test playback search works. There is one more aspect of the search i.e. Window search which is well explained over here. From performance perspective, there are a huge number of MSDN forum threads existing that you can refer to by doing a quick search.

 

Troubleshooting playback search failures

1. Validate the control’s existence

Double check (visually) that the control indeed exists in the application when the search failed. The application may be having certain behavior such that the control does not consistently appear. An example would be a troubleshooting page in MSDN where you may or may not see an optional feedback section at the bottom of the page. If this is indeed the case, you would need to tweak your test automation code to make this action optional by enabling ‘ContinueOnError’.

Playback.PlaybackSettings.ContinueOnError = true;

// Action on the control that occurs inconsistently in the test flow.

Playback.PlaybackSettings.ContinueOnError = false;

2. WaitForReady Issues

In continuation with [1], if the control comes into existence albeit after some delay, you might need to apply some wait features provided in the playback. Check the related blog here.

3. Examining and fixing the “bad” ancestor in search hierarchy

Locate the UI control (for which search failed) using Coded UI Test builder’s control locator. You’ll come up with something like the following snapshot:

 

 
Assume the search failed for UIDCheckBox control. Starting from UIDCheckBox till UIBasicFormWindow, select the UI object up at each level and click on the Refresh properties button to validate if the search is successful for that control. Say, the refresh is not successful till UIDTreeItem, but is successful for UINotMyComputerTreeItem.  It implies, the search is broken in between these 2 objects. Check if the refresh of UINotMyComputerTreeItem actually locates (blue rectangle highlighter) the “Not My Computer” or some other control, say the “My Computer” Tree Item.  If this is the case, then both these items would be having the same identification properties and the
search hits the “My Computer” control and tries to search for the “D:” tree item within it (and hence fails; assuming there is no other ‘D:’ item under “MyComputer”).

Another way of validating this is (which is essentially a similar approach, but top-down) to highlight the controls along the hierarchy top-down in your test code and see where it goes off-track.

            WpfCheckBox uIDCheckBox = this.UIBasicFormWindow.UITreeView2Tree.UINotMyComputerTreeItem.UIDTreeItem.UIDCheckBox;

            // Highlight to visually verify the search path

            this.UIBasicFormWindow.DrawHighlight();

            this.UIBasicFormWindow.UITreeView2Tree.DrawHighlight();

            this.UIBasicFormWindow.UITreeView2Tree.UINotMyComputerTreeItem.DrawHighlight();

            this.UIBasicFormWindow.UITreeView2Tree.UINotMyComputerTreeItem.UIDTreeItem.DrawHighlight();

            this.UIBasicFormWindow.UITreeView2Tree.UINotMyComputerTreeItem.UIDTreeItem.UIDCheckBox.DrawHighlight();

            // Right-Click ‘D:’ check box

            Mouse.Click(uIDCheckBox, MouseButtons.Right, ModifierKeys.None, new Point(7, 8));

There are 2 ways to fix this issue. First, it is highly recommended to fix the accessibility (in this case UI Automation) of the tree item controls to associate unique properties to them. If you do not have access to the application sources and/or cannot rebuild the app, you might try differentiating the controls by adding some secondary properties such as “Instance”. In this case, you can add an Instance search property of value “2” to the UINotMyComputerTreeItem object.

    public class UINotMyComputerTreeItem : WpfTreeItem

    {

        public UINotMyComputerTreeItem(UITestControl searchLimitContainer) : base(searchLimitContainer)

        {

            #region Search Criteria

            this.SearchProperties[WpfTreeItem.PropertyNames.Instance] = “2”;

            this.WindowTitles.Add(“BasicForm”);

            #endregion

        }
        …

    }

There is another non-trivial way to tweak your search property using the NextSibling search configuration as explained here.

4. Virtualization issues

An application could be designed in such a way that it’s UI objects are being fetched from a data source on demand basis. For example, say a grid table does not want to fetch all 10000 rows of data; instead it will just load the first 10 rows that show up in the view port and as and when the user scrolls the grid, it would load the rows from the data source. Playback of actions on these scrolled out rows will fail since there is no corresponding UI Object (and their accessible objects) existing at the time of search when the grid has just loaded and is at its initial state. The forum thread here has a mention of this issue.

To work around this, you would then need to evaluate the amount of scrolling required to get the target grid rows into view and insert explicit scroll actions in your test code such as Mouse.MoveScrollWheel() or any other Mouse APIs here on the scroll bar.

For certain UI technologies such as WPF which has virtualization support from accessibility layer (UI Automation), Coded UI Test playback does fine. Note that UIA virtualization support in WPF is provided for .NET4.0 and above. Hence, if you have a WPF application containing virtualized item containers built in .NET3.5 or below, you need to do one of the following:

  1. Recompile the application on .NET4.0 or above.
  2. Add an application configuration file (yourapp.exe.config) specifying the supported runtime, something like

<?xml version=”1.0″?>

<configuration>

<startup>

<supportedRuntime version=”v4.0″/>

</startup>

</configuration>

You might come across one playback issue related to WPF virtualization on nested containers such as a Datagrid inside a Tab Page. The blog here describes in detail the problem statement and the hotfix released over Visual Studio 2010 Service Pack 1.

5.  Hand coding search for expandable controls

If you are scripting the search hierarchy of a control which has expandable ancestors (for e.g. a multi-level tree view item), make sure that all the expandable ancestors have SearchConfiguration.ExpandWhileSearching added. (uiControl.SearchConfigurations.Add(SearchConfiguration.ExpandWhileSearching)). You need to worry about this if the search hierarchy is auto-generated by Coded UI Test; the recorder does this automatically.

6. Dealing with stale controls

SearchConfiguration.AlwaysSearch is another useful configuration when you want to be precisely sure that your playback is not picking up any cached UITestControl for search. Just to give some symptoms about playback failures where you would like to apply this configuration, a search for a control A inside a control X passed. Then the next action refreshed this control X and the subsequent action was search for a control B inside the same control X. Now control B has unique identifiable properties inside the sub-tree scope of X, yet the search for B still fails. This could be a case of some stale accessible objects not yet disposed of and the search for B is happening under the stale Control X reference.  In such a scenario, add an AlwaysSearch configuration to the Control X.

One word of caution: AlwaysSearch will have a performance hit in normal scenarios. Hence, do not include this configuration by default in your control search.

 

          
Troubleshooting playback action failures

 

1. FailedtoPerformActionOnBlockedControlException

One of the most common failure encountered is FailedToPerformActionOnBlockedControlException. Coded UI Test’s playback engine makes a best effort to get the target control into screen area and into the foreground prior to performing the action. So ideally no explicit “EnsureVisible” logic is required to be hand-coded into your test automation. However, accessibility issues in the application can cause failures otherwise.

When an action fails with this blocked control exception, visually verify if there is another control indeed that is blocking this control in the screen area. There could be instances of spurious popups not been dismissed or not handled in the test code. If the control appears to be in foreground, use an accessibility tool (say UISpy) to validate what is the actual control returned at that point in screen within the target control’s bounds. There could be possibility of transparent overlay controls that is actually blocking this control. In such situations, you might have to modify your test code to validate this and do absolute screen coordinates click on the control. The post here will be helpful. Alternately, you can search for similar issues reported in the MSDN forum related to FailedToPerformActionOnBlockedControlException.

2. UITestControlNotVisibleException

Verify (using an accessibility tool) that the control has a valid bounding rectangle. It could be an accessibility issue with the bounding rectangle implementation of the control itself.

3. Handling spurious popups

There has been queries in past on how to handle popups in the test code which are intermittent i.e. primarily, how to detect and dismiss the popups and continue with the next action. A quick search on the MSDN forum post here
will get your some useful answers related to this.

4. Recording Mouse Hovers

Non-recorded mouse hovers could be another reason behind blocked control exceptions. For plugins like IE, Coded UI Test records hovers on controls. However, for other technologies such as WPF, Silverlight, Winforms, etc, this information is not captured since the accessibility layer does not provide this level of information. Consider a scenario where a control ‘X’ has a OnMouseHover implementation that overlays another control ‘Y’ on top of it. So when the user hovers and clicks on the control, the Coded UI Test recorder will actually record a click on the control ‘Y’. During playback, the control ‘Y’ is obscured behind the control ‘X’ unless hovered upon explicitly.

To overcome this issue, you would need to record an implicit hover. In the codeduitestbuilder.exe.config file, you will find entries

    <!– HoverKey to use. –>

    <add key=”HoverKeyModifier” value=”Control, Shift”/>

    <add key=”HoverKey” value=”R”/>

    <!–Use this to enable/disable recording of implicithovers.–>

    <add key=”RecordImplicitHover” value=”false”/>

Setting RecordImplicitHover to true, you can record a hover action by placing the Mouse cursor over a control and pressing {Control + R}. 

5. Resilience issues with mouse action

You might in rare in occasion hit an issue wherein the Mouse.Click() was actually successful but the control did not respond to the click (which typically it would otherwise). These could be some one-off resilience issue and cause would vary across different scenarios, so not completely known. Coded UI Test playback has in-built WaitForReady logic (to ensure that the control is actually ready to receive the next input) and UI synchronization mechanism (to ensure that the control’s window actually received the mouse (and keyboard) input). So the reason of control not responding could be something more than the input not reaching its destination window. First thing you can try out in such scenario is validate if any explicit introduced delay (such as Playback.Wait()) fixes the issue. If not, you may then try to raise the robustness level of WaitForReadyLevel to AllThreads ( by default it is UIThreadOnly). This could slow down the playback, so do not keep this setting as default.

              
Other playback issues

1. Search taking a huge amount of time

If the search is taking a lot of time to find the control, there could be the possibility that the recorder was not able to create an optimized search hierarchy.  Or you may have hand-coded your test automation script which is not optimized for search. There is a high chance that WaitForReady, MatchExactHierachy (a.k.a SkipIntermediateElements) and SmartMatch settings are related to this issue. Check the MSDN forum post here
to understand these better and apply them to your test code. The UITest logs will give a fair idea on which configuration setting needs to be tweaked to optimize the search (if at all there is a scope for it).

 2. Use WaitForReady settings judiciously

WaitForReady is inbuilt into the playback for a strong reason and has some default settings such as the WFR level and the WFR Timeout. At times though, you might need to raise the timeout value or the WFR level for applications which takes a huge amount of load. In the flip side, you might sometimes need to turn off WFR temporarily to speed up your playback. Ensure you know what you are doing with these WFR settings and apply it with caution. Also,
check out the section on “How to handle search failures because of slow page loading” here.

 3. Optimize your test code while fetching UITestControl item collection

A common mistake I’ve observed  among Coded UI Test users is something in the lines of

            WpfTable myTable = FindTable(…);

            for (int index = 0; index < myTable.RowCount; index++)

            {

                UITestControl row = myTable.Rows[index];

                // Do some action on the row.

            }

There is no caching built into the .Rows implementation. You are essentially re-fetching the entire row collection in each iteration. This has a huge performance hit. So avoid this by getting the UITestControlCollection
out of the loop.

 4. UITestControl.GetProperty failures

Coded UI Test provides the access to the control properties to the extent of what is provided by the accessibility layer.  First of all, if you encounter an issue where the property value obtained via Coded UI Test GetProperty()
is not what you are expecting, verify this control’s properties via an accessibility tool. Coded UI Test does not, in almost all cases, do any special processing over the value received. So it could be a genuine accessibility issue.

To extend support for additional properties fetching and validating, check out some of the blogs here: [1] [2] [3] . Another interesting blog on validating properties on nested Silverlight
controls can be found here.

5. Running playback outside a Coded UI Test method

Ensure that you are calling Playback.Initialize() and Playback.Cleanup() explicitly. Blog here describes this.

 

Troubleshooting recording failures

1. Unable to identify the control

Verify using an accessibility tool if it can identify the control. If it too does not, in all probability it is the lack of required accessibility implementation for the control. The blog series here
gives a good direction to extend the support. Another sequence of blogs here shows how to add support for 3rd party controls. And some more internals of the extensibility points are described here.

2. “Last action was not recorded because the control with Name ‘XXX’ and ControlType ‘YYY’ does not have any good identification property”

You will typically hit this issue for data bound item controls which do not have ToString() implemented properly to disambiguate it from its siblings. The value returned from ToString() is what is being picked
up by default by the accessibility layer as the ‘Name’ property value. The blog here describes how to get automation working for data bound WPF list or combo box.

3.  “No Silverlight controls were detected. Verify that the application under test is built using Silverlight assemblies …”

Check out the MSDN documentation here to understand how to setup a Silverlight 4 application for test.

4 .Configuration options in recorder.

Keep note of settings in the codeduitestbuilder.exe.config file such as ThinkTimeThreshold and RecordThinkTime, AggregatorTimeout, ExcludeProcess and IncludeProcess. The options are self-explanatory in the configuration file and can be tweaked to address recording issues such as multiple actions not getting aggregated into a single actions, actions not getting recorded on certain application, etc.

 

The above mentioned points cover some of the common issues encountered and may not be totally exhaustive. There is scope to drill down to some more of the UI technology specific areas and collate the set of trouble
shooting measure. Meanwhile, I hope the above is useful to get starting and captures a good proportion of the issues which you can debug at your end. If there are still blockers, feel free to post your queries in the Coded
UI Test forum
and ensure you provide the following information that would greatly help in investigating the issue.

 
1. Describe your problem.

2. Mention what all troubleshooting measures you have already applied.

3. Create a minimal repro step. Huge logs and UIMap file just makes it difficult to debug.

4. Provide the log files. Check here on how to enable logging.

5. The recorded .uitest file, or better still, the minimal Coded UI Test project that you created.

6. Snapshots of the application to show the visual UI hierarchy would be very useful. Also, the UI hierarchy snapshot from accessibility tool such as UISpy, Inspect, etc.

7. Sample application is always helpful ! 

 

 

Comments (23)

  1. Vikram Reddy says:

    Excellent post.

  2. Sandeep says:

    Hi Tapas, I am automating an application using multiple UIMaps approach. I added test objects across multiple screens (created multiple UIMaps). Now I want only one test containing multiple methods where each method does some operation on the test objects (UIMaps). I am not defining methods on UIMap.cs files and then calling them from test by instantiating those classes. I want to run these methods from a single test straightaway but that doesn't seem to be possible. Please advice

  3. manjunath says:

    Hi Tapas,I have a doubt regarding adding assertion objects. for example i have validate the logo for the home page and generated the scripts for that. I have added assertion equal to 'Img'. if any case if the developer did some code  change because of that logo has moved from original position to some where. In case if we run the generated script whether it will pass or fail. Awaiting for your reply.

  4. Sumitra, says:

    Hi Tapas,

    I m very new to coded UI test. In my application there are many check box. While recording everything seems fine. But when i play back i get an error . That is the controller is not identifying the check box. Could you please help me to overcome from this issue. I m not able to proceed further because of this.

    I need to click the check box and then proceed to next step.

    Waiting for your reply.

    My email id : sumirathi@gmail.com

    Sumitra

  5. Jeyanthi says:

    I'd like to know about the mismatch in screeen resolution between recording and playback. I was trying to demo my test to my colleague and couldn't see some of the pop-ups b/o the screen resolution, esp. with a projector connected to the laptop. How do I get over such config mismatch? My recording was done at the highest resolution on a WIndows 7 OS. Thank you.

  6. Anu says:

    Hi Tapas ,

    I am automating desktop applciation uisng Coded Ui .

    I am facing some many times issues for assetion of values. As control id dnt have automation Ids.

    Can you please share some links ,by which i can go through to get more knowledge on CodedUI.

  7. Junaid says:

    We are automating a web application with knockout.js implemented. Problem arrives on playback in IE when for example, on button click, the click event of knockout.js overwrites over click event of the button itself and page redirection does not take place (in case of playback on IE only). Thus my test is getting failed. is there a way round this problem??

    As mentioned, this problem occurs on IE only and not on other browsers but creating tests on IE is required in my case.

  8. Umair Khan says:

    Junaid please use this.

    ConfigurationManager.AppSettings["WebWaitForReadyLevel"] = "3";

    add this in your app.config file under app setting

    <add key="WebWaitForReadyLevel" value="3"/>

    hope this will solve your problem.

  9. raja says:

    I am automating a application using coded ui .I have recorded once and then had some issues while re-reunning it.So i re-recorded the same module for the second time and replaced the old recordings with new recording.But now when i re-run it again  the first recorded item is getting executed.Can you please help to overcome this issue

  10. Ma Wei says:

    it is a nice post…but i still hit the issue “No Silverlight controls were detected. Verify that the application under test is built using Silverlight assemblies …” after i config my env follow by the msdn docs.. could you pls send me some more details(better with screenshot) docs about how to setup silverlight 4 automation ui test.? thx my emial is: 382238202@qq.com

  11. Bhavani says:

    Hi tapas,

    I have a recorded action recording in my CUI test method that was working fine on previous build of our target application. However, its failing to click a control when run on latest build.

    1. Firstly there is no UI change

    2. I have tried to locate the control using UIMap.uitest XML map and its locating the control successfully

    3. The control is loaded before the action recorded is executed – I have verified this, so the control is definitely available at the time of execution

    4. I have also tried by drawing focus on the said control before execution of further steps.

    5. When steps are re-recorded then its working fine. However, previous action method is not working

    6. I have tried comparing all the properties of the said control in both previous and current recordings and they exactly have same search properties, configuration, etc

    Any insight is truly appreciated

  12. Mon_bk says:

    Hi all,

      question-1: how to wait till perticular action get completed ,if i dont know how much time to wait, its totaaly depends on network speed. i have tried thread .sleep(),playback.wait(),palyback.playbacksetting for ui and all thread …still sometime page does not get loads and test failed.

          question-2: How to check control available or not? or how to verify button or pane visible on ui or not?

    Thanks in advance

  13. Rajesh says:

    Hi all,

    I have one issue in one of my scripts execution. while executing script once webpage got opened , playback is not identifying controls and if I take mouse hover on refresh button or mouse right click and select refresh then only its entering values to the respective fields. if I don't take mouse hover on refresh button script is getting failed and below is the error im getting.

    ;[Web, AlwaysSearch]ControlType='Document' && TagName='BODY' && Id='' && RedirectingPage='False' && FrameDocument='False' && FilterCondition(Title='Youforce by Raet', AbsolutePath='/icweb/raetonlinedienstverlening/raetonlineportaal/index.aspx', PageUrl='login.youforce.biz/…/index.aspx&)

    Please provide any insights to resolve this will be greatly appriciated

  14. Shailendra says:

    Hi all,

    I have a issue with cross browser testing, Cross browser test builder is not able to find the control in page while i try to add assertion for them(that's why it fails to run cross browser test,i think

    ).please give me some advice.

    I have already downloaded selenium component for cross browser advised by MSDN and prescribed Google chrome Version 32.0.1700.102.

  15. how to test a dot lock says:

    i have a app , the first page is a dot lock, only tap the right gesture that can we use the functions, but i can't find a way to record a broken line(the password of dot lock has at least 4 points).please give me some advice.

  16. Michael says:

    Would be nice if you could include playback on multiple test scenarios, such as when running all tests in a suite or plan in MTM.

    I have a situation where only the first test will work, for the remaining tests – I get control lookup errors.

  17. ygtyuyuyt says:

    Would be nice if you could include playback on multiple test scenarios, such as when running all tests in a suite or plan in MTM.

    I have a situation where only the first test will work, for the remaining tests – I get control lookup errors.

  18. Rashmi says:

    Hi,

    I am trying to automate WPF application(wpf with 3rd party devexpress) using coded ui(VS 2012) in my local machine (windows server 2008 R2).

    I am facing issues while identifying controls under dynamically generating content of the window.Tried different hierarchical levels to hit the control,But still not able to hit the control.

    Till some level I am getting the handle,but after I am not getting the handle.

    Application is huge hierarchically structured application with windows and wpf application.

    Tried coded UI record and play feature to generate the UI Map. Used same structure to identify the controls.It worked while debugging line by line code but its failing while running.

    e.g. Below is the one hierarchy,

    Dashboard

     Dashboard_Grid

           Dock_LayoutManager

      Dock_LayoutGroup

          Dock_LayoutPanel –> Till this level I am able to get the handle and later I am not able to hit the control

          Dock_Container

         Navtop

       Navtop_Grid

     Dock_TileContainer

       Navtop_TileLayoutControl

     ……..(all tiles)

    This issue is not only with one page,I am getting for all pages.So this is blocking our automation.

    Please help me on this.

  19. Anand says:

    Hi ,

    I am trying to automate Simple opening Calculator application with Coded ui  .As mentioned in many example I was able to record the opening of Calculator correctly but while running the same code with(Run Tests)  after building it display an error .

    ========== Build: 0 succeeded, 0 failed, 1 up-to-date, 0 skipped ==========

    Could any one please help me regarding this issue.I am not able to run the first given example for CODED UI

    with Visual Studio 2012 ultimate on Windows 8.

  20. Sucharita says:

    Hi,

    There are two applications one is coded UI project and another is Silverlight app.

    I am trying to use Silverlight app as client in my Coded UI framework to test its controls.

    I copied this url http://localhost:64938/SilverlightAppTestPage.aspx in app.config inside coded UI project.

    But the issue is when the UI is launched in browser through a test case  the Coded UI builder is not identifying the Silverlight controls.

    Below Pre-requisites for Silverlight are installed:

    1.       Silverlight_Developer

    2.       silverlight_sdk

    3.       UITestPluginForSilverlight

    I am using VS 2012 Premium and IE 11.

    Ideally Coded UI builder should identify the controls after all this.

    I get below error:

    The playback failed to find the control with the given search properties. Additional Details:

    TechnologyName:  'Silverlight'

    ControlType:  'BusyIndicator'

  21. Meenakshi Nagpal says:

    I have generated code using record option in coded ui in my machine and it is played back without any issue in my machine and search criteria also correct. But when I tried to run those test cases in another machine I ma getting unable to find control exception.

    I am using visual studio 2013 ultimate.

    Unable to resolve the issue. Please help me out.

  22. Tanisha says:

    I am trying to record an action in Coded UI.

    But while recording not able to record the browser open action.

    Ans while running the test its throwing the following error message.

    Microsoft.VisualStudio.TestTools.UITest.Extension.UITestControlNotFoundException: The playback failed to find the control with the given search properties. Additional Details:

    TechnologyName:  'MSAA'

    ControlType:  'Window'

    AccessibleName:  'Context'

    ClassName:  '#32768'

    —> System.Runtime.InteropServices.COMException: Error HRESULT E_FAIL has been returned from a call to a COM component.