Using Coded UI to test XAML-based Windows Phone apps
April 4, 2014
In order to ship a great quality app to the Store, it is essential that you test it and flush out the bugs. End to end scenario tests when automated can help you ensure that any regressions in the app are caught early. With Visual Studio 2013 Update 2, you can now write automated end-to-end tests for your app using Coded UI Test. In Visual Studio 2013, Coded UI Test support for XAML-based Windows Store apps was enabled and the experience to use Coded UI test for Phone apps is largely the same. This post walks you through the essentials of creating a Coded UI test for your XAML-based Phone app.
Getting Started
Writing a Coded UI test for XAML based Phone app is easy. When you install Visual Studio 2013 Update 2 RC, you will see a new template under the Windows Phone apps node that lets you create Coded UI Test for your Phone app.
After you create a new project, you can use the Coded UI test builder to spy on the UI controls in your app that you want to interact with. You can also use the builder to build a UIMap, a repository of UI controls, code for which is automatically generated for you.
Coded UI test builder connects to a Windows Phone Blue emulator instance running on your machine. You can start building your UIMap by dragging and dropping the cross-hair tool on
the control of interest. You can also use keyboard shortcut keys Ctrl+I (spy) and Ctrl+Shift+I (Add to UIMap) for the same – just hover over the control of interest in the emulator with your mouse and use the shortcut keys. A couple of points to note:
- If you use the cross-hair tool to spy on UI elements that are outside of the emulator, the builder will not detect those
controls. It will tell you that the control co-ordinates are outside of the emulator. - The builder can spy UI elements when the app is running on the emulator. Builder cannot be used to spy UI elements when the app
is running on a physical Phone device.
If you want to author your test from scratch and not use the builder to build a UIMap, you can choose the option to ‘Manually edit the test’.
Working with UI elements of the app
Coded UI test makes specialized classes available so that a rich set of properties is available for working with the UI elements. In the case of XAML apps, the specialized classes
for all the controls are prefixed with Xaml. XamlButton, XamlEdit, XamlList etc., are some examples. All these classes are available under the Microsoft.VisualStudio.TestTools.UITesting.WindowsRuntimeControls namespace. This is the same set of classes that are available for Coded UI test for Windows apps.
WebView control used to host HTML content in a XAML app is currently not supported.
You can also interact with Shell controls – controls that are not XAML, but essential for testing your app E2E – such as the tiles, confirmation dialogs, etc. These controls are provided by
the OS and are not XAML. These will be identified as UITestControl. The Shell controls are identified differently than big Windows, because the UI technology behind these controls is different on the two platforms. On big Windows, the shell controls such as controls in the settings charm, tiles etc. are identified as a DirectUIControl.
Performing actions on UI elements
Once you have built the UIMap to identify UI controls, you can start writing code to act on these controls. A series of actions will make up the scenario you want to test.
Actions on controls can be performed in two ways:
1. Touch gestures on controls: All gestures supported by the Windows platform are supported. Gesture class exposes all the touch gestures. For e.g., to invoke a button, the action code would be Gesture.Tap(myButton);
2. Properties and methods on control classes. Each specialized control class exposes a bunch of properties and methods that you can use to obtain information about or interact with the control. For e.g., to input text “ABC” in an edit control, you can use
myXamlEditControl.Text=”ABC”;
Launching an app
Launching an app on Windows Phone is similar to launching a Store app on big Windows. You can launch an app in one of two ways:
- Tapping on the app tile in the apps list
- Using the XamlWindow.Launch API. The Launch method takes in the unique identifier for your app. Just as in big Windows, the AutomationId for the app tile on Start screen or app item in the apps list is the unique identifier for your app. You can obtain this string value by observing the app tile in CUIT builder and copy/pasting the AutomationId property of the tile
control, as seen in the following screenshot:
Executing Tests
Tests can be executed from within Visual Studio using Test Explorer or from the command-line. Coded UI tests can be executed on the emulator or the device. When executing tests
from Visual Studio, use the device toolbar to specify the target device where the tests should execute.

Tests can also be executed from the command-line using vstest.console.exe. You can specify the target device for test execution using a runsettings file.
vstest.console.exe “pathToYourCodedUITestDll” /settings:devicetarget.runsettings
Sample runsettings file:
<?xml version=”1.0″ encoding=”utf-8″?>
<RunSettings>
<MSPhoneTest>
<!–to specify test execution on device, use a TargetDevice option as follows–>
<TargetDevice>Device</TargetDevice>
<!–to specify an emulator instead, use a TargetDevice option like below–>
<!–<TargetDevice>Emulator 8.1 WVGA 4 inch 512MB</TargetDevice>–>
</MSPhoneTest>
</RunSettings>
Data driving a Coded UI Test Method
Once you have written a scenario test, you might want to run the same test multiple times with different sets of data to test different conditions. Data-driving comes in handy for such cases. Data-driven Coded UI tests for Windows Phone can be defined using the DataRow attribute on a test method.
[DataRow(1, 2, DisplayName = “Add positive numbers“)]
[DataRow(-1, -2, DisplayName = “Add negative numbers“)]
[TestMethod]
public void DataDrivingDemo_MyTestMethod(int x, int y)
Data that drives your test is defined inline using the DataRow attribute. In the above example, x and y would obtain the values of 1 and 2 respectively for the first iteration and -1 and -2 resp. for the second iteration of the test.
Differences between Coded UI for XAML-based apps for Windows and Windows Phone
While Coded UI test for XAML apps on big Windows and Windows Phone is largely the same in capabilities, the table below indicates the differences between them.
|
Feature |
Coded UI for apps on Windows |
Coded UI for apps on Windows Phone |
|
Target for running tests |
Local or remote computer. Remote computers can be specified |
Emulator or Phone |
|
Execute from the command-line |
Settings file not required to specify |
runsettings file required to specify |
|
Specialized classes for Shell |
DirectUI class |
UITestControl class |
|
WebView control in a XAML app |
Supported, HTML elements can be |
Not supported. |
|
Execute automated tests from MTM |
Supported |
Not supported. |
|
Data-driving tests |
Using external |
Data is specified inline, using DataRow |
Limitations
1. Recording of action steps to create Coded UI test for XAML based Phone apps is not supported.
2. Only XAML based store apps are supported. Silverlight and HTML 5 based apps cannot be tested using Coded UI.



Are all the rows in the differences table correct? Can you really run Windows tests on a phone?
Hi Prachi,
Thanks for writing this blog.
I've installed Visual Studio 2013 update 2. But I cannot find Windows Phone Apps node. Please let me know if i need to install any additional component.
Note: My OS is Windows 8
Sorry for Typing error!!
I've installed Visual Studio 2013 update 2. (http://www.microsoft.com/…/details.aspx)
But I cannot find Coded UI Test Project (Windows Phone) under Windows Phone Apps node. Please let me know if i need to install any additional component.
Note: My OS is Windows 8
@Matt: Thanks for the question. Windows and Phone columns got mixed up in the first row of the table. I have corrected it now.
@Satish: You need to be on Windows 8.1 and have Phone development tools installed in order to develop Phone apps for Windows Phone 8.1 as well as write Coded UI test for them. Coded UI ships in Visual Studio Ultimate and Premium editions.
Thanks for your reply Prachi ๐
I am not able to find official download link for Windows Phone 8.1 SDK.
Appreciate if you could share the same.
I really enjoyed seeing this presentation at Build last week. It's exciting to see the direction the development tools are going in. Would it be possible for you to share the project you used in the demo, please?
Hi Ms. Bora,
Thanks for the clear Build presentation, and this great follow-up blog! I'd like to formally echo Simply Ged's request. A copy of your project (or something similar) would be so very helpful to me – not only for the testing sample code, but for the mvvm-with-database code, as well. I'm currently working on an app that includes a relational database, and I'm having a tough time finding samples that cover that level of complexity for Windows Phone 8/8.1.
Can we bribe you with cookies or something? ๐
@ Satish: Tools for development for Windows Phone 8.1 are bundled in the Visual Studio 2013 Update 2 RC. Download link is http://www.microsoft.com/…/details.aspx
@Simply Ged, @Pretty Please?,
Yes, I can make the demo project available. I am traveling right now, so I should have updates in this regard late next week.
@Pretty Please? : Cookies are awesome :-), even virtual ones would work. I love double chocolate chip ๐
@Pretty please: Forgot to mention that my demo app did not use a database. All the data is written to files in the app's isolated storage.
For folks looking for help with MVVM and sample Store app using it, Prism for Windows Runtime is a great resource. prismwindowsruntime.codeplex.com
Is there a way to reset the state of the app after each test?
Excellent, thanks! Dunno why I thought you used a database. *shrug* I'm still looking forward to your post.
(PS: Bribe delivery –> onedrive.live.com/redir)
Thanks for the blog Prachi and for the BUILD sessions too.
I'm working through some WinPhone Coded Ui tests at the moment as a follow up to github.com/…/WindowsPhoneTestFramework, and your blog seems to provided the best documentation available ๐
Is it OK if I ask a few technical questions?
1. Is there any way to get screenshots off of the phone – I've dug into the resources of WindowsPhone.CodedUITestFramework and seen that there are some "ScreenShotMessage" and "ScreenShotFailure" resource messages – but can't see any API for screenshots – is there one? I've seen msdn.microsoft.com/…/microsoft.visualstudio.testtools.uitesting.uitestcontrol.captureimage.aspx – but can't find that in code.
2. Is there any way to send keyboard messages – I've seen I can set text on a XamlEdit but I also want to be able to send Key strokes – especially "Enter/Go" so that I can trigger a navigation. Old CodedUI seems to have a Keyboard class – but that doesn't seem to be available in WinPhone. There seems to be a SendKeysAction class available but I can't work out how to use it from C# code. Are there any ideas here? Thanks.
3. Calls to Gesture seem really fast – but searching for controls or iterating over controls is a lot slower – are there any tips for speeding up operation?
4. Are there any samples available for how to use more native elements – things like MessageBox's, notification Toasts, ApplicationBar buttons and menus, the phone home screen, etc – the more samples the better really ๐
5. If we find suspected issues or bugs then what's the best way of reporting them? Is there a specific Connect category we should use?
Sorry for so many questions! Overall, I love having official CodedUI support – thanks for the work you and your team are putting in ๐
Stuart
@Stuart,
1. Currently, there is no way to take screenshots at will on the Phone. By default, a screenshot will be taken if the test fails.
2. Keyboard is only available on the desktop. Is there no other affordance in your app to trigger a navigation such as a submit button / next etc. that you could tap on?
3. Some guidelines to improve Coded UI test has been blogged at blogs.msdn.com/…/guidelines-on-improving-performance-of-coded-ui-test-playback.aspx.
4. Messageboxes, appbar controls, start screen etc. can be automated in the same way – capture them using the builder and use the Gestures / APIs to interact with them. Some controls such as Start screen are not XAML, so they will be recognized as UITestControl. There's no difference otherwise. Let me know if you run into any specific issues with these. Testing notification toasts is not supported.
5. If you find any issues / bugs, please file them on Connect. Adding Coded UI test to the title should route it to my team. For any suggestions / feature requests, please file on uservoice.
Thanks for this,very helpful
Hi prachi ,
I deployed an application on the w8.1 phone emulator and then tried to generate code using the coded ui test builder but as soon as the test builder gets initiated , i'm getting the following error – "Coded UI Test Builder could not connect to the emulator. Please restart Coded UI Test Builder and the Emulator and try again."
please let me know how to connect the test builder to the emulator in order to generate my test script.
Hi prachi ,
The app is installed on my emulator but still when i'm trying to launch the app from coded ui i'm getting an error with the following exception <b>" Message=Exception occurred while trying to launch the app. Verify that the app is installed: Value does not fall within the expected range., AutomationId of Tile: default'"</b>.
Thanks in advance !
Thanks for the answers prachi – very useful
I have some more questions:
– is there any way to determine the device profile at runtime – e.g. what model the device is? whether it's an emulator or a real device? the screen size?
– what's the best way to determine the rotation – portrait or landscape – at runtime? Is there even a way to programmatically change it?
Thanks again for any help ๐
Stuart
@Sai Kiran,
Have you verified the parameter you are passing to app launch? This value is the same as the AutomationId of the tile for your app. This is illustrated in the "launching an app" section in the post.
@Stuart: No, we don't have a way to determine the device model etc. Can you specify the scenario for which you need this information when executing tests?
Currently, there's no way to determine / change the rotation information. It's on our backlog. I recommend that you raise a user voice item for this, so that others who need it can vote for it as well. This will help drive the prioritization of this item in our backlog.
Hi Prachi,
Thanks a lot for the decent demo in build. Can you please let me know whether I should buy Ultimate/Premium versions just to run automation? I mean, can I have 1 Premium license with that I'll develop my Coded UI recording. Can I play those CodedUI recording on some test system where i do not have Premium licenses?
can I use only MStest to execute coded UI recrodings?
Thanks in advance
-Mahendar
When using CodedUI for WindowsPhone, I frequently seem to get the tests into a state where they won't start. It's not obvious what is causing this – when I try from the command line I get a stack trace starting:
"Unhandled Exception: System.Exception: TestHostCommand_LaunchTestHostAgent failed on the device side with error code -2147024629
at Microsoft.VisualStudio.TestPlatform.Utilities.PhoneTestHostExecutorLauncher.LaunchTestHost(IDevice _deviceInstance, TestHostCommandID command)"
This state seems to persist across emulator restarts, and I also see the same sort of problem on real phones too – the problem eventually clears, but I''m not sure why…
I've not worked out what causes this. Are there any suggestions?
Thanks again
Stuart
Hi Stuart,
Which version of VS are you using? If the issue still persists, can you please enable logs for vstest.executionengine.exe and QTAgent32.exe and post the logs along with the issue to vstestsup@microsoft.com ?
To enable logs you can refer: blogs.msdn.com/…/how-to-enable-ute-logs.aspx and blogs.msdn.com/…/how-to-enable-test-agent-logs.aspx
Thank you.
Vaibhav
Hi Prachi,
It looks like Coded UI unable to interact with out of process Shells controls.
Here's more details: stackoverflow.com/…/coded-ui-unable-to-interact-with-out-of-process-shells-controls
To reproduce – add a line into some event handler in your target app "var sc = Windows.Media.Capture.ScreenCapture.GetForCurrentView();", invoke that event, see the message, and try to automate that message with Coded UI.
Hi,
My apps is XAML-based apps which using a lot of custom controls, when i using the coded ui test builder to spy UI elements on the app, it says my app is not XAML-based apps, so i want to ask if the coded ui test builder has some restrictions on custom control?
Thanks in advance!
Is there a way to launch the emulator from the Coded UI API. If so please post the sample code
HI Prachi
How to collect reports for passed and failed test for coded UI
Is there any conditions to be enabled to collect reports
Please advice
Thanks
Shasan
Hi Prachi,
I'm not able the launch the app using the unique ID. I'm getting an error "Exception occurred while trying to launch the app. Verify that the app is installed: Value does not fall within the expected range., AutomationId of Tile.."
But when I tried by creating sample app it worked. How to launch an existing application?
Also, controls though they are of XAML coded UI test builder is not able to identify the controls and showing an error "Coded UI works only for XAML based apps"
Can you please help me on this..
Thanks in Advance ๐
Hi Prachi,
Iโm running coded ui tests on windows phome 8.1 using โvstest.console.exeโ with Test.RunSettings and /logger:trx
Test.RunSettings :
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<MSPhoneTest>
<!–to specify test execution on device, use a TargetDevice option as follows–>
<TargetDevice>Device</TargetDevice>
<!–to specify an emulator instead, use a TargetDevice option like below–>
<!–<TargetDevice>Emulator 8.1 WVGA 4 inch 512MB</TargetDevice>–>
</MSPhoneTest>
</RunSettings>
But if the tests fails there are no screenshots.
Am I missing something?
Best regards,
Levi
Hi,
there is any option to read or write txt file to local computer?
or any other way to pass data to the test?
Do you have any plan to support the webview, silverlight and winjs in the future? like in VS 2015, win10?
does Coded UI support localization testing ? how to discerm the control with the localized name while this control has not AutomationId ?
Hi , WebView control in a XAML App for windows phone using Coded is supported in VS2013
I am Using coded UI for windows phone 8.1 want to automate Web browser , please help me on this…plz let me know if there is any tweak for this ?
Regards
Shasan M
Hi Prachi
I would like to test toast notification on windows store app using coded UI.
could you please let me know how it can be done !!!
Thanks in advance.
Hi Prachi,
I have retail MPOS application, Which is installed in my system in sideload. Can i use Coded UI to test it ? and do i need to install emulator for to run the same. (for manual testing we are accessing from start menu >> select application from tile) ?
Some ideas would be great help.
Thanks
Neeraj
+CodedUI Windows Phone project Issues – could not launch app with AutomationId+
Thank you for this nice step by step tutorial.
I am using :
Microsoft Visual Studio Enterprise 2015
Version 14.0.24720.00 Update 1
Microsoft .NET Framework
Version 4.6.01055
Installed :
Windows Phone SDK 8.0 – ENU
Windows Phone 8.1 Update and Emulators : windowsphone81sdkupdate1.exe
The app to be tested is deployed into Emulator 8.1 WVGA 4 inch 512MB. The app is running fine in emulator.
I have created a Coded UI test project as follows:
File -> New -> Project.
Templates->Visual C#->Windows->Windows 8-> Windows Phone-> Coded UI Test Project(Windows Phone)
Used a proper project name.
In the “Generate Code for Coded UI Test” popup window, select Edit UI Map or add assertions.
Next, drag and dropped from cross-hair tool to my app to be tested. Then copied the value for the appโs AutomationId and pasted to CodedUItest.cs,
public void CodedUITestMethod1()
{
XamlWindow myAppWindow = XamlWindow.Launch(“COPIED_AUTOMATIONID”);
//XamlWindow myAppWindow = XamlWindow.Launch(“{5B04B775-356B-4AA0-AAF8-6491FFEA5660}:Default:”);
//XamlWindow myAppWindow = XamlWindow.Launch(“{9D98AFDC-D8D5-4E31-88B0-7B67C7F9F904}:App:9d98afdc-d8d5-4e31-88b0-7b67c7f9f904_gr9c0mmk005h2!App”); // success when you create an windows phone app using same VS 2015
}
Select the same emulator Emulator 8.1 WVGA 4 inch 512MB
Now, Build and in test explorer, run tests.
It fails.
It fails even if you try to launch default app such Calculator or any other default apps.
It only succeeds when you create windows phone app using same VS 2015
Error message:
Test Name: CodedUITestMethod1
Test FullName: CodedUITestProjectDemo.CodedUITest1.CodedUITestMethod1
Test Source: …documentsvisual studio 2015ProjectsCodedUITestProjectDemoCodedUITestProjectDemoCodedUITest1.cs : line 23
Test Outcome: Failed
Test Duration: 0:00:00.0755546
Result StackTrace:
at Microsoft.VisualStudio.TestTools.UITesting.WindowsStoreApplication.Launch(String tileAutomationId)
at Microsoft.VisualStudio.TestTools.UITesting.WindowsRuntimeControls.XamlWindow.c__DisplayClass4_0.b__0()
at Microsoft.VisualStudio.TestTools.UITesting.CodedUITestMethodInvoker.InvokeMethod[T](Func`1 function, UITestControl control, Boolean
firePlaybackErrorEvent, Boolean logAsAction)
at Microsoft.VisualStudio.TestTools.UITesting.WindowsRuntimeControls.XamlWindow.Launch(String tileAutomationId)
at CodedUITestProjectDemo.CodedUITest1.CodedUITestMethod1()
Result Message:
Test method CodedUITestProjectDemo.CodedUITest1.CodedUITestMethod1 threw exception:
Microsoft.VisualStudio.TestTools.UITest.Extension.PlaybackFailureException: Exception occurred while trying to launch the app. Verify that the app
is installed: Value does not fall within the expected range., AutomationId of Tile: ‘{5B04B775-356B-4AA0-AAF8-6491FFEA5603}:Default:’
As you have suggested to Sai Kiran:
Prachi Bora (MSFT)
@Sai Kiran,
Have you verified the parameter you are passing to app launch? This value is the same as the AutomationId of the tile for your app. This is illustrated in the “launching an app” section in the post.
I have verified AutomationId number of times and also tried with built in Calculator, Camera, Calendar, etc, but no success.
Thanks,
Mitul
Hi,
Please give me a clear picture about Environment setup for Windows phone app testing in coded UI…What are the needs and requirements for the installation…I have Windows OS 8.1
Hi,
Please give me a clear picture about Environment setup for Windows phone app testing in coded UI.What are the needs and requirements for the installation.I have Windows OS 8.1