Custom UI Automation Providers in Depth: Part 2


In Part 1, we introduced the idea of UI Automation custom control providers and created a TriColor control as a sample.  Now, we’re going to try to add one basic property: the “Hello World!” of UI Automation.  Here’s the sample code for this part.

To do this, we need to create a new object, called a provider, that answers questions on behalf of the control.  That object will implement the IRawElementProviderSimple interface, which defines its contract with UI Automation.  The interface name, although long, points out the important parts of this contract:

  • It is an ElementProvider, distinguishing it from other kinds of providers in Windows.
  • It is Raw, as opposed to the COM interface that a client would use.  This interface is intended to be easy to implement, rather than easy to consume.  (UI Automation itself provides the client objects and interfaces.)
  • This is the Simple interface.  There are more advanced interfaces with different suffixes.

There are just four methods on the interface, answering four basic questions about the object:

  • get_ProviderOptions(): What kind of provider is it? 
  • get_HostRawElementProvider(): What HWND does this belong to?
  • GetPropertyValue(): What are its properties?
  • GetPatternProvider(): What are its Control Patterns? (covered in a later lesson)

So, we simply declare a new object in our project, TriColorProvider, and start implementing this interface, starting with get_ProviderOptions:

        public const int ProviderOptionUseComThreading = 0x20;
       
public override ProviderOptions ProviderOptions
       
{
           
// Request COM threading style – all calls on main thread
            get
           
{
               
return (ProviderOptions)((int)ProviderOptions.ServerSideProvider | ProviderOptionUseComThreading);
           
}
       
}

ServerSideProvider describes where our provider runs: inside the server (UI) process.  Also, I’m using a new Windows 7 feature here – I can tell UIA to use STA COM-style threading, such that all provider calls will be made to the thread that created my provider.  This avoids threading issues when I need to communicate with UI elements on the UI thread. 

The HostRawElementProvider method is similarly easy.  This function returns the HWND provider that corresponds to this control and supplies some basic properties for it, as we noted in the last post.  I just need to call the HostProviderFromHandle() function with my window handle and return its result.  At this point, though, I started thinking: this seems like cookie-cutter code.  If I ever have a second provider, I will have almost the same function body, but a different window handle. 

So, I refactored the class into a BaseSimpleProvider and a TriColorProvider.  The base provider will implement the interface literally, but delegate where useful to protected virtual methods, which the TriColorProvider can override.  This helps to separate what is common from what varies between classes.  If I ever need to write a second provider, I will have much less work to do.  I add the following method to BaseSimpleProvider:

        public IRawElementProviderSimple HostRawElementProvider
       
{
           
get
           
{
               
IntPtr hwnd = this.GetWindowHandle();
               
if (hwnd != IntPtr.Zero)
               
{
                   
return AutomationInteropProvider.HostProviderFromHandle(this.GetWindowHandle());
               
}
               
else
               
{
                   
return null;
               
}
           
}
       
}

And then the derived method to TriColorProvider:

        protected override IntPtr GetWindowHandle()
       
{
           
// Return our window handle, since we’re a root provider
            return this.control.Handle;
       
}

This technique turns out to be extremely useful when we get to GetPropertyValue().  GetPropertyValue() needs to return the value of whatever property the caller requests.  Much like a window procedure, the body of GetPropertyValue() is a big switch statement that routes various property queries to the code that answers them.  To avoid making GetPropertyValue() complicated and long, we factor the various queries into separate methods.  To be clear, you don’t have to do it this way – you can do the work in the switch statement – but it smells bad to me.  Following our pattern above, we add the base GetPropertyValue() method to BaseSimpleProvider, along with a protected GetName() virtual method.  Once we do that, all that TriColorProvider  has to do to say Hello World! is override the name getter:

        protected override string GetName()
       
{
           
// This should be localized; it’s hard-coded only for the sample.
            return "Hello world!";
       
}

We’re almost done.  We have a provider object that provides a complete implementation of IRawElementProviderSimple.  Your code would compile now – but it wouldn’t do anything different.  We still need to hook up our new object.  We’ll add a field to TriColorControl and then a property wrapper that creates the provider on demand (saving memory if we never actually use it):

        private TriColorProvider provider;
       
private IRawElementProviderSimple Provider
       
{
           
get
           
{
               
if (this.provider == null)
               
{
                   
this.provider = new TriColorProvider(this);
               
}
               
return this.provider;
           
}
       
}

Finally, we need to respond to the WM_GETOBJECT window message by returning our provider.  WM_GETOBJECT is the message requesting the Accessibility provider for an HWND – if you are familiar with MSAA, you’ve seen this before.  In UIA, there is one correct way to implement WM_GETOBJECT, and it looks like this:

            if (m.Msg == 0x3D /* WM_GETOBJECT */)
           
{
               
m.Result = AutomationInteropProvider.ReturnRawElementProvider(
                   
m.HWnd, m.WParam, m.LParam, this.Provider);
           
}

You do not need to examine WParam or LParam – just pass them straight through to ReturnRawElementProvider.  This is important to ensure that the UIA-to-MSAA Bridge, which helps MSAA clients to use your UIA object, works properly.  Also note that I’m passing back the same provider object every time.  You don’t have to do this, but creating a new one each time would be wasteful. 

And I’m done.  I can compile and look at my control with Inspect and see my “Hello World!” property coming through:

original[2]

We’ve successfully provided one property!   

Next time: The rest of the basic properties.


Comments (14)

  1. Vallarasu S says:

    Hi There,

    We have a control that has MarshalByRefObject as its subitems, I have been looking around these posts to make our control work with Coded UI Test Builder (VS2010).

    My question is that the subitem does not have a window handle to supply to 'HostProviderFromHandle()'

    In that case what could be the solution or do i have to implement a different provider ?

    Please assist me with this.

    Thanks and Regards

    Vallarasu S.

  2. Vallarasu S says:

    Hi,

    I have them done with IRawElementProviderFragmentRoot, However i encounter another exception with the subitems.

    "could not find accessible object of foreground window ", any idea of what causes it?

    Is there any way i can tell the builder which item has the focus?

    Best Regards

    Vallarasu S.

  3. Vallarasu,

    Blog comments aren't monitored as much as our MSDN forum is — you might have more luck here:

    social.msdn.microsoft.com/…/threads

    Thanks,

    Michael

  4. kumar says:

    the link " Here’s the sample code for this part." is not working

  5. YOUR CODE DOES NOT CLEAN UP- TITLE SHOULD BE COPY AND PASTE CODE FROM BEGINNER says:

    MSDN STATES

    When Microsoft Active Accessibility clients are listening to events raised by a UI Automation provider, UI Automation maintains a map of the providers that have raised events. When the Microsoft Active Accessibility clients request further information, UI Automation uses the map to route the requests to the appropriate providers. When a window that previously returned providers has been destroyed, you should notify UI Automation by calling the UiaReturnRawElementProvider as follows: UiaReturnRawElementProvider(hwnd, 0, 0, NULL). This call tells UI Automation that it can safely remove all map entries that refer to the specified window. This call can save memory because it releases references to the providers being held by the raised-event map. The function returns zero when called with these special parameters. Microsoft recommends making this call from the WM_DESTROY message handler of the window that returns the UI Automation providers.

    YOUR COPY AND PASTE CODE DOES CLEAN UP. I CAN FIND IDENTICAL EXAMPLES OF YOUR CODE ALL OVER THE INTERNET. THE TITLE IN DEPTH IS BS YOU SHOULD CHANGE YOUR TITLE TO COPY AND PASTE CODE FROM A BEGINNER

  6. Interesting suggestion. In the context of a standalone application, this cleanup is not strictly necessary.  When a process terminates, it's memory is always cleaned up.  

    But this cleanup is helpful if the application creates and destroys a lot of different HWNDs (which my sample does not).  I'll consider incorporating that the next time I revise this sample.

  7. Ray says:

    Hey All,

       The Definition of IUIAutomationElement.UIA_RuntimeIdPropertyId states that: "Identifies the RuntimeId property, which is an array of integers representing the identifier for an automation element.

    The identifier is unique on the desktop, but it is guaranteed to be unique only within the UI of the desktop on which it was generated. Identifiers can be reused over time.

    The format of RuntimeId can change. The returned identifier should be treated as an opaque value and used only for comparison; for example, to determine whether an automation element is in the cache."

    The part about "Identifiers can be reused over time." is unclear to me, can anybody explain what that means more clearly, specifically i want to know if this is unique per instance of the given application? Is this runtimeid reused in the same instance of the given application?

    Thanks in advance,

    Ray

  8. Hi, Ray,

    The point about identifier re-use is that you could have an element 'A' with a runtime ID (ID(A)), and then have 'A' be destroyed.  Later on, an element 'B' is created and it turns out to have the runtime ID = ID(A).  

    This commonly happens in containers, where the container's runtime ID might be a concatenation of the container's ID plus an item position.  If you have a list of items, and item #5 has a particular ID, and then you refresh the list, you might find that the new item #5 has the same runtime ID as the old item #5.

    Runtime IDs need to be unique at a given point in time, but not unique across all points in time.

    – Michael

  9. Eric says:

    Michael – is it possible to imlpement an automation provider for a Visio document?  One that would show the shapes as navigatable items, and properties such as location, context menus, etc?

    thanks,

    -es

  10. Guy Barker (MSFT) says:

    Hi Eric, I don't think it's possible for you to inject your own UIA provider such that a UIA client interacting with Visio will get custom results from your provider. But I've just been pointing the Inspect SDK tool to Visio 2010 to see what's currently exposed in a diagram. I posted the results of my test at social.msdn.microsoft.com/…/318cf9ca-4c36-4492-badd-b4a2f7fc632f. Do you think that the existing Visio functionality might be sufficient for your scenario?

    Thanks,

    Guy

  11. Venkatesh K says:

    Can anyone help me with the bounding rectangle problem ? How to get the same for Toolbar button..plse help..

  12. Please bring your question to the MSDN forum for Accessibility:

    social.msdn.microsoft.com/…/threads

  13. Qwicks says:

    I program mainly in Vb.net so this example was a little too complex for me because of the Tri-Color

    control and the way it was embedded in the code, it made it completely impossible to reverse engineer so i could understand. After about a month i found this Simple Provider Sample in VB.net. I hope this helps somone. technet.microsoft.com/…/ms771658(v=vs.90)

  14. Lucy says:

    This might be irrelevant, but I searched online without a good solution, so I post here to see if there is any hint to solve it.

    When I try to find a simple button in my product, an error throws:

    [ERROR] Create a recode: Exception from HRESULT: 0x80042002

     System.Runtime.InteropServices.COMException (0x80042002): Exception from HRESULT: 0x80042002

        at System.Runtime.InteropServices.Marshal.ThrowExceptionForHRInternal(Int32 errorCode, IntPtr errorInfo)

        at MS.Internal.Automation.UiaCoreApi.CheckError(Int32 hr)

        at MS.Internal.Automation.UiaCoreApi.UiaFind(SafeNodeHandle hnode, UiaFindParams findParams, Condition findCondition, UiaCacheRequest request)

        at System.Windows.Automation.AutomationElement.Find(TreeScope scope, Condition condition, UiaCacheRequest request, Boolean findFirst, BackgroundWorker worker)

        at System.Windows.Automation.AutomationElement.FindFirst(TreeScope scope, Condition condition)

        at SXAuto.Support.Control..ctor(ControlType type, String name) in D:lucydocumentsAutoSXAutoSXAutoSupportControl.cs:line 64

        at SXAuto.Steps.Test.ClickButton(String name) in D:lucydocumentsAutoSXAutoSXAutoStepsTest.cs:line 74

        at lambda_method(Closure , IContextManager , String )

        at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)

        at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)

        at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(StepInstance stepInstance)

        at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()

        at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()

        at SXAuto.Features.TestFeature.ScenarioCleanup() in D:lucydocumentsAutoSXAutoSXAutoFeaturesTest.feature.cs:line 0

        at SXAuto.Features.TestFeature.CreateARecode() in d:lucydocumentsAutoSXAutoSXAutoFeaturesTest.feature:line 21

        at TechTalk.SpecRun.Framework.TaskExecutors.StaticOrInstanceMethodExecutor.ExecuteInternal(TestThreadExecutionContext testThreadExecutionContext) in c:TeamCitybuildAgentwork1ace6ed01d0a43bbTechTalk.SpecRun.FrameworkTaskExecutorsStaticOrInstanceMethodExecutor.cs:line 40

        at TechTalk.SpecRun.Framework.TaskExecutors.StaticOrInstanceMethodExecutor.Execute(TestThreadExecutionContext testThreadExecutionContext) in c:TeamCitybuildAgentwork1ace6ed01d0a43bbTechTalk.SpecRun.FrameworkTaskExecutorsStaticOrInstanceMethodExecutor.cs:line 21

        at TechTalk.SpecRun.Framework.TestThreadExecutor.ExecuteTestNodeTask(TestNode testNode, ITaskExecutor task, TraceEventType eventType) in c:TeamCitybuildAgentwork1ace6ed01d0a43bbTechTalk.SpecRun.FrameworkTestThreadExecutor.cs:line 220

Skip to main content