A UI based automation test is failing

  • Article

Last week I had an automated test I own begin to fail. The functionality tested is our keyboard shortcuts to expand/collapse outlines. SHIFT+ALT+1 shrinks an outline to just level 1 items, and SHIFT+ALT+0 expands it all back.

In other words, suppose I have

Line 1
Line 2 (indented)
Line 3

When I put the cursor on Line 1 and type SHIFT+ALT+1, I get this:

clip_image001

Notice where the cursor is, by the way. It's just to the right of the 1.

And SHIFT+ALT+0 gets me back to the original presentation.

The difficult part about the investigation is that the test is not failing due to a OneNote bug. I very quickly verified the behavior manually and we work just like we always have. So the test is failing.

I looked through the test code and noticed that to perform this test, we are performing quite a few SendKeys methods to send keystrokes to the current page. If you have never used this before, you can read about it here: https://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.aspx

Notice one of the Notes near the end:

"The SendKeys class is susceptible to timing issues, which some developers have had to work around. The updated implementation is still susceptible to timing issues, but is slightly faster and may require changes to the workarounds. The SendKeys class tries to use the previous implementation first, and if that fails, uses the new implementation. As a result, the SendKeys class may behave differently on different operating systems. "

While I am investigating this failure, I also want to keep in mind this article I found: https://msdn.microsoft.com/en-us/library/dd375731%28VS.85%29.aspx Ths article covers how to ensure I am simulating presssing the Number Pad 1 key vs. the 1 key across the top of the keyboard. OneNote uses the 1 key on top of the keyboard so using the number pad key will not work.

So as I write this, I am investigating what may be going on here. I'll keep you posted on results. This is a rather typical test task, by the way. An automated test that has been passing for quite some time now fails and needs attention. I need to track down what might have changed and get this test back up and running.

One last thing I will point out is that I am not a fan of UI based automation. If I can rewrite this test to avoid the UI I will - especially considering the MSDN note about timing issues and Sendkeys working differently on different operating systems.

Questions, commnents, concerns and criticisms always welcome,

John