Killing Me Hardly With Its Trash Talk

I’ve been trying to figure out how to unit test Avalon applications. One might think (I sure did) that anybody building a brand new framework would automatically make it unit test friendly, but when that brand new framework is Avalon one would be wrong. If you want to unit test an Avalon application, I am finding, you have a hard row to hoe.

[Aside: I know that Avalon works the way it does on purpose, not by accident. My various “Why can’t I…” queries always bring well-thought-out answers. In general the Avalon team has focused on making life super easy for application developers. They just seem to have left unit testing out of the equation completely.]

My battles in this arena have been going on for quite a while now, and although I keep running into brick walls (with Avalon jeering at me each time I do), I am determined to win this war. Here’s what I’ve figured out so far.

Model-View-ViewModel (similar to Humble Dialog) is your friend. Start with a Model, wherein reside your business/domain objects. Nothing in this layer knows about presentation or UI and so it is eminently unit testable through automated tests.

Next you need a View Model, which knows how to represent your business/domain objects in your UI. For example, it might have an object that knows how to stuff a collection of Person objects into a list box and that raises a PersonSelected event when one of those list box items is selected. (Avalon concepts like Data Templates, Value Converters, and Type Converters dramatically simplify doing this.) Because these objects are not implicitly bound to specific bits of UI but rather are attached to them at runtime, you can use test automation to unit test each of them as well.

Enter the thorny problem of actually manipulating all that UI from an automated test. You can use Dispatcher.PushFrame to do this directly from your unit test thread, but you have to talk directly to Avalon to do so as UIAutomation purposely does not work from the UI thread. And you almost certainly do want to use UIAutomation. To do that, though, you have to run your UI on a different thread. My first thought was to stuff content into the main window of an Application, but it turns out that Avalon only lets you create one Application object per process. Creating an Application the first time you need one is easy enough, but you need some way to know when to shut it down – which knowledge most test harnesses do not hand out. So that’s pretty much a non-starter. On to option #2, then: showing a Window on its own, without an owning Application. Avalon allows any number of Windows to be created, so this at least has a chance of working; I’ll update this post when I know for sure.

Finally you have one or more Views, which stitch all those View Model objects together. If you are using Avalon the way it’s meant to be used, this will be near-trivial and thus easily verified simply by code inspection. Testing this layer can be simplified by hooking it up to a fake View Model rather than the real one, so that you can just verify that the View Model is poked and prodded correctly rather than having to verify that the right things happen to the underlying business/domain objects. You may want to automate this testing, but I am beginning to believe (heresy!) that doing so is more trouble than it’s worth, especially since all the individual pieces can be unit tested so easily. I don’t have enough experience here to say for sure yet, but I think that every bug you find at the application layer will be a sign of a missing unit test against your model or view model, and that the number of true integration could-only-have-caught-by-testing-the-application-with-everything-hooked-up bugs will quickly approach zero. Techniques like Exploratory Testing may be more productive. I’m not quite convinced yet, but when I finally am (or, alternatively, when I become completely convinced that it *should* be automated) you can be sure I’ll post the news here.


*** Want a fun job on a great team? I need a tester! Interested? Let’s talk: michhu at microsoft dot com. Great coding skills required.

Comments (3)

  1. "Techniques like Exploratory Testing may be more productive. I’m not quite convinced yet, but when I finally am (or, alternatively, when I become completely convinced that it *should* be automated) you can be sure I’ll post the news here."

    I like a hybrid approach of automated and manual testing. The mix of each depends on a lot of factors. In my experience, there’s nothing like a good ET session on a build to find certain classes of unit tests that were missed very quickly. It’s a great way to incrementally move towards more effective unit tests. We don’t know all the problems up front, so it’s a nice way to complement the automated tests. There are too many variables to predict all possible failures all up-front.

    I’m willing to bet you already do some Exploratory Testing without thinking about that term. Cem Kaner, James Bach and Michael Bolton all explain that Exploratory Testing isn’t a testing technique, it’s a way of thinking about testing. My goal is to get testers to be aware of how much they use the best tool we have: our brains. ET doesn’t displace other techniques. As James says, it’s important to remember what is behind all of our tools: our beautiful minds.

    I firmly believe that until computers become intelligent, they can’t really do testing. All automation efforts are computer assisted testing, and the better the testers minds are developing tests, analyzing them and using techniques tools in different ways, the better the testing. The computer only does part of the job, it can’t be curious, aware and investigative like a good tester is. A good brain and a good tool are a powerful combination. These days, the former is not really encouraged, but we pour tons of resources behind the latter, and expect tools to solve problems for us.

    Last time I used a nail-gun, it didn’t build my book shelves for me, but it was a powerful automated tool that increased efficiency and drove nails in better than I can manually. To me there is a balance with automated and manual testing.

  2. John says:

    Quite intresting.I enjoyed reading this.

  3. I wrote last month about my travails in attempting to unit test an Avalon – I mean, Windows Presentation…