Usability and prototypes

It's been a while since I last posted and the main reason for my silence has been that I've been busy preparing and running a usability study this week which involved the creation of an HTML prototype to mock up the UI of the product we were studying.

The study is investigating the user experience for a portion of the Whidbey product that has yet to be fully implemented. In order to get good feedback about the design from participants we needed to create a prototype of the product. I suggested that we create HTML prototypes with separate web pages for each step that participants needed to complete in the usability study. Each page would contain a screenshot of VS in a particular state (e.g., the solution explorer showing the current project selected) and would have a set of hotspots within which the user could click to move on to the next page in the sequence. With correctly defined pages and hotspots we could simulate different actions that the user could take with VS. For example, to simulate a right click on a project in the solution explorer, we would have one page containing an image of the Visual Studio IDE with the solution explorer open and a hotspot over the appropriate project in the solution explorer. When the user clicks in the hotspot, we show the next page containing an image of the VS IDE with the solution explorer open as well as the context menu for the project that the user clicked on.

It seemed like a good idea but I underestimated how much work it would involve. For example, to accomodate the different ways that a developer could create a new project in Visual Studio required that we create almost 50 separate web pages with links and hotspots relating each of the pages together as appropriate. To create prototypes for each of the four tasks in the study eventually required two of us to spend three full days creating images etc.

The effort was worthwhile though. We're finishing up the study today and we've been able to use the prototypes to get some pretty useful feedback. It's reasonably early on in the lifecycle of this feature that we should be in a good position to respond to most of the feedback we've collected. You can tell the prototype has worked when you have to keep reminding participants that they are working with a prototype and that what they are looking at is just a bitmap and so they won't be able to scroll through the code they are examining, let alone change the code.

An interesting side effect that I think arises from the fact that we were using a prototype is that participants were more likely to talk about the interaction flow as opposed to the lower level details of the UI such as button labels, colors, positions etc. I think the nature of the prototype made it more obvious to participants that we were more interested in how the different parts of the product flowed together in the different tasks. So the effort involved in creating the prototype, although large, was definitely worthwhile. Something to keep in mind if you're thinking about using prototypes in one of your own usability studies.