Usability Testing the WF Designer vNext (or, Yelling at Customers)

One of the things that my team is working on is the next version of the workflow designer.  In order to help us get real feedback, we engaged with our usability teams to design and execute a usability study. 

For details on what the test looks like (when we did them 3 years ago for the first version of the WF designer, see this great channel9 video).  The setup is still the same (one way glass mirror, cameras tracking the face, screen, posture of the subject), the only difference is the software, we're busy testing out some new concepts to make workflow development much more productive.  At this stage of the lifecycle, we're really experimenting with some different designer metaphors, and a usability test is a great way to get real feedback.

One thing I've always tried to do since I came to Microsoft is being sucked into the Redmond bubble.  The symptoms of placement inside said bubble are a gradual removal from the reality that everyday developers face.  When I came to the company two years ago, I was chock full of great thoughts and ideas from the outside, and much less tolerant of the "well, that's just how it works" defense. 

Slowly, though, as you start to get deep into thinking about a problem, and tightly focusing on that problem, those concerns start to fade away, as you look to optimize the experience you are providing.  Sitting in on the usability labs yesterday was a great reminder to me of how easily one can slip into the bubble.  Our test subject was working with a workflow in the designer and had a peculiar style of working with the property window in VS.  Now, when I use VS, I use the property grid in one way.  I have it docked, and I have the dock set to automatically hide.  I have known some developers who prefer the Apple / photoshop style where the property pane floats.  The customer's way of working with the property grid was that he had it floating, but he would close it after every interaction.  This required him to do one of two things in order to display the grid again, either go to the View menu, or (and what his style of work was) right clicking on an element and selecting properties.

The prototype we were doing the usability testing with, however, does not have that feature wired up, in fact, it currently doesn't displaimagey the properties item in the context menu at all.  Not because we have an evil, nefarious plan to remove the properties item inconsistently throughout our designer, but rather because no one gave it any thought when we put the prototype together as we had other UI elements we wanted to focus on. 

This became a serious problem for our customer, as the way he expected to work was completely interrupted.  At one point, we asked him to dock the property window so we could continue with the test.  This is the most fascinating part of the study to me, and that was watching him work to dock the property grid in the left panel.  I've become so used to the docking behavior in VS (see screenshot below), that it didn't even occur to me that this might present a problem for the user.  Instead, we watched for 3 minutes or so as he attempted to figure out how to move the window, and then try to process the feedback that the UX elements give.  About 60 seconds in or so, the property grid was about at a similar location to the screenshot, with just a centimeter or two's distance away from being in "the right place".  Watching his face, we saw him look slightly confused and then move it elsewhere.  Two more times he came back to that same spot, just far enough away to not get the feedback that might help him in the right direction.  It was at this point, the spontaneous yelling started among the observers in the room.  Something that has become so obvious to us, something we have internalized and accepted as "just the way the world," was becoming crystal clear to us how much difficulty this was causing.  The yelling was things like "Move up, move up" "no, wait, over, over" "oh, you almost, almost, no...." trying to will the customer through the soundproof wall what we wanted him to do.

This situation repeated itself time and time again with different UI elements, and it was very, very educational to see the way different users manage their workspace and interact with a tool that I've become so familiar with that I forget to see the forest for the trees.  I also realized, that although I had worked with a lot of customers and other developers, very rarely had I paid attention to how they work, rather than simply their work. 

Now, here's where I open up the real can of worms.  We're looking to make usability improvements in the WF designer.  Are there any that really bother you?  What can we do to make you a more productive WF developer? 

Comments (13)

Cancel reply

  1. Benoît Dion says:

    Hello Matt,

    It’s true that docking a window in VS is not easy..the first time you do it. Once you get it, it’s really not a problem anymore.

    In a previous post you asked some feedback about refactoring. In addition to the one you offered, a "convert to XAML workflow" refactoring could be interesting.

    To improve the usability of the WF designer, it would be nice to be able to resize activities. Sometimes, the name of an activity is too long and isn’t shown entirely. It is not really a problem when developing a workflow but it is if we want to print the view of a workflow or use it in a report.

    A better understanding of XAML only workflows in VS would be nice.

    I have read a lot about the "VS becoming slow" issue when workflows contains too many activities or a project contains too many workflows. But it seems there will be some improvements in this field in VS 2008 SP1. (About that SP1, anything else concerning WF?)

    To improve productivity, I currently have to close VS 2-3 times a day, empty the ProjectAssemblies directory and restart VS (I am using VS 2008 on Vista). It would be nice to get this fixed as well as the "VS forgets the layout of state machine workflow" issue.

    It would be cool to have some GUI tool to generate activity validators, activity designer and activity designer themes.

    In a previous post, you asked us to take a survey about rehosting the workflow designer. Too bad for me, I missed it :(. So here is my opinion about it: I use it to let non programmers create XAML only workflows. I think it is a really nice feature but it would be even better with an easier programming model. A WPF wrapper?

    If you don’t mind, I would also like to give you some improvement idea of WF in general:

    – Generic activities support (or at least generic base classes).

    – Not having to write WorkflowMarkupSerializer anymore.

    – Provide access to TimerSubscription through a runtime service instead of (read "as well as") through the workflow.

    – All my custom UITypeEditor are in Winform. I remember trying without success to make them in WPF. Maybe it’s just me and then ignore this idea.

    – Merging WPF and WF dependency properties

    I know the two first suggestion are difficult to implement because of XAML. The third is not really needed. It just goes better with the WF programming model.

    Could you tell us what will be released in .NET 3.5 SP1 concerning WF?

    Sorry for this long post which is not always related to your post but I thought it would be good place to write it here. I really enjoy working with WF and didn’t want to criticize it too much but it’s how things move on, isn’t it?

    Feel free to contact me if you need more feedback 🙂

  2. Benoît Dion says:

    Sorry but I forgot to ask one last thing in my previous post:

    Pageflow and BPEL for WF have been in CTP for months. Are you planning on updating these? If the answer is "No" (which would seem logical to me), don’t you think it would be a good idea to release BPEL for WF in a codeplex project?

  3. Benoît Dion,

    Pageflow is not a CTP, it is a sample, so there is no official roadmap for it (although the patterns and practices group has been looking at it).  With regards to BPEL, I’m not sure what the status on that is, I’ll send an email around internally.

    In .NET 3.5 SP1, there are no features introduced, but there have been a number of internal improvements made, including some substantial perf gains in the WF designer (for certain scenarios).  I’m not sure if those are in the beta or not.  

    Thanks for all the suggestions!

  4. Benoît Dion says:

    Thanks Matt,

    A few more questions and I’ll give you a break for a week or two :).

    Is it possible to have a mid/long term roadmap concerning WF?

    I heard a rumor about a new WF community web site. Can you confirm it?

    Is there any plan on a deeper integration between WF and UI technologies (WF as a controller)? Do you think it could simplify developement of complex UI (wizards, etc) or just make everything more complicated?

  5. kwaazaar says:

    I would like to see some timeout-feature for activities, at least for EventDrivenActivity, including an alternative view/handler (just like the fault- and cancel-handlers).

    It’s already possible to implement timeout-logic, but I’d prefer to simply configure the duration and implement the Timeout-handler. Perhaps even support multiple timeout-handlers to enable sending reminders, etc.

    Furthermore, VS is pretty slow when used in a virtual machine. Especially expanding the toolbox is pretty slow, which is why I have it expanded all the time. Maybe loose (or simplify) the animations, cache icons, whatever, but speed it up please!

  6. From my experience the WF designer is very  unpredictable. Let say I work on a project and there are days when WF designer works fine and there are days when it crashes VS 2005 a few times a day. What is more it always crashes VS 2005 when I want to autogenerate a handler of an event using Properties windows.

  7. @kwaazaar, do you mean that VS in general is slow in virtual machines, or that the workflow designer is noticably slower than the rest of VS when in a virtual machine?

  8. Benoît Dion says:

    Something I forgot, which is maybe more important than everything else:

    When an activity is renamed, bindings should be updated in consequence.

  9. lcorneliussen says:

    I don’t like the way error handling and cancelation is handled in the designer.

    You can’t even see whether there is some ErrorHandler or not. If you click to see, you created one. The icon could indicate the status by beeing a little bit transparent if no error/cancelation handler is registered yet.

    I would also like to switch the whole view to "error handling" or "cancelation" mode.

    Generally WF should help better with handling workflow changes for running workflows. Real Versioning (not only CLR versions), etc would be great. Just beeing able to submit WorkflowChanges by code is not enough.

    By the way, have you ever thought of an 3D WF Designer? I just have some blury ideas up there in my head – can’t really describe them. But this could be a cool thing!

    Will the new designer be done in WPF?

    Regards from Germany,


  10. cberthold says:

    In my use I think it would be nice to be able to split the views up a little bit.  It would almost be nice to be able to do two types of drill downs; drag and drop; and like a CTL-DBL-CLICK which would open a new window without leaving the main design surface. It seems like most developers these days have 2-3 monitors.  At least I do.  Sometimes I find myself looking at one event-driven activity and realizing I also need to a) copy and paste some activities (I know refactor into a custom activity) or b) just trying to refresh myself on what will be happening in the next/previous event-driven activity in another state.  Being able to open multiple windows of the same workflow would be AWESOME.

  11. MarkBosley says:

    On the topic of rehosting the designer I would like to see the WorkflowView reworked so it can be rehosted in silverlight. I know this would also require changing the out of box activities since the attached designers are using gdi but I believe going forward more business apps will be built in silverlight. ASP.NET Intranet Apps are under threat. long live silverlight!

  12. MattJordan says:

    A really useful feature, would be to provide > go to definition in the context menu when you have selected a custom activity in a workflow.

  13. CoderX says:

    1. Make if faster. It is painfully slow on anything larger than trivial workflows. Just wiring up an external method’s arguments takes minutes, when it should take seconds.

    2. State Workflows don’t keep their layout. This has caused more hair pulling and swearing than anything else in our office. I can’t even print a workflow because my layout is completely hosed by the time it hits the printed page. Never mind when the layout designer spontaneously decides to put nodes in places they can’t even be seen (with connectors ending in thin air) and I have to re-size nested state rectangles for several minutes just to find them again. The designer should leave nodes and connectors where I put them. Period.

    3. Give us different layout choices for Events. I have some states that take dozens of events and despite the state being quite wide it insists on stacking them all in one ridiculously long column, when it should wrap them to make better use of the space.

    4. Give us real versioning. We NEED to update our workflows in the middle of long-running processes and the existing update mechanisms are not good enough. We have to resort to keeping our workflow instance states in a DB and rebuilding the WF instances when we update..

    5. Please, please, please give us the option to not kill a WF instance when it has an unhandled exception. In long-running state workflows this is disastrous. If an event handler throws an unhandled exception just leave the workflow be (and in the state it was in). Killing the whole thing is like scuttling a battleship because someone can’t find a mop to swab the deck.

    6. Do everything you can to support Silverlight. We’re using WF as the backend to our SL/WCF apps and it is not trivial. The built-in web-service WF support is useless for SL.

    7. Give us "static/class-level" events for workflows. These would run independant of any workflow instance (and be able to call instance events). This could be very useful for many situations.

Skip to main content