of babies and bathwater


A couple of weeks ago, Mary gave us a behind-the-scenes look at talking to the press. One of the articles that came from the interview that she talks about is in APC: First details of Office:Mac.

In that interview, Mary was kind enough to give usability testing a shout-out. Here’s the relevant part of that interview:

Design and usability testing on the Office 12 interface is already underway in the MacBU labs at Redmond and Cupertino, and the team has already made one trip back to the drawing board based on user feedback.


“We have usability experts and usability labs at both of our campuses, and we’re spending a lot of time bringing people through for each iteration of the UI. That’s part of why it’s changing so much right now” explains Starman.


“We had what we thought was going to be this perfect UI solution, and the first time we put it in the labs, no-one understood it! It was so different they were completely confused! We just finished up another round of usability testing on the new UI yesterday, and the program manager said the difference is like night and day.”

This is such a great example of what I do and why I do it. We are refreshing our UI (don’t worry! we’re not killing your toolbars!). We put in a lot of effort to design the refreshed user interface. But we’re not perfect, and our original design wasn’t perfect. When we took our new UI into the usability lab, it didn’t do as well as we’d hoped. Users were really confused. So we went back to the drawing board.

Going back to the drawing board doesn’t mean that we threw the whole thing out. There was definitely a baby in that bathwater. We had to figure out what worked in our design and what didn’t. For what didn’t work, we had to figure out why it didn’t work. Were we totally wrong, or were we just a little bit wrong?

So we came up with some ideas for fixing the issues that we observed. For something as big as a UI refresh, this is quite a big deal. There were lots of discussions and brainstorms. We decided what we would change.  Mostly, they were pretty minor tweaks: making something look a bit more like a button, changing a name, that kind of thing.

The week that Mary and Sheridan did that interview, we were in the usability lab trying out the new design. Those little changes that we made had a huge impact. In the original study, none of the users really got it. In the second study, the difference was, as the PM that Mary mentions in the article said, like night and day.

Of course, it still wasn’t perfect. It might not ever be perfect — possibly because perfection doesn’t actually exist for something that’s as complex as what we’re doing. So we made a couple of extra changes. Right now, I’m writing this blog post while sitting in the usability lab to see if this change gets us where we want to be. While I’m here, I’m also testing out a couple of new features that I didn’t have enough time to test earlier. I’m not done with this study yet, but the preliminary data is fantastic. I think we’ve gotten what we wanted: a new UI that looks great and that helps our users do their jobs easier.

Before anyone asks, I can’t answer specific questions about it yet. Not all of our functionality is hooked up, so it’s still not finalised. And while I’ve been in the lab this week, I’ve discovered that there are other things that I want to look at. We solved the big problem, and the existence of the bigger issue hid many of these smaller ones now. I’ll be back in the usability lab again to look at other related items. (Although our labs are in Redmond and Mountain View, not Cupertino.) If you’d like to join me and get a sneak peek at the new UI and features, sign up to participate in our usability tests!

Edited to fix a broken link. Thanks to the anonymous commenter for pointing that out!

Comments (27)

  1. anonymous says:

    you might want to fix the link at the bottom of this article.

  2. Office for Mac says:

    She just fixed it. Thx!

  3. John Muir says:

    It’s good to hear about the new UI in the works, as the Win Office 2007 beta is definitely an improvement in terms of usability so we know you have a challenge on your hands!  Playing the role of Joe Mac Average, I’ll pose a couple of questions we’d like to hear your thoughts on but I know not necessarily to expect a reply… 😉

    1.  What’s the reason the Office team for Windows go for public betas whereas their Mac counterparts don’t?  I understand that Office 12 may well be further from release than Office 2007, but I’m sure I’m not alone in wishing there would be a public beta for us to get our teeth into once it’s well advanced.  Alas I’m an ocean away from the usability labs.

    2.  Since Office 12 will be universal binary and you’re cooking up new interfaces, what’s your direction with regards to Cocoa and general OS X look and feel?  Office 2004 is by no means poorly presented, but you know how many of us are with Cocoa v Carbon styled interfaces.  Office 12 seems like a good opportunity to re-jig the development process and embrace the big trends so many Mac apps have been following.  For instance, the system wide dictionary available in every Cocoa app isn’t accessible in Word, I hope many such integration issues are in for a refresh when we do get to see our new Office!

    I’m not in software development myself so forgive any confused labels if they’ve arisen.  I’m just like so many Mac people: a user with a somewhat technical eye.  Keep the blog rolling even if you can’t quite address the big questions head on right now!

  4. MacBill says:

    Why must you guys reinvent the wheel? Can’t you leave good enough alone? We’ve reported dozens of bugs over the last 5 years that still aren’t fixed in Entourage & Word, yet you guys are completely overhauling an interface that mostly works just fine?

    The new “Track Changes” functionality, however is awesome — how it now puts the changes into the side margin instead of in the text itself.

  5. John — Big public betas require a lot of resources to manage.  We’re a very small team, much much much smaller than the WinOffice team.  We don’t have enough people to be able to adequately handle a big public beta.  If I recall correctly, one of the other bloggers is planning on talking about our beta programme in the future, so stay tuned for that.

    Bill — We have to balance adding new features and fixing old issues.  You like a change to a particular feature, but I bet that someone else would say that we shouldn’t’ve done that instead of fixing a bug that they’ve reported.  Everyone has different priorities. 🙂

    One of the reasons that we’re refreshing the UI is to solve the discoverability problem: a feature isn’t useful to to our users if they can’t find it.  

  6. Paul Berkowitz says:

    “Design and usability testing on the Office 12 interface is already underway in the MacBU labs at Redmond and Cupertino, and the team has already made one trip back to the drawing board based on user feedback. “

    Cupertino? Surely Mary did not confuse Microsoft with the Apple campus? Did the interviewer just invent that? Or have you left Mountain View? (Or maybe the interviewer thinks it’s the same town?)

  7. Paul Berkowitz says:

    Comments are still off on the”Funkiness” post, although everything looks good now. So I’m commenting here. In fact, somebody fixed the “yellow bar too short” problem while they were at it. That’s  good – it was driving me nuts. Much nicer now that the date doesn’t sit half in, half out. Very nice now.

  8. opensourcefan says:

    I hope you will take these comments back to your usability lab since my MAJOR metropolitan area isn’t included in your “test centers”.

    Please, please, please adopt STANDARD Mac OS keyboard shortcuts!!!!!!! Please already! Not one stinking Control key anywhere! Make EVERY menu item have a keyboard shortcut (e.g. “Save as” is CMD-Shift-S) already!

    Please let me be able to turn off any and every “helpful” feature and to choose things my way. For example, I am infuriated every stinking time I try to print something with track changes and it reverts to the stupid balloons that I turned off. Guess what! I like saving trees and dont’ need my papers to look like the kindergartners were coloring on it!

    Please make all those stupid button things in Excel that never have any information and get in the way get out of the flipping way already! It’s soooooo annoying to try to edit a cell and have some freaking yellow button refuse to uncover the cell that when you click has not one single recommendation.

    AAaarghhh.

    If you want usability make it simple. Word/Excel/Powerpoint have gotten way too complicated.

  9. John Muir says:

    Thanks for the reply on my beta question.  While I won’t be looking out for Mac Office beta, I’ll still keep my fingers crossed for a more Cocoa consistent look and feel.  The new UI on Office 2007 isn’t bad (huge improvement over previous windows versions in my opinion) so dare we expect such bold steps in your own redesign?  I hope so!

  10. flyermoney says:

    Disclaimer: understand that I’m not part of the beta program, so my comments are those of an outsider.

    For a coherent OS X experience, I’d really love to see the next Mac:Office apps to have the Cocoa-style toolbar and inspector palettes (think OmniGraffle or OmniOutliner; actually take a long and studious look at the simple and excellent GUI of those two apps), seen in all well behaved and modern Mac apps. Pity, actually, that The OmniGroup aren’t in charge of Adobe CS 3! 😉

    It’d be a wrong choice to sacrifice UI coherence to Windows Office UI legacy (or cross-over look’n’feel). Mac:Office ARE Mac OS X applications, so they SHOULD act as such.

    Speaking of system-wide Cocoa services (one poster mentioned spell-checking) I guess you’re implementing Cocoa text services as well, such as alt-arrow word navigation, alt-shift word selection, ctrl-arrow line and paragraph navigation, on-the-fly alt-escape mid-word suggestion pop-up list, ctrl-command-D Dictionary pop-up and so on and so forth.

    Take care.

  11. SL says:

    Very much looking forward to Mac Office 12.  I love the Win Office 2007 beta, and hope you take some of those ideas.  Alternatively, toolbars like as in Apple’s iWork are quite nice too.  Mac Office 2004 was the best office version back in its day too so I’m confident the team will do a great job with Office 12.

    Maybe not the right place, but I was wondering about the following:

    Fonts:  How come Office (2004, at least) seems to treat fonts differently from every other Apple program?  E.g. I installed some of the Office 2007 fonts (e.g. Cambria) on the Mac, and Office 2004 displays it as “Cambria” “Cambria Bold” “Cambria Italic” — but isn’t smart enough to choose “Cambria Bold” when I click the “Bold” button on the toolbar. Instead, it comes up with a faux bold which looks weird.  (applications which use the Font panel switch over to Cambria Bold automatically…)  Will Office 12 be smarter about fonts?

    Open format support:  When will we get filters for the Office 2007 file formats?  I sure hope I can open Office 2007 file formats (docx, etc)  when the Windows version is released, without having to wait for/pay for the next version of Mac office.  (NB the Windows version of 2000 and 2002/2003 can already open docx with a beta plugin by Microsoft…)

    And more to the point on usability testing – what about usability testing in other locales?  I could think of Asian markets e.g. China and Japan; as well as places where people read right-to-left being key places where you might want to conduct focus groups too…  (This is something Apple isn’t good at either – e.g. the Address Book insists on dates that go M/D/Y even when you set the setting to D/M/Y…)

  12. joecab says:

    Usability tests? Going back to fix what didn’t work? This is not the Microsoft we Mac users have been brainwa.. er, trained about.* O:)

    Glad to hear this, actually. I trust you folks to get it working. And I hope AppleScript/Automator gets plenty of attention now that we’ve lost VBA.

    * Apple still has a great track record for usability, which is surprising since most (all?) of their early testing is done in house from what I hear. But even they’re not perfect: I still don’t like that damn Dock, even though I’m sued to it. I should be able to tell at a glance what application is what and what’s running. Grr.

  13. Odysseus says:

    “”We had what we thought was going to be this perfect UI solution”

    This is a perfect example of the observation that MS developers sometimes don’t have a clue 🙁

  14. INTPMann says:

    I have a stock saying on Microsoft applications (in both Windows and Mac OS X): “Any feature you want is in there. Somewhere. You’ve just got to find it. Somewhere. You’ll spend 15 minutes trying to find some function, and when you finally do it will do _almost_ exactly what you want it to do. But not quite…”

    It’s various problems, such as poorly named menu items, or functions located three levels deep in a tabbed window. It’s evident that the geeks drive the UI more than the UI experts themselves, and it should be the other way around.

    Give me multiple ways to get to functions, and let me tweak my UI to suite my style. Give me logically arrayed menus. Give me extensive use of the cntrl-click to access context-sensitive functionality if that’s my style, but give me one or more ways to do it it that’s not my style. How about a floating toolbar for context-sensitive functions? And I second the comment someone made about turning the damned tracking off for printing and having it stay off.

    Fix all that, and people will actually ENJOY using Office…

  15. John Muir says:

    joecab: I pulled all the icons from my dock so it only shows running apps now.  I use Quicksilver as an app launcher which I find much handier than mousing anyway.  Makes for a much cleaner interface, espeically down on 1024×768 which I hope Office 12 will still be good for, hint hint.

  16. SL — Our main facilities are in Redmond and Mountain View, and we do the bulk of our usability testing here.  However, we also travel around the US and the rest of the world to meet with our users and get additional perspectives on what we’re doing.  I understand that usability people who work on our various Windows-based apps (and, of course, the Windows OS itself) talk about the Redmond Effect in terms of usability testing, and do a lot to try to ameliorate that.  

    We will release file format converters for Office:Mac 2004 so that you’ll be able to open files created on Office 2007.  

    opensourcefan – Our testing centers are where MacBU itself is located (about half in Redmond, and half in Mountain View).  We do travel frequently.  If you haven’t yet, signing up to be a part of our usability tests means that we might call you in the future to participate in a usability test.

  17. opensourcefan says:

    Nadyne, OK. I’ll sign up in case you ever get to this part of the country. But you have to promise to never let anyone know I was fraternizing with Microsoft. 😉

  18. joecab says:

    Yep I have QS installed too (heard about it from Merlin Mann’s 43 Folders) but I still haven’t gotten around to really learning about it and using it properly. Sigh.

  19. OSfan — I promise that I’ll leave my MSFT ROX tattoo kit at home if I ever meet you. 😉

  20. I have no idea if anyone here isn’t reading Mac Mojo , but in case not, my latest post over there is

  21. SL says:

    nadyne – cheers for the reply.  just one thing – will we also be able to  can we also *save* in the new docx xlsx pptx formats?  (It would help in a mixed Office 2004:mac and Office 12:mac environment to be able to standardise on/use the new formats when people haven’t all upgraded…)

  22. vgolf says:

    Quote: “Give me multiple ways to get to functions, and let me tweak my UI to suite my style. Give me logically arrayed menus. Give me extensive use of the cntrl-click to access context-sensitive functionality if that’s my style, but give me one or more ways to do it it that’s not my style. How about a floating toolbar for context-sensitive functions?”

    Oh please don’t! Give me a *single* easy, logical, and most of all consistent way to do each thing, and then stick to it! There is no other way that productivity can be improved.

    And visit humanized.com website regularly.

    P.S. This shows how diverse your user base can be. Oh boy, I don’t envy you at all…

  23. Schwieb says:

    SL — regarding your question of font behaviors…  Yes, this is an area where Office 2004 doesn’t get it right.  I believe Office 12 will do a better job with font variants.

    Also SL, regarding being able to save to the new Open XML formats, yes, the converters will let you do that.

  24. It will be interesting to see what you come up with for Office 12. In terms of discoverability, I have not had as much problems as on Office for Win which has been improved in version 2007. Office for Mac presently has a convenient UI through features such as the Formatting Palette.

  25. Jon says:

    – – – – – – – – – – – – – – – – – –

    “Quote: “Give me multiple ways to get to functions, and let me tweak my UI to suite my style. Give me logically arrayed menus. Give me extensive use of the cntrl-click to access context-sensitive functionality if that’s my style, but give me one or more ways to do it it that’s not my style. How about a floating toolbar for context-sensitive functions?”

    – – – – – – – – – – – – – – – – – –

    “Oh please don’t! Give me a *single* easy, logical, and most of all consistent way to do each thing, and then stick to it! There is no other way that productivity can be improved.”

    – – – – – – – – – – – – – – – – – –

    Well, it sounds to me like Microsoft should continue on its present course. Because, obviously, the “perfect” configuration for one person is “hell” for another.

    Office should have a single DEFAULT easy, logical, consistent way to do things, but here’s the problem. The easiest way to do something is not always the most efficient. Power users know this. Apple’s mouse-centric simple philosophy is easy to learn but hits a productivity ceiling. When the mouse is not fast enough you move on to keyboard shortcuts. When you need to be productive, you quit iMovie’s limiting simplicity and move on to Final Cut Pro with its plethora of keyboard shortcuts and alternate methods to fit various scenarios and workflows, all of which would stun a newbie.

    Supporting this is one killer feature that’s part of both Microsoft Office and the Adobe suites: The customizable UI. In Office, Photoshop, etc., if you don’t like a keyboard shortcut or a menu item, you can edit it. I’ve extensively customized my Word, Excel, and Photoshop interfaces and increased my productivity as a result. It sounds like a lot of the whiners haven’t found these features. Want to simplify the menus? Just remove the commands you don’t use. Wish a program used your preferred shortcuts? CHANGE THEM. I did. I’m not even a programmer. I’m a writer and artist.

    Office 12 and the Adobe suite also make extensive use of context menus. I love context menus. The less I have to dig through the menu bar, the better. If Mr. “One Simple Way” included having a command only in the menu bar, Office would become less usable.

    Applications need a basic approachable initial UI, but should never stop there. I hope Office 12 keeps the customizable UI and the context menus. A basic face but with a user interface that’s customizable for specialized tasks or for geeks with more time than deadlines. There is no other way that productivity can be improved.

  26. Jon says:

    Oops, in my last paragraph in my previous post, when I said “I hope Office 12 keeps the customizable UI and the context menus,” I meant to say Office 2004, not Office 12. I haven’t seen Office 12 yet!

  27. vgolf says:

    Probably I shouldn’t say ‘easy’ at all. I meant efficient, the best and most consistent way to do stuff. Providing for any easy way (e.g. point-and-click as you state) quickly makes the need to find a faster way evident. And then you have to start thinking about how best to do your work again, when you should have done that only once. Spending a little more time at the rookie level should prove more efficent in the end.

    Anyway, this is more philosophical discussion than actual feedback to MBU. I understand that they cannot risk skaring off first-time users of their products, or alienating existing ones who have been accustomed to a certain way of working.