Inside Deep Thought (Why the UI, Part 6)


This is the sixth part in my weekly series of entries in which I outline some of the reasons we decided to pursue a new user interface for Office 12.  You can read the last installments here: Part 1  Part 2  Part 3  Part 4  Part 5.

Microsoft is tracking your every move!

Soon after you install Office 2003 on your computer, a balloon pops up asking if you would like to “Help Make Office Better.”  If you click on it, you are given the opportunity to enroll in something called the Microsoft Office Customer Experience Improvement Program.  If you opt-in, anonymous data about how you use Office are uploaded to Microsoft occasionally in the background.

If you’re the curious type, you might have wondered where your data goes.  Well, today I’m here to answer the question: it goes into an Excel spreadsheet I have sitting on my desktop.

OK, back up.  Back in the olden days of designing software at Microsoft (say, pre-2003), design decisions were mostly supported by guesswork.  There’s a classic Microsoft interview question (that I’ve never heard of anyone actually using) “How many gas stations are there in the United States?”  Many have criticized that type of question as being feckless; personally, I agree and it’s not representative of how I choose to spend my interview time with a candidate.  But the rough “estimate an answer and defend it” style required to answer the gas station question was at the heart of how many design decisions used to be made at Microsoft.

Suppose you were designing the adaptive menus in Office 2000 and you wanted to know what features people use the most.  Well, you start by asking a “guru” who has worked in the product for a long time.  “Everyone uses AutoText a lot,” the guru says.  The louder the “experts” are, the more their opinions count.  Then you move on to the anecdotal evidence: “I was home over Christmas, and I saw my mom using Normal View… that’s probably what most beginners use.”  And mix in advice from the helpful expert: “most people run multi-monitor, I heard that from the guy at Best Buy.”

So much of what we did was based on feel, estimation, and guesswork.  How much that was true only became clear with the introduction of a technology called SQM (pronounced “skwim”).

SQM, which stands for “Service Quality Monitoring” is our internal name for what became known externally as the Customer Experience Improvement Program.  It works like this: Office 2003 users have the opportunity to opt-in to the program.  From these people, we collect anonymous, non-traceable data points detailing how the software is used and and on what kind of hardware.  (Of course, no personally identifiable data is collected whatsoever.)

As designers, we define data points we’re interested in learning about and the software is instrumented to collect that data.  All of the incoming data is then aggregated together on a huge server where people like me use it to help drive decisions.


Hard at work in the SQM data center.

What kind of data do we collect?  We know everything from the frequency of which commands are used to the number of Outlook mail folders you have.  We know which keyboard shortcuts you use.  We know how much time you spend in the Calendar, and we know if you customize your toolbars.  In short, we collect anything we think might be interesting and useful as long as it doesn’t compromise a user’s privacy.

How much data have we collected?

  • About 1.3 billion sessions since we shipped Office 2003 (each session contains all the data points over a certain fixed time period.)
     
  • Over 352 million command bar clicks in Word over the last 90 days.

Reflected in these numbers is that we don’t even retain all the data points we receive… particularly, we get so much Word and Outlook data that 70% of it is thrown away.

So, one of the biggest reasons that we decided to do the new user interface for Office 12 is simply that, for the first time, we have the data we need to make intelligent decisions.  Anything we would have done in the past would have been based more on guesswork and bias than on reality.  Data is just one input to the design process, of course, but there’s something extraordinarily empowering about knowing which commands people use often and which they don’t.  And knowing which commands are used in sequence with which other commands.  And which commands are used 7x more with the keyboard than with the mouse.  And how big people’s screens are… and how much of the time they use Excel maximized… and how many documents they use at once… and which commands literally are never used… and which are used much more frequently by East Asian users… and on and on…

Knowledge is power.  And having that knowledge makes this the right time to reinvent the user interface of Office.

Want to guess what is the most-used command in Microsoft Word?  The top 5 commands used?  Post your guesses in a comment and I’ll answer in next Monday’s post.  (MVPs who saw my talk at the summit and therefore know the answers, please refrain from showing off.)

Comments (95)

  1. beekay says:

    ctrl+c/ctrl+v

  2. Tom Bradley says:

    Close?

    😉

  3. dhchait says:

    "Yes submit this crash to Microsoft and restart the app" :-)

    Just kidding. Sounds like a fascinating initiative and drilling around in all that data has yielded tons of new intelligence already. Nice job

  4. BradC says:

    Here’s what I would expect is in the top 5:

    Font

    Bold

    Italic

    Save

    Print

    I’m probably not even close… am I?

  5. Zack says:

    File-Save

    File-SaveAs

    File-Print

    Those seem the most popular on my desk. :)

  6. CharlieW says:

    File-Open

    File-Save

    File-Print

    Edit-Copy

    Edit-Paste

  7. Robert Hir says:

    view,toolbars,customize,options,uncheck menus show recently used commands first

    Seriously, the ever changing menu’s makes me mental.. I do appreciate it’s easy to disable, but the fight between visio, and office for enabling and disabling is still sorta annoying..

  8. Randy Rettinger says:

    Top 5

    Save

    Paste

    Copy

    Cut

    Spell Check

  9. dale says:

    save,

    open,

    print,

    cut,

    paste,

  10. Xavier says:

    I’m going to go out on a limb and guess that the most often used command is Undo. After all, to do five different things, I use five different commands; to undo five different things, I use the same command five times.

  11. Andrew says:

    Ctrl-Z Undo has *got* to be one of the top 5. I’m sure that bold/italic are in there too.

  12. tester says:

    #1 Undo

    #2 Paste

    #3 Copy

    #4 Cut

    #5 Save

  13. Everyone’s top five:

    Save

    Cut

    Paste

    Bold

    Underline

    My own top five:

    Ctrl+Z

    Ctrl+C

    Ctrl+X

    Ctrl+S

    Ctrl+Shift+S

  14. Michael Zuschlag says:

    Heh, so we’re playing "How many gas stations are there in the United States?" anyway.

    I’d say Undo, Cut, Copy, Paste, Save.

    Defense: These are used multiple times when editing a document. Commands like Open, Close, Exit, and Print are usually only used once per document. Bet even formatting commands like Bold and Italic beat out Open and Print.

  15. PS – I meant to justify my answers! You only OPEN a document once per visit to the document.

    But you will probably do lots of saving and formatting.

    You may only print it once or twice.

    Therefore, Open won’t be too common. Save will be down the list, but higher up than printing…

  16. Orion Adrian says:

    My thoughts are:

    #1 Undo (for the same reason back is #2 on browsers; would be typing if it was a command)

    #2 Paste

    #3 Save

    #4 Cut

    #5 Copy

    Select Text if it’s considered a command would be close to number #1 if it’s a command.

    However, if this were a trick question, then there’s also the possibility of AppMove or AppRestore or anything else to do with windows management.

    Also prime possibilities: Accept Changes and Reject Changes.

  17. Stefano Colasanti says:

    Fonts

    Copy

    Paste

    Undo

    Print

  18. Abigail says:

    I’m betting it’s Save and Paste. Well, the former might be wishful thinking.

  19. Will says:

    Save and undo would top my list. Why? Well, aren’t those the only buttons that remain on the one remaining "toolbar" in Office 12?

  20. Bert Quackman says:

    I’m betting Save is very rarely used. Most end-users I’ve known are very hostile to the idea of saving frequently. The problem is that they have to stop and grab the mouse to do almost anything, and that’s a major hassle when you’re trying to get work done. Word has had auto-save since forever, hasn’t it?

    If backspace counts as a "command", it’s probably the most commonly used.

    The punchline of this "guess the most common command" exercise, I suspect, is going to be just how far out of touch with the common user programmers tend to be. That’s an important point. It can’t be over-stressed.

    I love this idea of collecting real usage data. There’s a truism that the best software is developed by the people who use it, but that’s never going to work for mass-market stuff like Word because the average user will never be a programmer. So this is what you might call "prosthetic empathy".

  21. Wow, how very 1984 😉

  22. Andy says:

    Save rarely used by end-users? I think I press Ctrl+S unconsciously every other word I typ in Word.

  23. Step says:

    Copy, Cut, Paste, Undo, Save

    Looking forward to see how far off we all are! :)

  24. I’d say it would have to be in no particular order:

    Bold

    Center

    Bullets

    undo

    Paste (Because we always paste more than we copy inside of word)

  25. shammy says:

    File >Print

    File> Save

    File> Save As

    File> Send to> Recipient as attachment

    Exit!

  26. John C. Kirk says:

    1. Paste

    2. Print

    3. Undo

    4. Copy

    5. Cut

  27. Felix says:

    I disagree with everyone.

    My mother can’t cut and copy and paste, and she’s probably much more of a typical user than any of us. Most people will only save and print a document once in a session.

    My guesses: spellcheck, bold, underline, undo, print preview. But not necessarily in that order.

  28. Chris C. says:

    I’ve always wondered about that… specifically how much and what was collected.

    For all users I’ll say:

    Paste (crap they’ve selected from IE)

    Print (normal ppl don’t use Print Preview)

    Open (if you count clicking to open as "open")

    Undo

    and I guess… Bold, Find, or spell

    And could you make it easier to transport my prefs to a new machine? "Save my settings" was a good start… but doesn’t work job to job for us contractors (if the save were to xml and vaguely human readable that would suffice)

    Looking forward to Monday for once 😉

  29. Chris C. says:

    Hmm, my guess is showing as garbage. (which it might be… but I digress)

    I`m trying again without the single quote character:

    For all users I`ll say:

    Paste (crap they’ve selected from IE)

    Print (normal ppl don’t use Print Preview)

    Open (if you count clicking to open as open)

    Undo

    and I guess… Bold, Find, or spell

  30. Tobias Lehtipalo says:

    I think backspace/delete is the most used command in Word – provided it counts as a command of course.

    After that perhaps underline is among the top 5 and I think spellcheck might be in that list too.

    Great blog by the way. It’s fun to read about what you can do if you take usability and usability research really seriously.

  31. aeberhar says:

    Wow, we’ve made it through 30 posts and no one’s whined about privacy or misleading opt-in dialogs yet. Amazing…and refreshing.

    Now, the top five commands, in descending order:

    File Search

    Record Macro

    New Frames Page

    Document Map

    Outline View

    😉

  32. "MVPs who saw my talk at the summit and therefore know the answers, please refrain from showing off."

    By now I’ve completely forgotten. <smile>

    Nice blog. Very, very interesting reading.

  33. Asd says:

    Hmm. Probably quite popular are Open and Save As. I say that because there are many people who don’t know how to use Explorer so to put files on a floppy they open it and save it again on the disk.

  34. Fabian Dreher says:

    Toolbar->Undo

    ctrl+v

    ctrl+x

    Toolbar->Bold

    Toolbar->Save

    I really think that Undo wins the race, because Autoformat often does crazy things, but nobody actually disables it.

  35. 1 ) Open

    2 ) Indent

    3 ) Header Style

    4 ) Save

    5 ) Undo

  36. Raj K says:

    Open

    Print

    Font

    Save

    Copy

  37. herzi says:

    paste

    copy

    save

    print

    undo

  38. Brad Shannon says:

    Open

    Save

    Save As

    Spell

    Close

  39. Erin says:

    Undo

    Paste

    Copy

    Italic

    Save

  40. Toshi says:

    Close

    Exit

    Save

    Print

    Undo

  41. LeoPetr says:

    If I may be pessimistic:

    Centre

    Left

    Bold

    Italic

    Size

    Or if I should be optimistic:

    New Style

    Apply Style

    Update Style from Selection

    Save

    Hide/Show Paragraph Marks

  42. Paul Kavanagh says:

    Exit

    Font Dropdown

    Font Size Dropdown

    Save

    Font Bold

  43. Jack Mathews says:

    Word Count

    Check Spelling

    Find

    Print Toolbar Button

    Format/Font…

  44. rumorsofsurf says:

    Open

    Save

    Save As

    Print Preview

    Print

    And not sure if it counts as a command… but I imagine the recently used file list on the file menu would be quite popular, as well.

  45. Kurt Thomas says:

    Save

    Copy

    Paste

    Switch Document Windows (CTRL+F6)

    Find

  46. Matt Lacey says:

    print preview – as I do this more than printing

    save – if setting the system to automatically do this counts

    word count – I know people who use this with frightening regularity

    copy/paste – they’re really two halves of the same function to most people

    align > center – well i always do it a couple of times per document.

  47. Assuming that main-keyboard single-keystroke commands don’t count (e.g., backspace, enter, tab)

    Selection (e.g., Selection.MoveRight, Selection.MoveLeft)

    Paste

    Save

    Open

    Exit

  48. Kawigi says:

    Adding a guess, since there’s almost 50 of them:

    font

    bold

    copy

    paste

    font size

  49. weevil says:

    Given that Open, New and Print haven’t been deemed worthy of icon status in 12 they may struggle to make it to the top 5…First thing I will be doing is configuring the toolbar and bunging New and Open back to their ‘rightfull’ places.

    Let’s see: Undo, Paste, Cut, Font, Font size.

  50. I’d guess

    Buttons:

    Undo

    Save

    PrintPreview

    Print

    New Doc

    Keyb:

    Ctrl+z

    Ctrl+c

    Ctrl+v

    Ctrl+s

    Ctrl+Enter

  51. Pazu says:

    Ctrl-Z, Ctrl-B, Ctrl-U, Ctrl-V, Ctrl-X, Ctrl-S & their mouse&button & menu equivavalnts

    Do not forget, that for less-so-used commands your results are strongly biased by discoverability, the fact that there is workaround or alternative way to do it and a tradition (e.g. using tabs instead of moving left margin). The way you use your counted data you can establish strong positive feedback (in a "cybernetic" sense), so you take the commands-winners and make them more easily accesibly in Office12 – then no surprise, they will become even more popular in your next polls and on the other hand the commands-losers will get definitely worse results ever. After few cycles you burry any new, only-for-somebody features.

  52. As another guess…

    Menu options:

    Font Size (if i had to say specific, i’d say 12)

    Justify

    Bold

    Bullets

    Increase Indent

  53. 1. New, which is done automatically for you when you open Word anyways

    2. Open, which people can call without Word even being open

    3. Save, every five minutes, just in case

    4. Change font, because *everyone* loves Times New Roman!

    5. Print, because it’s hard to Xerox a laptop!

  54. sloan says:

    Undo

    Bold

    Bullet

    Spell Check

    Font Size

    I wouldn’t guess that people change fonts that often. Maybe once or twice. I think the most frequent would be other formatting because people like to "highlight" parts of their docs with titles or section headers, hence font size, bold, bullet, and maybe italic. Undo because we screw up a lot? So then maybe spell check, but so much of that is automatic by default, I wouldn’t think that would make the list… so it is a toss up between spell check and italic… and how many people just use bold over italic?

  55. Ok, now for a few non-sarcastic guesses:

    Open Doc

    New Doc

    Bold Text

    Center Text

    Set Font

  56. Assuming that we’re limiting it to just commands accessed through menus and toolbars (otherwise things like PageDown would be in the list) my guess would be as follows:

    Save

    Undo

    Paste

    Open

    Print

  57. Alexander Sadovsky says:

    Undo

    Print

    Paste

    Font size

    Spelling and Grammar

  58. ckim says:

    I’m assuming that you’re not asking about file menu commands, which is what I would expect most beginners to use, so I’m basing it on just the toolbar. So my guess as a standard user:

    – Zoom Page Width

    – View Final instead of (Final Showing Markup)

    – Font

    – Table

    – MS Office Word Help

  59. Todd says:

    I have trouble getting beyond three:

    Bold

    Italic

    Underline

    I love a lot of the features of word … style sheets are my friends, but in all the jobs I’ve had, I’m the ONLY person I have ever known who uses them. Everyone here formats using bold, italic, underline, and font size, so I should add that to my list.

    I don’t think open and print will make it. You only open a document once, but you bold things in it a lot.

    I wonder if there are statistics which show what commands are used after creation of a new document as opposed to editing of an old document. Does "delete selection" count as a command?

    OK, so maybe I’ll finalize my guess as

    Bold

    Italic

    Underline

    Font Color

    Font Size

    (Font color because many people I’ve seen don’t understand revisions, and use font color to do revision tracking.)

  60. Mark says:

    Open

    Close

    Font size

    Bold

    Undo

  61. BradC says:

    Alright, I’m revising my guess. Here was my original 5:

    Font

    Bold

    Italic

    Save

    Print

    I’m changing that to:

    Undo (#1 by far)

    Save

    Bold

    Italic

    Underline

    Hope I didn’t cheat TOO much, but I based this partially on the contents of the "floatie" in your PDC presentation.

  62. fred65 says:

    Can’t believe no-one has guessed it!! So I’ll try some others:

    Help/F1

    Select All/Ctrl+A

    Maximize

    Insert Picture

    the Window menu (many people are used to office 97)

    who knows!

  63. My guess on a top 5:

    Save

    Undo

    Print preview

    Bold

    Bullets

  64. Thomas Tallyce says:

    Getting 5 *in the right order* is a bit harsh!

    Because most users seem not use Word in a sane fashion, I’ll guess:

    Font size

    Bold

    Center

    Font [font face selection]

    Underline

  65. Bold

    Tab adjustments

    Font size

    Delete

    Paste

  66. Swati Jain says:

    I’d think:

    – Save ( for those pessimists )

    – Help ( people who have no idea what’s going on )

    – Paste

    – Copy

    – Spell Check

  67. Pia Bork says:

    My guess:

    New

    Print

    Save

    Close

    Exit

  68. Mike B says:

    Most to least:

    Exit

    Print

    Undo

    Center

    Bold

  69. danb1974 says:

    print

    save

    font size

    insert table

    bold

  70. Very interesting blog. Thanks for sharing your thoughts with us, Jensen.

    Here is my guess (saw your comment that nobody has guessed the right top five yet).

    undo

    save

    open

    font selection

    list item

    Not even close, am I?

  71. I did see your talk at the summit, so I’ll keep mum. :-)

    I’ll guess, though, that your "SQM Data Center" photo is really an IBM 360 circa 1965. IIRC, the big knob on the upper right corner of the console was a red "pull to cut power in an emergency" switch. Don’t touch that!

  72. BradC says:

    This is the problem with posting at the same time each day: people start to EXPECT you to continue posting at that same time!

    So where’s my Monday JensenH fix?? :)

    Very much looking forward to finding out the "real" top 5 most-used commands in Word.

  73. EForce says:

    Another interesting series of&amp;nbsp;articles from Jensen Harris, sharing with us the rationale why Microsoft…

  74. Today, just thinking aloud…
    A minor design conundrum we face is as follows: based
    on the data we collect,…

  75. This is the seventh part in my weekly series of entries in which I outline some of the reasons we decided…

  76. This is the eighth part in my weekly series of entries in which I outline some

    of the reasons we…

  77. This is the sixth part in my eight-part series of entries in which I outline some of the reasons we decided…

  78. This is the seventh part in my eight-part series of entries in which I outline some of the reasons we…

  79. I really enjoyed having a beer with Andy Bounds last night. Andy is an inspirational communications coach

  80. I really enjoyed having a beer with Andy Bounds last night. Andy is an inspirational communications coach