Inside Deep Thought (Why the UI, Part 6)

This is the sixth part in my eight-part series of entries in which I outline some of the reasons we decided to pursue a new user interface for Office 2007.

Microsoft is tracking your every move!

Soon after you install Office 2003 on your computer, a balloon pops up asking if you would like to “Help Make Office Better.” If you click on it, you are given the opportunity to enroll in something called the Microsoft Office Customer Experience Improvement Program. If you opt-in, anonymous data about how you use Office are uploaded to Microsoft occasionally in the background.

If you’re the curious type, you might have wondered where your data goes. Well, today I’m here to answer the question: it goes into an Excel spreadsheet I have sitting on my desktop.

OK, back up. Back in the olden days of designing software at Microsoft (say, pre-2003), design decisions were mostly supported by guesswork. There’s a classic Microsoft interview question (that I’ve never heard of anyone actually using) “How many gas stations are there in the United States?” Many have criticized that type of question as being feckless; personally, I agree and it’s not representative of how I choose to spend my interview time with a candidate. But the rough “estimate an answer and defend it” style required to answer the gas station question was at the heart of how many design decisions used to be made at Microsoft.

Suppose you were designing the adaptive menus in Office 2000 and you wanted to know what features people use the most. Well, you start by asking a “guru” who has worked in the product for a long time. “Everyone uses AutoText a lot,” the guru says. The louder the “experts” are, the more their opinions count. Then you move on to the anecdotal evidence: “I was home over Christmas, and I saw my mom using Normal View… that’s probably what most beginners use.” And mix in advice from the helpful expert: “most people run multi-monitor, I heard that from the guy at Best Buy.”

So much of what we did was based on feel, estimation, and guesswork. How much that was true only became clear with the introduction of a technology called SQM (pronounced “skwim”).

SQM, which stands for “Service Quality Monitoring” is our internal name for what because known externally as the Customer Experience Improvement Program. It works like this: Office 2003 users have the opportunity to opt-in to the program. From these people, we collect anonymous, non-traceable data points detailing how the software is used and and on what kind of hardware. (Of course, no personally identifiable data is collected whatsoever.)

As designers, we define data points we’re interested in learning about and the software is instrumented to collect that data. All of the incoming data is then aggregated together on a huge server where people like me use it to help drive decisions.

Hard at work in the SQM data center.

What kind of data do we collect? We know everything from the frequency of which commands are used to the number of Outlook mail folders you have. We know which keyboard shortcuts you use. We know how much time you spend in the Calendar, and we know if you customize your toolbars. In short, we collect anything we think might be interesting and useful as long as it doesn’t compromise a user’s privacy.

How much data have we collected?

  • About 1.3 billion sessions since we shipped Office 2003 (each session contains all the data points over a certain fixed time period.)

  • Over 352 million command bar clicks in Word over the last 90 days.

Reflected in these numbers is that we don’t even retain all the data points we receive… particularly, we get so much Word and Outlook data that 70% of it is thrown away.

So, one of the biggest reasons that we decided to do the new user interface for Office 12 is simply that, for the first time, we have the data we need to make intelligent decisions. Anything we would have done in the past would have been based more on guesswork and bias than on reality. Data is just one input to the design process, of course, but there’s something extraordinarily empowering about knowing which commands people use often and which they don’t. And knowing which commands are used in sequence with which other commands. And which commands are used 7x more with the keyboard than with the mouse. And how big people’s screens are… and how much of the time they use Excel maximized… and how many documents they use at once… and which commands literally are never used… and which are used much more frequently by East Asian users… and on and on…

Knowledge is power. And having that knowledge makes this the right time to reinvent the user interface of Office.

Want to guess what is the most-used command in Microsoft Word? The top 5 commands used? Post your guesses in a comment and I’ll answer in the next post.

(OK, this is a repost so a lot of you know the answer already. If you want to view the guesses from the first time I posed this question, view the comments with the original post here.)

Comments (44)

  1. Alrik says:

    I guess the following commands are still at the top…











  3. Ryan says:






  4. Shawn Murphy says:

    Today SQM stands for Software Quality Metrics. The term "Service Quality Monitoring" dates back to the humble beginnings of SQM when it was used to measure the client performance and usability of the MSN Client software.

    As more applications adopted SQM, the name was changed to more accurately describe what SQM did.

  5. Delordson says:






    I know this because having discovered this blog last week Tuesday, I’ve spent the last few days faithfully reading every entry only to discovered that if I’d discovered it on the Wednesday I could have limited myself to the most important bits.

    Thanks Jensen.

  6. Everson says:

    I was reading the original post and the comments and wondering about the many ways to do the same thing.

    Many readers pointed "CTRL+C", "CTRL+V" and so on as commands. Altough I expected "Copy" and "Paste" to rank high on the list, I don’t think that those particular shortcuts will do top 5 per se.

    That’s because there are different ways to do it. One can use those shortcuts or use the Edit menu or the context menu or do just what I do: I never use them, because I’m old school, I guess. I’m a "CTRL+INS"/"SHIFT+INS" person. It’s just automatic to me.

    I wonder how does the use of Copy/Paste distributes among the many ways to do it. Am I part of a minority?

    Anyway, great post (allas, great series) and I’m looking forward to see Office 12’s new UI.

  7. David Heffernan says:

    Come on Jensen, you can do better than this.  You really would be better off posting nothing.  I’m quite sure you are very busy……..

  8. Mariangeles Rodriguez says:

    >>Come on Jensen,

    >>you can do better than this.

    > You really would be better off

    >>posting nothing

    may be he ( or MS ) thought some "evangelization" was necessary at this point … to help users swallow the pill of all this Office fundamental UI Changes.

  9. Sebhelyesfarku says:

    "What is the average IQ of the population of the United States?"

  10. Herb Tyson says:


    Is it possible to tell us what % of Office 2003 users opted into the CEIP?

    I’d also be interested in knowing how representative the sample population is. The new UI is so utterly different from how I use Office, that it really makes me wonder.

    Whereas the most used command in the sampled population seems to be paste, the paste "command" I use most often is a PasteUnformatted macro. Would the CEIP have been able to detect that, or would it have reflected only that I pressed Ctrl+Shift+V, or had executed a macro of some kind?

    I’m also wondering if there is any systematic bias in the data. I.e.: are power users more likely to opt out? I know that people [like me] who are cynical and suspicious are more likely to opt out. But, what about the population from whom the CEIP has been collecting data. Is MS able to draw any inferences based on hardware or other data points collected?

  11. Jeff says:

    Herb, thanks.  Would love to know what percentage of users use a Paste-Unformatted command.

    It would be super if MS could allow users to change CTRL-V so it pasted unformatted by default.

  12. MSDNArchive says:

    I remember back in the mid 90s, one of the first web sites I designed for a company. Every menu item actually linked to a Redir.asp page, which would log the link in a database somewhere. After I had a bunch of data, I went through this data using Excel and made sure the top 10 menu items were always accessible within 1 click from anywhere on the site, and in the same place.. Then, the top 100 menu items were accessible within 2 clicks from anywhere in the site. I guess I had SQM before Microsoft, damn – should’ve got a patent..


  13. This is the seventh part in my eight-part series of entries in which I outline some of the reasons we…

  14. Ed says:

    Add me to the list who want to be able to set  paste unformatted as the default.

  15. John Elliott says:

    Should be "where the data go", not "where the data goes". :-. And the dinosaur advertisements for Office ("this data is multiplying like rabbits") irk me as well; I have to resist the urge to draw green wavy underlines on them.

  16. Greg M says:

    Keep on resisting, John.  Data is singular, not plural.

  17. This is the eighth part in my eight-part series of entries in which I outline some of the reasons we…

  18. I mentioned in mid-March that we had been putting a lot of thought and effort into improving the Office…

  19. Regular readers may remember how I have complained in the past about the way Word interferes with key…

  20. One of the most discussed aspects of the new Office 2007 UI has been: "Does it take up too much room?"…

  21. Luke says:

    Good design!

    [url=]My homepage[/url] | [url=]Cool site[/url]

  22. P-Diddy says:

    Thanks for taking the time to blog, makes us remeber that real people work at Micro$oft — not just BillyG 😉

    …and I’ve always wanted to know where my data went.

  23. gay rape says:

    We are wellocme to it’s configuration.

  24. One of the most well-understood and salient principles underlying the ergonomics of graphical user interface…

  25. While walking through Jensen’s museum of Office Past I came across several fun posts: Combating the perception

  26. The Windows Vista SDK is live and we want to get to know you better. For the first time, the Microsoft

  27. Catching up on my reading, today I finally read this Microsoft Watch article on the Office 2007 redesign:

  28. Copy Paste Unformatted Text in Word