There are many ways we collect feedback about the new UI. I've detailed some of the ongoing usability activities in a previous article.
We have so many outlets for feedback: customer visits, instrumentation data, the beta program, this blog, usability tests, early rollouts, etc., that just organizing the feedback systematically is a huge task.
With so many high-tech feedback mechanisms, it might be surprising that one of the ways we review designs is extremely old-fashioned: our so-called "Wall of Ribbons."
Simply put, all along the hallway outside of our offices on the fourth floor, we have posted 11x17 pieces of paper with every single Ribbon in every program: Outlook, Word, Excel, PowerPoint, and Access. They are arranged in clusters of related functionality; all of the chart tabs are together for instance.
The wall is useful to us because it forms an instant gathering place to visualize and collaborate on the relationship between different related pieces of the UI. For instance, if three of us want to compare the Page Layout tabs in Word and Excel, the pictures are directly on top of each other and easy for us to refer to them together.
One of the coolest things, though, is that people constantly make suggestions just by drawing on them with whatever writing utensil they happen to have handy. We've garnered a number of good suggestions from people just writing their suggestion down on the appropriate piece of paper, often with an arrow or circle annotating the part of the UI they are commenting on. Every so often, we go through the comments and make sure we haven't missed any good ones.
Eventually, we hope to use the wall as a visual record of which Ribbon tabs are totally ready to ship and have no more changes pending. We're not to the point where we could mark any of them as "ready to ship" yet, but we'll be getting there in not so many months, either.
Even in our ultra high-tech world, there's still a place for good old pen and paper sometimes.