Product Planning for Windows...where does your feedback really go?

Ed. Note: Allow me to introduce Mike Angiulo who leads the Windows PC Ecosystem and Planning team. Mike’s team works closely with all of our hardware and software partners and leads the engineering team's product planning and research efforts for each new version of Windows. --Steven 

In Windows we have a wide variety of mechanisms to learn about our customers and marketplace which all play roles in helping us decide what we build. From the individual questions that our engineers will answer at WinHEC and PDC to the millions of records in our telemetry systems we have tools for answering almost every kind of question around what you want us to build in Windows and how well it’s all working. Listening to all of these voices together and building a coherent plan for an entire operating system release is quite a challenge – it can feel like taking a pizza order for a billion of your closest friends!

 

It should come as no surprise that in order to have a learning organization we need to have an organization that is dedicated to learning. This is led by our Product Planning team, a group of a couple dozen engineers that is both organized and sits with the program managers, developers and testers in the feature teams. They work throughout the product cycle to ensure that our vision is compelling and based on a deep understanding of our customer environment and is balanced with the business realities and competitive pressures that are in constant flux. Over the last two years we’ve had a team of dozens of professional researchers fielding surveys, listening to focus groups, and analyzing telemetry and product usage data leading up to the vision and during the development of Windows 7 – and we’re not done yet. From our independently run marketing research to reading your feedback on this blog we will continue to refine our product and the way we talk about it to customers and partners alike. That doesn’t mean that every wish goes answered! One of the hardest jobs of planning is in turning all of this data into actionable plans for development. There are three tough tradeoffs that we have been making recently.

 

First there is what I think of as the ‘taste test challenge.’ Over thirty years ago this meme was introduced in a famous war between two colas. Remember New Coke? It was the result of overemphasizing the very initial response to a product versus longer term customer satisfaction. We face this kind of challenge all the time with Windows – how do we balance the need for the product to be approachable with the need for the product to perform throughout its lifecycle? Do you want something that just boots as fast as it can or something that helps you get started? Of course we can take this to either extreme and you can say we have – we went from c:\ to Microsoft Bob in only a matter of a decade. Finding the balance between a product that is fresh and clean out of the box and continues to perform over time is a continual balance. We have ethnographers who gather research that in some cases starts even before the point of purchase and continues for months with periodic visits to learn how initial impressions morph into usage patterns over the entire lifecycle of our products.

 

Second we’re always looking out for missing the ‘trees for the forest.’ By this I mean finding the appropriate balance between aggregate and individual user data. A classic argument around PCs has always been that a limited subset of actions comprises a large percentage of the use case. The resulting argument is that a limited function device would be a simpler and more satisfying experience for a large percentage of customers! Of course this can be shown to be flawed in both the short term and the long term. Over the long term this ‘common use case’ has changed from typing & printing to consuming and burning CDs and gaming to browsing and will continue to evolve. Even in the short term we have studied the usage of thousands of machines (from users who opt-in of course) and know that while many of the common usage patterns are in fact common, that nearly every single machine we’ve ever studied had one or more unique applications in use that other machines didn’t share! This long tail phenomena is very important because if we designed for the “general case” we’d end up satisfying nobody. This tradeoff between choice and complexity is one that benefits directly from a rigorous approach to studying usage of both the collective and individual and not losing sight of either.

 

Third is all about timing. Timing is everything. We have an ongoing process for learning in a very dynamic market – one that is directly influenced by what we build. The ultimate goal is to deliver the ultimate in software & hardware experiences to customers – the right products at the right time. We’ve seen what happens if we wait too long to release software support for a new category (we should have done a better job with an earlier Bluetooth pairing standard experience) and what also happens when we ship software that the rest of the ecosystem isn’t ready for yet. This problem has the dimension of working to evangelize technologies that we know are coming, track competing standards, watch user scenarios evolve and try to coordinate our software support at the same time. To call it a moving target isn’t saying enough! It does though explain why we’re constantly taking feedback, even after any given version of Windows is done.

 

These three explicit tradeoffs always make for lively conversation – just look at the comments on this blog to date! Of course being responsive to these articulated needs is a must in a market as dynamic and challenging as ours. At the same time we have to make the biggest tradeoff of them all – balancing what you’re asking for today with what we think you’ll be asking for tomorrow. That’s the challenge of defining unarticulated needs. All technology industries face this tradeoff whether you call it the need to innovate vs. fix or subscribe to the S curve notion of discontinuities. Why would two successful auto companies, both listening to the same market input, release the first commercial Hummer and first hybrid Prius in the same year? It wasn’t that 1998 was that confusing, it was that the short term market demands and the long term market needs weren’t obviously aligned. Both forces were visible but readily dismissed – the need for increased off road capacity to negotiate the crowded suburban mall parking lots and the impending environmental implosion being predicted on college campuses throughout the world. We face balancing acts like this all the time. How do we deliver backwards compatibility and future capability one release at a time? Will the trend towards 64 bit be driven by application scenarios or by 4GB machines selling at retail?

 

We have input on key tradeoffs. We have a position on future trends. That’s usually enough to get started on the next version of the product and we stay connected with customers and partners during throughout development to keep our planning consistent with our initial direction but isn’t enough to know we’re ready to ship. Really being done has always required some post engineering feedback phase whether it’s a Community Technical Preview, Technology Adoption Program or a traditional public Beta. The origin of Beta testing and even the current definition of the term aren’t clear. Some products now seem to be in Beta forever! We work to find the best possible timing for sharing the product and gathering final feedback. If we release it too early it’s usually not in any shape to evaluate, especially with respect to performance, security, compatibility and other critical fundamentals. If we release too late we can’t actually take any of the feedback you give us, and I can’t think of a worse recipe for customer satisfaction than to ask for feedback which gets systematically ignored. I was just looking at another software “feedback” site where a bunch of the comments just asked the company to “please read this site!” For Windows 7 we’re going to deliver a Beta that is good enough to experience and leaves us enough time to address areas where we need more refinement. This blog will be an important part of the process because it will provide enough explanation and content and guidance to help you understand the remaining degrees of freedom, some of the core assumptions that went into each area and will structure our dialogue so that we can listen and respond to as much feedback as you’re willing to give. Some of this will result in bugs that get fixed, some will result in bugs in drivers or applications that we help our partners fix. And of course sometimes we’ll just end up with healthy debate – but even in this case we will be talking, we will respond to constructive comments, bugs and ideas and we will both be starting that conversation with more context than ever. So please do keep your comments coming. Please participate in the Customer Experience Improvement program. Give us feedback at WinHEC and PDC and in the newgroups and forums – we’re listening!  

 

Thanks,

- Mike