How we use User Voice to make a better product

Ewald Hofman (MSFT)

To increase our engagement with our customers, we created the Visual Studio User Voice site to capture your feedback for Team Foundation Server (TFS) and Visual Studio Online (VSO). Since then more than 20,000 people have been participating and we have received over 2,300 suggestions. In the last year we have completed 150 suggestions. We are excited to have so many people who have taken the time to give us feedback. The goal of this post is to describe how we use the User Voice data, and what we have changed to improve our response to the User Voice feedback. If you think we can improve how we use User Voice, please leave a comment or reach out to me via email: ewald.hofman at microsoft.com.

Gaining insight

When we started with User Voice, we received a handful of suggestions per week. Many of these items were already on our backlogs and to be as transparent as possible to our supporters we marked these suggestions as “Under Review”. However we haven’t made any progress on them yet and as a result we haven’t responded to these suggestions for more than 4 years. It may not be a surprise that we created the perception of not listening to customers. We felt really bad about it, so late 2013 we started an initiative to improve how we interact with User Voice and its community.

The first step in the process was to groom the list of User Voice suggestions. We merged duplicates, updated the status of already completed suggestions and assigned the most popular suggestions to the correct feature teams. When we shared the top 20 most popular suggestions, trends and whether we were addressing them for the next major release with all the feature teams, it generated a lot of interest about User Voice. And we didn’t stop with that one time exercise, we continued to groom the User Voice suggestions to keep the list healthy, and regularly we share a summary report to all feature teams working on TFS/VSO.

Giving insight

After we gained the insight in the User Voice data, it was time for the next step. Working with User Voice was a one-way communication: we listened to the community, but did not update the status of completed suggestions nor did we tell you if we were working on something. To improve in this area we made two changes.

1. Track suggestions on our backlogs: We did not complete the User Voice suggestions, because we did not have a link between our backlog and User Voice. Therefore the feature team was just not aware that a shipped feature was backed by User Voice. To improve that we created bi-directional links between our backlogs and the User Voice for the most popular suggestions. When we ship a feature we can easily find the User Voice suggestions that we need to update. If you see an open suggestion which you think is already completed, please let me know and I will get it fixed.
2. Frequent updates on in-progress items: The “Under Review” state was overloaded because we used it for so many suggestions in the past. To better indicate which one we are actively working on or which ones are on our 6-month plan, we introduced the states “Started” and “Planned”. The goal is to give an update on in progress suggestions at least every 3 months. The majority of the Under Review suggestions are still on our backlog, but they are not part of the 6-month plan. We are going to bite the bullet and reset the state of all these Under Review suggestions, to make it very simple to you. If a suggestion has a status, you can expect us to give regular updates until we have completed the item.

We are listening…

To prioritize our backlogs we use many channels for input, and User Voice is one of these channels. And we take it very seriously.

• Below you see the list of 150 suggestions that we have completed in the last year. Based the User Voice feedback we completed items which were not in our original plans, including the “Creating Queries – Token for Current Iteration” and “ability to add multiple activities per team member in sprint”. But we heard you loud and clear and we prioritized these items higher on our backlogs.
• There are currently 49 items in progress (state is “Planned” or “Started”). Some of them have gone stale, but in the next few weeks all in progress suggestion will have an recent update. You can hold us accountable for that, so if you see an in-progress suggestion without a recent update don’t hesitate to reach out to me.
• Recently Brian Harry has published a blog post which shows some of the items that we are planning to work on in the next few months, and that list is influenced by User Voice.
• In each planning cycle the feature teams are being questioned why they are not working on one of the top 10 items

These are the suggestions that we have closed in the last year sorted on the number of votes.

Ewald Hofman (MSFT)

Follow