Product Feedback Center after VS2005 ships
In a suggestion I responded to (FDBK17182: “Hire more people, so you can fix more of customers’ suggestions”), Phylum asked me to clarify a comment I made about fixing bugs and suggestions in a future release – he asked whether I met the next official version or in a service pack.
In most cases I mean in the next official version, but it does beg the question about how Product Feedback Center (PFC) will affect our whole servicing strategy for released products. The internal focus up until now within the Developer Division has been just making it work for VS2005 – getting the teams used to handling large volumes of customer feedback, understanding what “good job/quality responses” means beyond simple turnaround times (we have more to do here – e.g. we sometimes see great, lengthy repro scenarios from customers that get a terse “no repro” back from us), figuring out which few suggestions we can address given that we’re late in the ship cycle, etc.. However, we’re also starting to think about the transition from pre-RTM to post-RTM for feedback from PFC.
Here’s what we don’t want to happen: We get lots of great feedback prior to shipping, and address as much of it as we can. On the day we ship there’s a set of suggestions and probably even low severity bugs that we’ve postponed. And then we shut down feedback on VS2005 and/or never bring up the postponed feedback again and/or stop responding on VS2005 feedback that comes in after RTM.
What we want is something more like:
- Postponed suggestions and bugs for VS2005 are the FIRST thing that gets considered for the next release.
- Customers can continue to submit feedback on VS2005 even after we ship. If you have the shipped product and you find a bug, let us know about it via PFC. Customers shouldn’t have to use some separate feedback tool just because we ship.
- We continue to acknowledge feedback as it comes in, even if we can’t provide a clear resolution/fix quickly. Then customers at least know it’s on the list.
- We use customer feedback to do a better job with servicing on existing products. This is probably the most interesting part. One of the challenges we have today is that we have no good way of knowing whether there are common bugs in shipped products that customers frequently hit. I’m sure there are a few counterexamples, but this is true in general – what we know is more anecdotal than scientific. With PFC, we potentially have the ability to get the list of bugs people most want to see serviced, and to have a goal that any service packs fix a high % of the top bugs.
- I focused that last paragraph on bugs. I think Phylum’s original question was about suggestions. It’s unlikely we’ll fix a significant number of suggestions via a service pack. I wouldn’t say we’ll never do it, but I know for sure we won’t do a lot of it because the test costs on service packs are fairly high and still not 100% coverage vs. RTM, and therefore we try to reduce all unnecessary churn. However, PFC again creates the possibility that we can set a small goal around the suggestions, e.g. look at the top 20 suggestions and see which are possible to address through low-cost/low-risk implementations.
- Even if we don’t release official fixes through service packs, PFC gives us much better way to consider alternative solutions, e.g. releasing a sample, whitepapers, etc. that get customers part of what they ask for.
I said this in my response to the bug report, but we rolled out PFC late in the VS2005 dev cycle not by design (as in “let’s not ask for feedback until it’s too late to act on many of the suggestions”) but because PFC wasn’t ready until then. PFC is a long-term commitment – we’ll turn it on earlier in future cycles, other products will start using it, etc.. And we want to make sure that we’re benefiting from all current and historical feedback at all points of our dev cycles – late in the cycle when we’re trying to make sure the quality is right, early in the planning phase when we’re figuring out what to do next, mid-way into the cycle when we’re trying to make sure the features/scenarios work well, and in-between cycles when servicing is an issue.
Let us know your thoughts here.
Mark Cliggett
Group Program Manager - DevDiv Customer Connection Team