There’s a fairly common industry term that gets used at MS a lot. It’s called Eat your own dog food (or dog fooding for short). I’ve heard that the term arose when a dog food producer was trying to convince stores to sell its product. Iin order to so the salesperson opened up the bag and started crunching, thus showing that there dogfood was high enough quality.
For us this means working on the absolute bleeding edge of VS with the absolute bleeding edge of VS. Developers commonly pull down a new build when the version they use gets about a week out of date (on average). This can be painful at times; performance might have regressed dramatically, or stability might be compromised through a little bug in a common code path. It also means that we use (or are encouraged to use) the betas of products like Office and Windows in order to help hit more of the functionality in those products. This sort of real world use stresses the application in ways that are sometimes hard to automate.
For the rest of the company this means that every so often we take a snapshot of our current state and branch it into a build that we will let them use. We’ll then do a lot of QA work on that to make sure that that build doesn’t having any glaring problems (hangs, crashes, data corruption) that would prevent people from being able to do the work they need to do. These aren’t people using VS for the fun of it, they’re other programmers working on MS products and they need to be productive or else it’s back to notepad.
That’s a failure for us. Not only have we lost the direct feedback we desperately need as to how useful features are, but we’ve also possibly just lost the user period. They’ll start thinking “VS is slow, it’s buggy, the programmers don’t care about the users.” And when they have the choice of what to use in the future they might say “screw it, i don’t want the hassle, i’m using something else.”
This is one of the reasons why we ship things like the VS Community Preview and beta1. We use them 100% to get feedback from users as to what’s the good, the bad, and the ugly. However, the loop between us and the user is too long. (This is also one of the reasons we’re seeing blogging as being incredibly useful). There’s something about getting an email from within the company that will light a fire underneath you. You try to work with that person (usually by remoting into their machine and debugging) so that you can get a fix that will hopefully make it into the next build. I wish that was possible to do with all users, and I would love to get ideas as to how anyone thinks that might be possible. Note: this is a major reason why we have Watson. While not as good as direct debugging into a user machine, Watson’s crash dumps allow us to know that you are having problems and what they are with the only thing you having to do is click a button saying “send crash report”.