A problem has occurred, would you like to send this data to Microsoft?


I’m watching the current Tech-Ed session on the new additions to the debugger to help simplify understanding what’s going on when debugging complicated applications/structures.


Everything was going great, and then BOOM VS fell flat on it’s face. Not a very uncommon event at Tech-Ed since we are using bleeding edge alpha bits to demo the latest stuff we are doing. Scott recovered great, with a lot of humor, however he dismissed the dialog saying “would you like to send this report to MS”. Habib, noticed this and mentioned “We’re not connected to the network right now, however, if this happens to you please send us this information.”


I can’t express how right he is. No matter how much time and effort we spend in testing and QA on our products, bugs still happen and things will not always behave properly. We take all of these problems very seriously and consider things like crashes, hangs, and data corruption/loss to be critical problems that must be addressed ASAP. However, if the crash only happens on your machine then we might never know about it.


The system that detects crashes and uploads it to MS is called Watson and you can read about it here. These crashes do get to us devs and we do analyse them and fix the problems. (It’s really sucky to do since it’s usually working on extremely optimized code, and i hate X86 assembly, but it usually ends up revealing some very hard to find bug that only occurs under a very specific set of circumstances).


I’ve talked to many people who tell me they don’t send the information because they don’t think it does any good or gets looked at by anybody. I just want to let you know that that most definitly is not the case. If there are any reasons you don’t use the crash reporting system, please let us know why!



Update: John pointed me to this page: https://winqual.microsoft.com/default.aspx where ISVs can sign up and get that crash report data.  Interesting stuff.  Thanks John


Comments (32)

  1. Jerry Pisk says:

    Personally I do think that nobody looks at the data, or at least nobody really cares to do anything about it. I’ve been uploading several blue screen reports caused by Creative’s Audigy card drivers and yet they’re still certified by Microsoft as good. Yet they have major thread synchronization issues and won’t work for more than couple of hours on an SMP box. So why bother sending the data to Microsoft…

  2. It’d be nice if the vendors of the products could specify that bug reports for this application should be sent to them. Most of the time I dismiss the dialogs because I consider it to be a problem with the application or hardware, and not Windows XP.

  3. kevin white says:

    I have a laptop with crash reporting enabled but sometimes that machine is not connected to a network. It would be really smart if the reporting tool was able to store and batch the transmissions. It would also be really nice to have the tool give some basic reporting feedback to the user. One of the reasons many people have the perception that the reporting is useless is because they don’t see anything tangile back. A basic reporting interface might solve that.

  4. I personally don’t do it becouse most of craches are doing with the cold passion to kill this software 😉

    And going serious, I just don’t trust you so much to send you my dump after Outlook crash, ok ? If drivers, IE, or Explorer crache I’m always at your door with raport 😉

  5. Noah Coad says:

    So I always click ‘send’ and it gets annoying to do so every time the dialog appears. Especially as a dev & trainer, I get this dialog frequently and really wish it would simply send each time without adding yet another dialog. Many dialogs have a "an always apply this setting" and this is much needed. People who do click yes, usually do for all products. For simplicity, there shouldn’t even be a dialog, but a little pop up or icon in the system tray like updates, new hardware, etc. Please pass this on!

  6. Jerry: one thing that I should mention (and SteveB brought it up at the Tech-Ed Keynote http://www.microsoft.com/presspass/exec/steve/2004/05-24teched04.asp), is that Watson allows us a statistical view of your issues. And it allows us to working on fixing the issues that are affecting the most customers. I.e. if we have a crash affecting millions of users we will probably address that one before one that we have affecting a few users, or only affecting one user many times.

    By not sending us the report we end up not knowing that it’s affecting you.

    I’ll get back to this in a later post.

  7. Brian: You raise a very good point. I don’t know how MS deals with crashes in non-MS products. I also don’t know if Watson is available for use with other software vendors. I’ll try to get this information for you.

  8. Kevin: I believe that one of the features of watson is that it can queue up dumps to be uploaded at a time when the network becomes available. However, i believe (but could be very wrong) that apps have to inform watson that that is ok). I’m working with Anson (our PM) to make it so that VS 2005 will have this behavior and I will try to get more information on this.

  9. Stefan: I understand your concern. Your private data is just that, private. The only thing I can say is that you should read our privacy policy to see if that alleviates your worries.

  10. Noah: Two things.

    a) I will send that feedback to team responsible for Watson. People are very often annoyed when they are prompted with dialogs incessantly when they always will respond to it in the same way. However, there are many issues that come along with this. Watson is not something that always has a simple dialog. It’s possible that depending on the issue we might send more data than you might be comfortable with. See the link i posted above for examples of that. Because private information is so sensitive we probably felt that the safer choice was to make sure it was ok with the user every time.

    b) One thing I’ve read a lot about is how annoyed people are with pop-ups. They tend to dismiss them immediately without reading them. There’s a good blog on this that i can’t seem to find. This is a big problem, especially with the critical updates warnings from WindowsUpdate. People automatically dismiss it because they don’t want to be distracted from what they’re currently doing and then bad things happen because their system is in trouble. People have gone as far as to think that pop-ups of that sort are the result of a virus infection!

  11. kevin white says:

    Cyrus, there are several ways to address:

    "It’s possible that depending on the issue we might send more data than you might be comfortable with."

    One way is to setup a configuration that allows people to set the default level of information sent. This configuration would also have to explain that sometimes a more detailed report is required and give options on how that situation could be handled. The person could choose to allow the automated sending of additional details or could choose to only send the normal level.

    There may even be an option to display a notification only when the higher level of detail is required. Application-specific restrictions might also be feasible. One person mentioned refusing to send Outlook crashes. There may be other people who do not want crash information sent for specific applications. A default level of detail combined with a dialog that is displayed only on the first time each application crashes could offer a great deal of customization with minimal interruption.

    You would be shocked how a good interaction design could actually increase both the volume and quality of the information you recieve. Right now the crash reports are an immediate positive to the developers but an immediate negative to the users. The positive to the users, less crash prone software, is too far away in time to be percieved as valuable. A good configuration experience that builds trust while explaining the benefits will do wonders.

  12. Kevin: Fantastic information. I’ll forward appropriately 🙂

    I think one of the issues that went into the original design was that we wanted to walk the fine line between interaction and bothering you. The more questions, the more check boxes, the more wizards, the more text, and you increase the risk that a user will say "the hell with this, i’m just canceling".

    Many users might not ‘get’ that this is a per application setting. So they would be bothered when Outlook crashed, they made all the settings, then IE crashed: "hunh? i have to do this all again?? i thought I already told it what i wanted." You could solve this with global options, but then that comes with the issue of even more complexity presented to the user.

    I do think it can be better, I’ll just leave it to the people that are good at user interaction (i.e. not me) to figure out how 🙂

    BTW, is this something that you work on (interaction I mean)? I’d love to talk more with you on this and other similar topics.

  13. Noah Coad says:

    Cyrus: What I meant by popup (which I agree is frequently ignored) was simply an indication that a bug report was being sent to Microsoft after the user has already selected an option to always send reports. The simple fact that delivery was taking place is the issue, the vehicle is trivial. What I’m looking for is an option as with Windows Updates to always download updates meets WinXP SP2 where updates are automatically applied as well … the option to avoid human interaction.

  14. We want to fix your crashes.

  15. John says:

    Brian and Cyrus: I work for an ISV, not Microsoft, and I have arranged to get access to my company’s crash reports through http://winqual.microsoft.com . So do, by all means, send crash reports for applications.

  16. John,

    Well that answers that question! Thanks, I will definitely begin sending them in again.

    Brian

  17. William Gault says:

    I’m curious how Microsoft devs can sort out what crashs are caused by bad hardware setups (ie. bad RAM, bad bios setup, etc), a poorly programmed piece of software, or a problem with the OS. Last week I was testing software that crashed my machine, submitted the dump and it stated that it was a driver issue, but I know is was this software’s bug…

  18. William: Great question. I can even think of a simpler case:

    What does a dev do when there is a problem due to memory corruption? Any random thread in the program could have corrupted the memory and we’re only crashing when our code hits the corruption.

    This is where the statistical properties of watson come into play. Watson collects dumps and automatically processes several properties automatically (but I’m not sure what I can talk about).

    So if we have one crash on one machine in the entire world, we might not fix the issue because we’re not necessarily what casued the crash. However, if we have 5, or 10, (or more if the bug is particularly bad) we have all sorts of techniques of diagnosing, repro-ing and resolving the issue.

  19. John says:

    If any of you Microsofties knows how to get in touch with Jason Hardester, could you remind him that his slide deck for WINC08 – Corporate Error Reporting still hasn’t shown up on CommNet? Thanks…

  20. John:

    Here was Jason’s response:

    I have pinged the track owner (*******) to find when we can expect to see the PPT out on CommNet. …Seems that the person who owns the Cabana sessions postings to CommNet and the Tech-Ed DVDs have been OOF sick and thus… the delay.

    ******** just informed me that we should expect to see the sessions posted by Tuesday COB at the latest.

  21. John says:

    Get well soon, ********.

  22. John says:

    I hate to be a pest, but the slides are still not there. http://www.msteched.com/TechEdSlides.aspx

  23. John. Not a problem. I’ll look into it 🙂

  24. Sean Duggan says:

    Honestly, while I’ll often send in the crash information, if I’m in the middle of doing something, I’ll often decide not to, particularly if this program has a habit of crashing, if I’m in the middle of something because it tends to slow the computer and almost inevitably, I just get a message saying that Microsoft suspects that the program caused the crash. Although I did appreciate the occasional candid message that said that the fault was caused by Microsoft Windows and that I should contact the manufacturer for updated software.

  25. When a Windows program crashes, Windows XP gives you the opportunity to send an error report to Microsoft. The process is called Online Crash Analysis. My advice: Do it. Here’s a perfect example of why it’s good for you and for your fellow PC users. For years, I’ve encountered a sporadic…