We spent a good amount of time trying to reproduce the exact customer reported problem trying to repro the crash in the merge pages addin. Theo and I sent about 8 emails backs and forth, and three testers here donated our time to try to solve this problem. After the big clue that Theo was not seeing the crash with English settings and was only seeing it with Dutch, we set our test machines to use Dutch and still could not see the problem.
Suspecting the crash was due to some international setting, we started to change our regional settings to Spanish, Hindi, Chinese and Greek. We managed to reproduce the problem with Greek but still did not know why the crash occurred. As it turns out, it was the date format, but international characters or Unicode text could potentially have been a problem as well.
Jeff stepped through the debugger, found and fixed the problem, which was the comma used as a numerical separator.
Looking back on the bug in hindsight, there was a much simpler way to repro the crash. On any machine, change the Windows regional settings to use a comma (or any character) as a decimal separator instead of a period. Wham – crash.
One of the ongoing tasks in testing is to find the simplest way to reproduce a bug. Those simple steps make it easy for anyone to verify the bug remains fixed during further (regression) testing. For this example, I can regression test this bug on any machine now that I know there is a common Windows setting I can change to put the addin in the needed state. With the original steps, I would (perhaps) have needed to install font support or other settings to get Windows to work with the UI for the language the repro steps called for. Since I do not need to install anything, I can much more quickly regress the bug. Plus, whoever is fixing the bug does not have to install any unneeded components, saving him or her time as well.
Minimal repro steps would have saved hours of time in this case (three testers, one customer, about two hours each to isolate) and could have resulted in a faster fix. So we will apply the lesson learned and use it for our future testing.
Comments, concerns, questions and criticisms?