Reproducing Hopper runs … and why it doesn’t work!

I am asked frequently if it is possible to reproduce Hopper runs – and the answer is a resounding “sort of”. The desire for this feature is logical: you have a recurring exception that happens after approximately 10 hours of Hopper and you want to set a breakpoint to catch the problem before it happens. It makes sense, but unfortunately Hopper does not work this way and reproducing all but the shortest Hopper runs is almost impossible.

Let me explain what *is* possible: Strictly speaking, you can reproduce any particular Hopper run by using its random seed as a parameter (-sxxx) when you invoke the test. Hopper derives its randomness by calling rand() which is completely predictable if you to know its seed. All Hopper actions are bound by this rand() functionality so keystrokes (and screen taps) sent to the device from one seed run will reproduce 100% to a similarly seeded device.

You also have the ability to use the Verbose (-v) flag which will print each key being sent to the device in your debug output. I have heard stories of saving this output to a file and then writing a simple input program that can ‘replay’ this file. But again, if you have the previous log, this isn’t really necessary since you can always reply the events with Hopper using the previous seed.

Next, why it doesn’t work: In general replaying Hopper runs longer than a few minutes will NOT reproduce the same run and will give you different results. This is frustrating because we want Hopper to work this way and it certainly seems like it should, but it doesn’t.

Hopper sends its keystrokes and screen taps directly to GWES as fast as the system will allow and often the UI will be trying to catch up. GWES is busy trying to process the keys and the UI is busy trying to keep up. As widows are being created and destroyed, the timing of each input is critical and easily missed - a keystroke intended for one window is actually sent to another. Once this happens just once - your run has been altered and you are no longer on the same path.

Lastly, what you can do about it: There are several ways you can deal with this issue, but my best advice is: don’t. The best strategy is to evaluate breaks, hangs and crashes as they stand and not necessarily worry about how they happened and focus on the fact that they happened. Microsoft has a dedicated team solving stability bugs found by Hopper and we don’t use it. Instead we focus on other strategies and clues left by the system (in fact that is the focus of this blog).

Challenge yourself to understand the need of the repro case – is it absolutely required to solve this problem? Are you absolutely certain that getting a repro case will shed light on the problem? It is often easy to think that a repro case will help, but my guess is that it will simply take a lot of resources to get there and the additional information you have gathered won’t be worth the effort.

But, if you must pursue the repro case, it might be possible to repro using the same seed (-sxx). Hopper has another feature that puts it under lazy mode and allows you to slow down (-lxx) inputs so that timing problems are less likely to occur. Unfortunately this often has the side-effect of not finding the original bug and can lengthen the Hopper runs considerably.