Some of you may be testing with Hopper now for certification and/or being asked to meet Mean Time To Failure (MTTF) bars as part of a relationship with a device maker. In order to be labeled a Windows Mobile powered device, the unit must pass a series of tests which are collectively known as the LTK (Logo Test Kit). If you ship your software in-ROM then your app becomes part of the stability tests they have to pass before release. There is nothing worse then to be sitting on a huge opportunity to ship in-ROM and find yourself as the focus of a failed Hopper test and holding up the device release.
Hopper isn’t very smart, but he’s surprisingly effective at exposing MTTF issues. Lately, I’ve had several partner become the subject of "Start Menu Dead" failures so I thought it might make a useful post.
For those of you unfamiliar with Hopper, it’s a simple little app that basically simulates a 5 year old, jacked up on sugar that has taken over your mobile device. Tapping like crazy– switching apps– doing all kinds of things that a normal user might never do. Maybe that’s not a completely fair description– but think of a lot of random events that are periodically paused to make sure basic system functionality is still responsive– like the START MENU.
Hopper runs for a period of time and reports a number of failure scenarios. Examples:
- Default system crash; this is where Hopper is not able to send keystrokes and mouse events.
- Start menu not responding. this is where Hopper is able to send keystrokes and mouse events but is not able to switch to any other application.
- Stuck in the same window, this is where Hopper is able to send keystrokes and mouse events but it detects that it is stuck in the same window for more than 15 minutes. Once this happens Hopper will attempt to dismiss the current foreground window by sending ENTER keys, sending simulated screen taps around the 4 corners and the center of the window, sending ESCAPE keys, try to find a button to click, and finally it will also try to switch away by clicking the Start menu. If none of the above attempts change the foreground window, it will quit and report a hang.
Applications that run in full screen (using SHFullScreen) can sometimes create problems for Hopper. Since Hopper likes to jump around, switch apps, and test the Start Menu– it needs to be able to get out of an application running in full screen mode. Many full screen apps like games, navigation software, etc., will use a custom UI to take over the display and omit standard exit controls and window messages that Hopper uses. Once Hopper gets into a full screen app like this, it can get stuck and then report failures when it can’t get out or access the Start Menu. When it does, your failure log will often look something like this:
Build = 15342 (OS 318)(Hop 22.214.171.12472).
Random Seed = 55592.
Previous runtime = 21 mins (0 hrs 21 mins).
Ended by: Start menu dead, not responding!
Boot count (prev): 3 (0)
ACTIONS/min = 149
Total States = 10
ACTIONS/min = 149; Total States = 10
FREE MEMORY: 75620 KB
FREE DISK: 25894 KB
EndType = Start menu dead, not responding
The important stuff
If you have an application that runs in fulls screen mode, be sure that Hopper can get out and access the Start Menu. If it sends a VK_THOME, it should always take you to the Home Screen. If it sends a VK_LWIN then it should always bring up the Start Menu. If you do anything to hold focus or prevent those messages from activating another window, then you will have problems. If Hopper cannot get out, it will try to dismiss your application by issuing a VK_ESCAPE and VK_RETURN on PPC (Classic/Pro). On Standard (Smartphone) it will attempt to dismiss by trying variations of the softkeys and sending a VK_TBACK. Make sure you handle those messages to close your app or release screen control in order to avoid the Start Menu Dead failure in Hopper.
Be sure to check out the Hopper blog for more info on using this tool.