Improving the Cat Parade (Part 1)

The focusapp described in a previous blog entry The Cat Parade is a great way for ISVs to get additional test coverage on their applications with hopper.  I recently had the opportunity to work with an ISV that was developing a suite of apps and we made use of focusapp, but I noticed it didn’t work quite as well as I expected.  After a hopper run of 10 hours, we looked at the hopper summery and found that hopper only spent ~30% of its time on the target app.  This is pretty good, but I expected better.

After watching hopper for about a minute, I realized that hopper frequently brings up the Dialer or the Start menu after just a few seconds in the target app, and from the Start menu hopper quickly finds its way into other applications.  We tried decreased the period for focusapp, but this didn’t seem to increase the hopper coverage significantly and would have prevented hopper from exploring the deeper levels of the sub-menus in the target app.  However, we found that we could get a huge improvement in our hopper coverage if we used ActiveSync’s File Explorer to delete all of the shortcuts in the Start Menu except for the target app and the Settings menu.  We also got some improvement by changing the home screen layout to “Large Font”, which doesn’t have application shortcuts on it.  These two changes took the hopper time from 30% to 90% for the target app.  We left the Setting menu on the Start menu because we wanted to make sure hopper had the ability the change the system and hardware settings.  In our target app, we wanted to verify none of the settings affected our app, but this may not be interesting for all apps.