The Hopper Cookbook (Step-by-step instructions to improve application stability)

  • Article

The intent of this document is to introduce Hopper to the application developer who is targeting applications for the Windows Mobile platform. The purpose of running Hopper against your application is to find exceptions, hangs and deadlocks that may be lurking in your application that may go unnoticed during testing and may cause device crashes during customer acceptance or field trials. This introduction will include all the necessary materials to begin Stability testing of your application using Hopper as well as pointers for advanced techniques.

 

Goals

The goal of this document is to educate ISV development community on Hopper so they can improve application quality and stability. Developing with Hopper, ISV’s will reduce customer acceptance time as well as find more application bugs that will contribute to the overall quality of their application. The expected steps from the ISV community are:

 

  1. Consistently run Hopper during the development cycle and sort the bugs into three categories:
    1. Category #1: ISV application dependent bugs (to be fixed by the ISV)
    2. Category #2: OEM device dependent bugs (to be reported to the respective OEM for fixes, if applicable)
    3. Category #3: WM dependent bugs (to be reported to Microsoft for evaluation and possible fixes)
  2. Coordinate with both the respective OEM and Microsoft (if applicable) to secure bug resolutions and proper application stability
  3. Run Hopper during WM device development by coordinating with their respective OEMs (if applicable)

 

The ISV development community following the above process will enjoy improved application quality and stability. Hopper takes the guesswork out of quality measurement, thus reducing application time to market.

 

Prerequisites for this document

This document builds upon existing materials published for the OEM and the ISV development community. It is highly recommended that the reader review the following topics before reading this document:

 

  1. Hopper for ISV’s – part I (https://blogs.msdn.com/hopperx/archive/2006/06/05/618018.aspx)
  2. The Cat Parade (https://blogs.msdn.com/hopperx/archive/2005/11/30/498113.aspx)
  3. Improving the Cat Parade (https://blogs.msdn.com/hopperx/archive/2006/11/21/improving-the-cat-parade-part-1.aspx)

 

The Hopper Doctor blog, which contains the above materials is updated frequently and often contains ISV-specific information, so it is recommended that ISVs check the main blog page (https://blogs.msdn.com/hoppeRx/) frequently. This document also assumes the reader has Visual Studio 2005 and the Windows Mobile 6 SDK installed.

 

Review – and overview of the Hopper system

As mentioned in the Hopper for ISV’s blog, Hopper is a stability test tool that randomly sends keystrokes and screen taps to the device. Hopper is designed to stress the entire device and not remain in a single application. However, application developers are not interested in stability of the entire device– they want to locate and fix bugs only in THEIR application. To achieve this, ISVs need a way to focus Hopper on their application and to minimize the time Hopper spends away from their application.

 

 

To successfully perform stability testing with hopper, three elements are needed:

 

 

The Hopper component is available under your SDK tools directory. A sample focusApp.exe is provided in the same Hopper tools directory and will need to be customized for each application you are testing. To begin testing, you need to perform the following steps:

 

  1. Using Visual Studio, deploy your application to a device or the device emulator and attach the debugger. This will enable you to debug any exceptions or hangs that your application encounters. Step-by-step instructions on how to do this are located below in the section labeled Demo.

 

  1. Customize the focusApp.cpp source to keep your application in the foreground. Refer to “The Cat Parade” blog entry for more information as well as the end of this document for more ideas. Once customized, this tool needs to be deployed to device and your application

 

  1. Deploy and run Hopper while your application is running. The focusApp will force Hopper to remain “focused” on your application and avoid the rest of the system.

 

 

By combining these three elements, we are able to remove platform stability from the equation and allow Hopper’s effort to be focused on your application and away from other potential problems. For example, if you are running this test on a device that has known stability issues, it is unlikely that Hopper will encounter those issue because your application is processing all the key strokes. This is very convenient since it is the application bugs that you are interested in, not the bugs elsewhere in the system.

 

Demo - Watch Hopper in action and find real bugs!

Below represents a step-by-step procedure for running Hopper on a sample application that is included in the SDK. Ultimately you will want to proceed through the following steps using the application you are developing and not CECamera.

 

Step 1: Launch Visual Studio. Launch the SDK tools folder from the start menu and navigate into the Hopper Folder and then again into focusApp folder. Double click on this solution to bring up VS.

 

 

 

 

Step 2: Customize focusApp. We need to customize focusApp to point to the application under test. For this sample, open the file FocusApp.cpp and replace the “\\Windows\\WMPlayer.exe” with “\\Program Files\\CECamera\\CECamera.exe”. Rebuild the solution. When testing your own application, you will replace the CECamera path and filename with the path and filename of your application

 

 

Once this application has been rebuilt – you can close this solution.

 

 

Step 3: Launch the CECamera solution provided under the Samples folder – this is the application we will torture under hopper.

 

 

Navigate to the CECamera directory under the following directory: Samples\Common\CPP\Win32\CECamera – double click on this CECamera solution to re-launch Visual Studio with this solution active.

 

 

Step 4: Add Hopper dependencies to your deployment.

Change the Project properties to automatically deploy your focsuApp.exe and Hopper.exe. Under the menus Project->Properties, select Configuration Properties -> Deployment and add the following lines to the existing:

 

focusApp.exe|C:\Program Files\Windows Mobile 6 SDK\tools\hopper\focusApp\FocusApp\$(OutDir)||\|0

hopper.exe|C:\Program Files\Windows Mobile 6 SDK\tools\hopper||\|0

 

Please note that your Path locations may be different from the example, and care must be taken to copy & paste the correct paths. When this is complete, your dialog should look similar to:

 

 

Test your deployment. Using the menu action Build -> Deploy solution, check the results of your deployment, you should see:

 

 

1>------ Deploy started: Project: CECamera, Configuration: Debug Windows Mobile 6 Professional SDK (ARMV4I) ------

1>C:\Program Files\Microsoft Visual Studio 8\VC\ce\dll\ARMV4I\msvcr80.dll

1>C:\Program Files\Microsoft Visual Studio 8\VC\ce\dll\ARMV4I\msvcr80d.dll

1>C:\Program Files\Windows Mobile 6 SDK\tools\hopper\focusApp\FocusApp\Windows Mobile 6 Professional SDK (ARMV4I)\Debug\focusApp.exe

1>C:\Program Files\Windows Mobile 6 SDK\tools\hopper\hopper.exe

1>c:\Program Files\Windows Mobile 6 SDK\Samples\Common\CPP\Win32\CECamera\Windows Mobile 6 Professional SDK (ARMV4I)\Debug\CECamera.exe

========== Build: 0 succeeded, 0 failed, 1 up-to-date, 0 skipped ==========

========== Deploy: 1 succeeded, 0 failed, 0 skipped ==========

 

You will note that the Device Emulator will launch in the background and fully boot before copies can begin. If there are any errors during deployment, please check your pathnames above to make sure you are pointing to the correct locations. You must be able to copy & paste the full paths and reference the binaries. Please correct any mistakes before continuing.

 

 

Step 5: Close off exit points.

If your application responds to WM_CLOSE to exit, we need to ignore this command so Hopper does not close the application. CECamera sample contains both a menu item and an “X” box that will sent a WM_CLOSE to the app – so we need to comment out this code. Locate the CECamera.cpp file and search for WM_CLOSE like below:

 

 

…

                case IDCANCEL:

                    // Fall through

                case IDM_EXIT:

                    // PostMessage(hwndDlg, WM_CLOSE, 0, 0); ** Comment out this line **

                    break;

  …

 

 

 

Step 6: Start debugging the CECamera.exe application.

 

Before we begin debugging we need to verify the debugger will detect and break when an exception is hit. Menu Debug -> Exceptions will bring up the following dialog – make sure all thrown check boxes are checked and your settings saved.

 

 

 

 

Next, Select the menu Debug -> Start Debugging to deploy and debug CECamera. This will launch the application under test and if you switch back to Device Emulator, you will be presented with the sample application.

 

 

Step 7: Launch the Hopper and focusApp.

 

 

 

In this step, we need to access the File Explorer so we can launch focusApp and then Hopper.

 

Click on the Start Menu, select Programs and then File Explorer.

 

 

 

Navigate UP to the root directory and select focusApp.exe – this will launch focusApp.exe which will quickly bring CECamera back to the foreground.

 

 

 

 

Bring the File Explorer back by clicking the Start Menu -> File Explorer. Use this opportunity to launch Hopper.exe.

 

If you wait too long, focusApp will bring the CECamera application back to the foreground and obscure File Explorer. Simply Bring the File Explorer back by clicking the Start Menu -> File Explorer and click on Hopper.exe. This tight-timing is expected and part of the design.

 

In a moment, Hopper will take over your system and begin pressing keys and tapping on the screen. Once Hopper encounters an exception in your application, Visual Studio will break and you will be notified.

 

 

 

How to debug breaks

Having Hopper focus on your application is only useful if it finds exceptions, hangs or deadlocks. Hopper’s job is to identify these bugs and it is left to the developer to actually find and fix the root cause for any break The Hopper Doctor Blog (https://blogs.msdn.com/hoppeRx/) is a great resource for how to debugging exceptions, hangs and deadlocks found by hopper.

 

Since Visual Studio is an application debugger you will notice that Hopper will continue stressing the device after your application has broken into the debugger. You will also note that focusApp.exe continues to run and will continue trying to bring your application to the foreground.

 

This is normal and should not interfere with debugging the break that stopped Visual Studio.