Next up: testing the addin

  • Article

Continuing with the "Replace Outlook Notes with OneNote" addin:

The first task a tester has to face when testing a few feature is deciding if the specification makes sense, is complete, solves the problem faced by the end user and is testable.  

Here's the mini-spec I created for the Outlook addin to "replace" the Outlook sticky notes module with OneNote.

Scenario: A user (let's call her Tina) has Outlook installed and has been using the sticky notes module for awhile. She is aware of the limitations of that module and has decided to try OneNote. Tina loves OneNote and has decided to migrate her notes from Outlook to OneNote. She also wants to use OneNote instead of Outlook's note modules in the future. We will add a lightweight feature to Outlook to open OneNote instead of an Outlook note module.

Design:

  1. Tina clicks the Notes module in the Navigation bar, or uses a shortcut key combination to open the notes module.

  2. OneNote will open and/or take the topmost level of focus.

  3. Outlook will continue to navigate and display the notes folder in the background.

    So the first task I have on my plate if given this new feature to test is to decide if it fits Tina's needs. I want to focus on step 2 and look at it from a test point of view   

This design step seems relatively easy to describe and to test at first glance. After thinking about this for a few minutes, I came up with this list of questions and comments about the design:

  1. I'll get the feature designer to specify whether a new OneNote instance is started or we reuse the existing window.

  2. I'll add cases to test for OneNote not able to boot

    Examples of why ON couldn't boot
    Low memory
    It got uninstalled (and should the addin detect this and disable itself?)
    Locked out by an administrative setting

  3. Do we pop up an alert if we can't start OneNote, or silently fail?

  4. As a tester, do I need to request a logging capability for this feature?

  5. There are multiple cases here in which other dialogs from other applications might get created during the time needed to start OneNote and want to have focus. My task is to get the designer of the feature to define the expected behavior - should ON assume focus?

  6. I'll want to automate this test with a test script. Dealing with focus and verifying OneNote has the top most level of focus may be tricky. I'll need to research how to do this (with the .NET framework. Most of our automation uses it).

  7. Does Outlook fall to the bottom of the focus hierarchy, or does it get put one level below OneNote?

  8. Is there any way to turn this feature off?

  9. How long should it be before OneNote starts and shows? What is the performance goal?   

It looks like this step wasn't so well defined after all. I have a list of questions about the design of the feature, some questions about whether I need to request some features that make this more testable (logging), a laundry list of research to do and I'll have to follow up with the designer to get some of these questions answered.