They’re (gasp) Everywhere!


Recently I was shopping for music at a well-known online store. This store often suggests other albums related to the one you are browsing and offers you a package deal where if you purchase both the CD you are browsing and a related CD you receive a discount over purchasing both CDs individually.

At least, I always assumed I received a discount. The other day, however, the price for the package deal didn’t seem all that different from the price for purchasing each of the CDs individually. I did the math – it was exactly the same price!

I wonder how many other times I’ve taken the package deal without paying attention the price. I wonder how many other people have done so. I wonder if the package deal ever is a discount. I wonder whether anyone at that online store would consider this a bug. (I could see the store doing this on purpose on the theory that people are likely to spend more money if they think they are getting a deal.)

Diligent Reader Akhilesh emailed me to relate a bug he found in a web mail client which Akhilesh tells me claims to be bug free:

  1. Log in to your web mail account.
  2. Click on the Settings link on the top-right.
  3. Click on the notifier link.
  4. A new page pops up with a scripting error.

Oops.

There is an internal website which forms one part of the reporting pipeline for the performance tests I run. When I add a new test case it pretends to allow me to specify the new test’s ID. When I do so, however, I am told that “The test case ID ‘4324’ must be a valid integer between 1 and 999,999,999″. Umm….

Where there is software you will find bugs. Guaranteed. I can imagine an application which was built by a stellar team who spent huge amounts of time rooting out the very few defects they injected into it. (And probably missed their ship date by years…but I digress). I have yet to see or even hear of such an application.

Great Testers test everything they see, and they find bugs in everything they test. If you don’t see all these bugs, or you would like to see more of them, you can certainly do so. Start by applying the Rule Of At Least Three: Pick something. Anything. A street lamp. A door. Your toaster oven. Your application. Your competitor’s application. Think of at least three ways you could get that object to do something which you believe is unexpected. Once you have three things, keep going until you have ten. And then twenty. And then fifty.

If you get stuck, take a break and come back to it later. Discuss it with someone else. Browse the bug database for your application and use the bugs other people have logged as a springboard for brainstorming more possibilities. Push yourself farther than you think you can go. And again. And again. And again.

Even the most basic, simple object can have bugs. For example, how would you test a cup? Post your answers in the comments, or email them to me at michael dot j dot hunter at microsoft dot com.

Comments (13)

  1. I’m not a good tester – I don’t have the discipline or attention to detail for it – but "test a cup" is an interesting exercise…

    - Fill it very close to full, see if it spills more easily than expected

    - Stand it on various kinds of uneven surface – both full, empty, and partially full – and see if it falls over or spills

    - Fill it with something very hot, then very cold, repeatedly, see if it cracks (especially for a glass cup)

    - Wash it – in the dishwasher and by hand, with various kinds of detergent and on various dishwasher settings; verify that it doesn’t end up damaged and that if it bears a design, the design doesn’t fade. Again, doing this repeatedly would be good.

    - Drop it from various heights, full and empty. This is especially relevant for plastic cups that aren’t expected to break when dropped. Or infant sippy cups that aren’t expected to spill when dropped.

    - Look for leaks. Not sure what method I’d use, but it’s an important test to include :)

    How’d I do?

  2. Autumn says:

    I’d drop it.  That happens pretty often, and even though most cups will break, I want to know *how* it breaks.

    Does it shatter into tiny little pieces that are going to be difficult to clean, or does it break into nice large chunks?

  3. Ooh. Fill it with something very hot, and then try to hold it, see if it’s too hot…

    Actually I suppose what I should have said first of all was "Depends. What’s the spec for the cup?"

    But also if it’s relevant, you might fill the cup up with something hot and measure how quickly it cools…

  4. Drink out of it. Duh. Check for comfort and whether it spills when you do so. Get people of different ages (and hence different size mouths) to do so.

    Am I posting too many comments yet?

  5. Andrew says:

    Could you maybe elaborate on either the actual characteristics of the cup to be tested (i.e. fine china tea cup, coffee cup with logo, infant sippy cup etc) to allow an exploratory testing approach or give some idea of the requirements of the cup (i.e. cheap cup suitable for drinking a 16oz soda while in a car) for a more traditional approach to test case generation?

  6. Apoorva says:

    Perhaps I’ll start off by asking what am I testing the cup for?

    - To drink hot tea?

    - Act as a pen-stand?

    - Use as a missile when a developer writes buggy code?

    Dont great testers channelize their efforts, may it be anything they test?

  7. Ivolved says:

    Put the empty cup in a microwave for a 30 minute cycle.

    Put the cup in a dryer on the high heat cycle.

    Leave the cup in a freezer overnight.

    Leave the cup on a hot stove eye.

    Boil the cup in a pot…let all the water evaporate and leave it on the pot for 20 minutes after.

    Pour acid in the cup.

    Pour steaming liquid into the cup, dump then pour ice cold liquid into the cup.

    pour saltwater into the cup, complete an electric circuit using the water, and measure the voltage on the outside of the cup.

    attempt to complete an electric ciruit using the body of the cup…(you could probably skip the prior test and just go to this one)

    this series of tests would probably cause a failure in any cup (e.g. glass wouldn’t be affected by acid but would shatter after being exposed to extreme heat and cold in succession)

  8. 1.) label your cup as beta.

    2.) invite 1m of your closest friends to use the cup for an indeterminate amount of time.

    ???

    4.) profit.

  9. grauenwolf says:

    Hard…

    When you wash it, make sure water doesn’t get caught in the bottom of the glass. (It sucks to pick up a ‘dry’ glass, turn it over, and find yourself covered in water.)

    Use it to boil water in the microwave.

    Use it to freeze water.

    Drop it, repeatedly, from various heigths.

    Leave it on a electric stove, then turn it on.

    Get it really cold, then fill it with very hot water. Reverse and repeat.

    Act as though you are trying to place it on a table, but miss and hit the bottom against the side of the table or counter.

    Hit the top of the glass against the top of the cabinet while removing the glass.

    Test for balance when full, empty, and somewhere in between.

    Test for holding from the top with the hand directly above the glass.

    Test for people hitting it with silverware, like during a toast.

  10. Brian Caswell says:

    McDs is proof that people can’t do math in their head.

    In my area, a "20 pc chicken nuggets" costs $5.57.  However, "4 pc chicken nuggets" cost $1.00.

    Yet, I’ve been bitched at by both the cashiers and people in line behind me for spending too much money when I buy 5 "4 pc chicken nuggets" instead of the 20 pc.  I’ve attempted to explain it, and get blank stares.  Or worse, someone telilng me I’m wrong.

  11. Henrik Gammelgaard says:

    1: Drink from it

    2: Hold it

    3: Hold it with one hand

    4: Hold it in the handle

    5: Hold it while contents are hot/cold

    Might do the above things on a couple of test persons, so see how they use it and if it fits their usage pattern. For instance I often hold a cup different when standing up compared to when sitting in front of the computer; so one might have different usage scenarios.

    oooh . and better have some sticky stuff if in, to see how easily it is to get sugar and the like, out of the cup when washing it in the hand and when drying it in the hand. They may be of major annoyences to people who dont have dishwashers. Does it fit into normal dishwashers, industry dishwashers, etcetera etcetera.

    At EuroSTAR in Manchester (2006) I went to the art gallery and saw a glass by a guy named Joe Colombo, called Smoke; with this glass the designer had made it so that you could hold the glass, at a soiree, while having a cigarette in your hand – all with the same hand, which blew me away as the designer had actually looked at how people hold a glass while smoking, which the glass weas designed for, I presume, given its name.

    Lots of testcases for something as simple as a cup :)

  12. madHiker says:

    Ah, yes, package deals. The nearby grocery store (QFC/Kroger) always has tags with "deals": 5 for $10! 3 for $7!

    About 40% of people I asked think they need to buy the listed number of items to get the discount, so they will buy 5 of this, 3 of that. In reality, of course, the discount applies even if you buy a single item.

    I’ll look into the cup tests when they go on sale (100 for $12!).