My Box is Round

Metaphors are cars – I think. There are many metaphors in software, a lot of them are useful, but most break down eventually. In hwtsam (chapter 3 to be exact), I compare creating software with cooking. I’ve mentioned on numerous occasions that  testing is like detective work (technically, that’s a simile, but go with me).

In testing, the concepts of black-box and white-box testing have been around for years, and are commonly used in introductory test courses to explain different ways to approach testing. Many explanations go on to say that a “grey box” approach is preferred as it combines both approaches into a more holistic approach.

This weekend, Phil Kirkham asked “What Colour Tester are you?” I whimsically answered “Bright Grey”, but Michael Bolton had a more interesting answer. He denied the existence of the box (which I found kind of strange since he teaches a course called Black-Box Software Testing). Anyway, his elaboration was somewhat obvious, but something that rarely comes out in the box explanation, and was good for the community to see.

Upon reading this, I was immediately reminded of a reply I made in another discussion forum just a few days ago in response to the question “How do you keep fresh ideas flowing”. My answer – which I thought was worth repeating follows.

When I’m feature testing, I try to test from as many different angles as I can (I call this 360 degree testing, but that’s just me). If I’m stuck, I’ll ask myself two questions: What haven’t I tested for yet?, and What testing approaches haven’t I tried yet? (sometimes these overlap). My initial exploratory tests may have been primarily looking for issues in functionality or negative input, so next, I may brainstorm on how the font feature may fail (fonts could overlap, fonts could be illegible, wrong font could be displayed, etc.). Then, I’ll try to make those things happen in as many ways as I can think of.

When I’ve exhausted that avenue, I usually change directions entirely and look for memory leaks (I wonder if large fonts make word use more memory? If they do, is the memory released when I change back to a smaller font? What other ways could I cause a memory leak with fonts.

Eventually, I may examine the code or code coverage data (since I work at MS, those are available to me). Gaps in code coverage, or code review may give me more ideas on what to test. Probably much earlier than this point, I’ll talk to the developers on the team who deal with fonts, as well as any other font experts I can find to see what kinds of tests they run. This ends up turning into a perpetual circle – by the time I’ve tried everything I can think of, I’ve learned enough that I can try the things I tried in the first place, because now I have a new perspective and have slightly different approaches to take.

It seems that I’ve flowed away from the original question, but I suppose I consider all 360 degrees exploratory (others may argue). I’m learning as I execute tests and changing my approach whenever I get stuck and want to get some "fresh ideas flowing".

So, I suppose my box is actually 360 degrees – which if my math is right, sort of round.


Comments (10)

  1. Shrini says:


    Quick question… do you distinguish between a metaphor and a model? I saw the post on software testing club… what immediately occurred to me was "box" model.

    (I understand that in Microsoft, the model used in conjunction with testing could be easily misunderstood with "Model based testing" – No .. I am not talking about state models)

    Coming to your concept of 360 degree testing or round box –  you still seem to be thinking about only one box … Are’nt you? If you read my responses that Phil Kirkham’s post – you can see that, there are actually many boxes ..not only one.

    BTW, as far I know Michael Bolton does not teach "Black Box Software Testing" but he teaches "Rapid Software Testing course" with James Bach … In the AST link that you mentioned about BBST – I find the names of Cem Kaner, Rebecca Fiedler and James Bach as instructors … I don’t find Michael Bolton’s name … Am I missing something?


  2. Alan says:

    I do indeed distinguish between a metaphor and a model. A metaphor is simply a grammar construct used to evoke a mental image and see connections between seemingly unrelated items. A model often has those same traits (hence the ability to use the terms interchangeably on occassion), but typically provides a much richer set of information to support and describe the concept.

    Sure – I’m thinking of one box – one box that consumes every other conceivable box in the universe.

    or – I’m thinking about multiple boxes – depending on where my mind is at the time.

    As far as the BBST course goes, I just assumed since JB teaches the course, and that MB does just about everything JB does that he taught it too.

  3. calkelpdiver says:

    I guess to be a smartaleck about this we should think about what Boris Beizer said a few years back (how many I don’t remember, I just remember his comment on a post), "There are no boxes, quite being ‘boxed in’.  Think beyond the borders."

    Which caused a heated debate, but made sense in that he was saying don’t limit yourself to one or two techniques/methods and levels of testing in the system.  Expand your view from the local trees to the whole forest.

  4. Point of clarification:  I don’t teach the Black Box Software Testing course.  Our course (which I write with senior author James Bach) is called <a href=""Rapid Software Testing</a>.

    There is a course called <a href="">Black Box Software Testing</a>, written by Kaner and Bach, and sponsored in its current incarnation by the Association for Software Testing.  I’ve trained to teach it, but haven’t taught it.  I don’t object to the title.  I don’t even object much to the black box/glass box model; I can use it to make myself more aware of the boxes such that maybe I can avoid being constrained by them.

    Boris Beizer also wrote a book called Black Box Software Testing, despite his repeated denials that such a thing exists.  This reminds me of Stephen Shapin’s book "The Scientific Revolution:  "The Scientific Revolution never happened," he writes, "and this is a book about it."

    And I like the round box metaphor too.  Or is it spherical?

    —Michael B.

  5. Shrini says:

    >> And I like the round box metaphor too.  Or is it spherical?

    This reminds me …. what is special about "round" or "spherical" with respect to box/color analogy?  No edges?

    Can one assume round or spherical box (or should called sphere) has no visible edges (like cube like box) ?

    Does round mostly implies a 2 Dimensional geometrical structure and spherical /surface/  – a 3 dimensional one?

    What would be the implication of a spherical box (apparently with no edges) to box/color analogy?


  6. Shrini says:

    >> And I like the round box metaphor too.  Or is it spherical?

    This reminds me …. what is special about "round" or "spherical" with respect to box/color analogy?  No edges?

    Can one assume round or spherical box (or should called sphere) has no visible edges (like cube like box) ?

    Does "round" mostly imply a 2 dimensional geometrical structure and spherical /surface/  – a 3 dimensional one?

    What would be the implication of a spherical box (apparently with no edges) to box/color analogy?


  7. Alan Page says:

    Michael – I stand corrected. I assumed (for a variety of reasons) that you also taught the bbst course. In context of the discussion, I could have said you "were involved" – or looked it up, but apparently I was in a hurry to make a point :}.

    Shrini – I said "round" for two reasons. One is that since boxes are NOT round, I was attempting to say that boxes don’t have to be boxes. The second, more direct reason, is that I mentioned 360 degree testing, and 360 degrees is a circle.

  8. I.M.Testy says:

    Actually Beizer’s book is entitled "Black-Box Testing: Techniques for Functional Testing of Software and Systems" and in the preface he clearly defines what he implies by black-box tesing. He also goes on to state that many people who start testing without formal training in the subject simply rediscover or "reinvent heuristic versions of methods in the literature."

    If one really reads and understand Beizer’s books and papers (from other than a metaphysical point of view) it isn’t too hard to understand that he disliked the limiting connotation of black and white and generally preferred phrases such as behavioral or functional or structural to descibe test design approaches. Thus, I suspect his use of "Black-Box Testing" to title a book was simply to appeal to the masses at a time when a large percentage of people entering the discipline had no formal background in testing or software engineeering. Unfortunately, the book was not very practical for the masses; especially for those who couldn’t comprehend it. It was very interesting to those who could and much of what is covered in that book is the foundational bricks of model-based testing.

    A very similar discussion to the whole black white thing occured in 2004 here. Bret does an excellent job of explaining the concepts and some of the comments are especially interesting as well.

  9. Alan,

    You have given what looks like a workable approach to generate fresh test ideas. However, it is possible that a tester does not have all those degrees of freedom (to go black box/ white box/ bright gray box or any other color). Such a situation is possible when testing is conducted as a business task, for example, when a company signs a contract with another one regarding the kind of independent testing it would do that would add significant business value to the latter’s product or service. In such a situation, the tester’s scope might be limited to the budgeted scope, pre-planned effort and sometimes even pre-planned tests. I suspect that an in-house tester might not face these constraints, at least not severely.

    Inder P Singh