Jamming on Testing


Over the last 15 years, on far too many occasions to count, I’ve explained to various people how software testing, and software engineering relate to music. I know other testers have made somewhat similar analogies, but I think I have a different take on the subject. I also have to admit that I get a little tired of the "Testing is like…" topics sometimes, but I’ll give this a spin anyway.

One topic I often bring up is how much great testing and great improvisation skills have in common. I know, and have heard many musicians who, through use of a good ear alone, can sound reasonably well improvising (playing without sheet music) on their instruments. To many people, these folks sound more or less like they know what they are doing. The good players, however, have mastered the fundamentals of their instruments. They know every scale throughout the entire range of their instruments. and have practiced hundreds of hours of scales, chords, and patterns for every minute of what you hear them play. The masters of improvisation, including some of my heroes like Sonny Rollins, Dexter Gordon, John Coltrane, Sonny Stitt, Charlie Parker, Hank Mobley, and Stan Getz,have studied and practiced the basics even more. The thorough knowledge of the basics is what makes them so good.

Testing is the same. You can learn a few licks and think on your feet a bit and get by in testing almost like you know what you’re doing. However, if you want to master testing (or just get good at it for that matter), you have to dig down and master the basics – no matter how smart or creative you may be. Thorough knowledge of testing fundamentals – along with domain knowledge of what you are testing will guide your testing and lead to success. The more you know,  the better job you’ll do, and the more effortless your testing will seem.

The funny thing about improvisation is that for the better players, it isn’t all that improvisational. A musician may "discover" a new melody, quote, motif (or lick) in the practice room, then practice it in every key until they can play it without thinking. Then, in the middle of their solo, when just the right rhythm and chord progression present themselves, they’ll play the lick. As a player builds up enough of these licks, they use them fluently throughout their solos. If you listen enough to any good improviser you’ll hear many of the same motifs throughout their solos. This, of course, makes sense – this is what they practiced. Coltrane’s Giant Steps is a great example of this. It’s fantastic playing, but when you listen closely, most of what he’s playing are the same patterns over chords that many other musicians are playing in the practice room (Trane is just linking them together and playing at breakneck speed).

Great testers use their knowledge of testing fundamentals and software applications in the same manner – they learn the basics (techniques and some knowledge of how computers work), and wait for the perfect opportunity to apply what they know in an effective and meaningful way.

Dig it?


Comments (8)

  1. Shrini says:

    Great Post Alan ….. Much needed …

    Improvisation lies on the opposite side of preplanned or premeditated.  Much like performance in music, public speechs, sports – testing provides perfect platform for improvisation.

    Testing approaches like Exploratory testing provide right environment for improvisation. Scripted testing forces tester to follow script (much like musician playing a set of written notes).

    Can you share some of your experiences of improvising testing? What did you discover as a result of improvisation? What did such improvisation led to? What did you discover?

    Shrini

  2. alanpa says:

    I guess my stance is that good testing (which I believe is exploratory in nature) is all improvisational. If you just read the notes and don’t interpret anything, you’ll sound like crap. If you just run test cases and don’t interpret anything you’ll test like crap.

    The point is that in order to interpret, you need to know what you’re doing – you can’t just fake it.

  3. Shrini says:

    It is being said about "improvisation" that only masters or experts can improvise (as mentioned by you – for them it is non-improvisation – no big deal).

    Can a learner improvise on what he is doing ? At what stage a practitioner can claim to improvise?

    I am asking as "learning" about what you are trying to test is a central to idea of exploratory testing. Unlike scripted testing approach, in ET – learning, modeling, analysis, improvisation all said to happen almost  spontaneously.

    How can learning and improvisation happen simultaneously?

    Shrini

  4. alanpa says:

    In regards to improvisation, anyone can do it – but to be good at it, you need a masters control of the basics.

    In music, someone can indeed improve a bit just by doing it, but progress will be slow and quite limited. You can certainly still have fun improvising, but if you want to go pro, you have to spend a few thousand hours in the practice room – there’s no way around it.

    My hunch is that testing is similar, but that may be just because I’m familiar with both sides of the story.

  5. Zach Fisher says:

    It is great to find fellow musicians in the software testing field. I have been noticing similarities since I started and enjoyed reading your perspective. All metaphors break down eventually – and I don’t want to beleaguer the point – but I wanted to add a few observations.

    Good improvisation happens within a framework and a context. The changes, form and structure create the framework. The context could be the style, the ensemble, the social event, the leader, the instrument, etc. I see good testing happening within a framework of ideas applied to a particular context i.e., domain.

    Good improvisation depends on that musical domain. Some forms are more accepting of exploration, while others are more rigid. Think free-jazz as opposed to bluegrass. Some jobs may not provide us with the opportunity to "blow over the changes", but that should not keep us from doing good work within the given constraints.

    Good improvisation is not merely the byproduct of relentless hours cycling through the circle of fifths or Slominsky’s Thesaurus of Scales and Melodic Patterns. It requires a diet of experience playing out with/for others. It also requires time devoted to listening to other people playing. Practice can be exalted to the detriment of making music. In our context, I would equate this to the idea that sheer analytical rigor will place us in the upper echelons of our craft – or worse – make us think that we’re better than we actually are.

    Good is in the ear of the beholder. While I was in the conservatory, good meant something different from what I found in the real world. It is easy to forget that not everyone is a musician. As such, they cannot be expected to appreciate the subtlety of our craft. But they do recognize when the quality of the music does not meet their standards.

    Finally – and I think you implied this in your post – the great musicians that you mentioned would probably be hard-pressed to provide a detailed Western analysis of their performances. I don’t think that they approach music by thinking, "ii-V-I = dorian-mixolydian-ionian" or whatever extended tri-tone subs that could be applied. They simply play – on a foundation built through experience, listening and practice.

  6. alanpa says:

    Fantastic comment Zach – everything you mention is spot on and much appreciated.

  7. After my last post on music, a reader emailed me and asked me about computers and music. They drew an