Scenarios and Underpants Gnomes

Data driven quality requires a certain kind of thinking. It took me a while to understand the right thought process. I kept getting caught up in the details. What were we building and what would use look like? These are valid questions, but there are more important ones to be asking. Not asking whether the…

0

Confirmatory and Experimental Metrics

As we experiment more using data to understand the quality of our product, the proper use of telemetry becomes more clear. While initially we were enamored with using telemetry to understand whether the product was working as expected, recently it has become clear that there is another, more powerful use for data. Data can tell…

0

The Data Driven Quality Mindset

"Success is not delivering a feature; success is learning how to solve the customer’s problem." – Mark Cook, VP of Products at Kodak I’ve talked recently about the 4th wave of testing called Data Driven Quality (DDQ). I also elucidated what I believe are the technical prerequisites to achieving DDQ. Getting a fast delivery/rollback system…

0

10 Years of Blogging

I’m a little late, but it’s time to celebrate anyway. Ten years ago, in March of 2004, I began blogging here. Hello World was, of course, my first post. You have embraced and sometimes challenged my words. Thank you for continuing to read and to engage with me. I learn a lot by writing. It…

3

Try.Fail.Learn.Improve

Try.Fail.Learn.Improve. That has been the signature on my e-mail for the past few months. It is intended to be both enlightening and provocative. It emphasizes that we won’t get things right the first time. That it is okay to fail as long as we don’t fail repeatedly in the same way. Try.Fail.Learn.Improve is a process…

0

Prerequisites to Data Driven Quality

A previous post introduced the concept of data driven quality. Moving from traditional, up-front testing to data driven quality is not easy. It is not possible to take just any product and start utilizing this method. In addition to cultural changes, there are several technical requirements on the product. These are: early deployment, friction-free deployment,…

0

Perceived vs. Objective Quality

I recently heard this story, but I can’t recall who told it to me. I don’t have proof of its veracity so it might be apocryphal. Nevertheless, it illustrates an important point that I believe to be true independent of the truth of this story. As the story goes, in the late 1990s, several Microsoft…

1

Data Driven Quality

My last three posts have explained how test lost its way. It evolved from advocates of the user to a highly efficient machine for producing test results, verifying correctness as determined by a specification. Today, test can find itself drowning in a sea of results which aren’t correlated with any discernible user activity. If only…

1

Halt and Catch Fire

I just finished watching the first episode of the new AMC show called Halt and Catch Fire.  The name comes from an old computer instruction which would stop the machine immediately.  The show follows a small Texas company trying to build IBM PC Clones.  The company and the people are fictitious, but it seems to…

1

Test Has Lost Its Way

In a blog post, Brent Jensen relays a conversation he had with an executive mentor. In this conversation, his mentor told him that, "Test doesn’t understand the customer." When I read this, my initial reaction was the same as Brent’s: "No way!" If test is focused on one thing, it is our customer. Then as…

8