We don’t make the software you use, we make the software you use better….


BASF used a variation on the above as it’s corporate tagline. Michael Hunter, the technical lead for my test team, used this as part of his auto signature for quite some time. It should be the motto for every software test/qa organization.


For those who don’t know him, Adam Barr worked for microsoft from 1990 to 2000, and wrote a book called proudly serving my corporate masters. He commented on a comment to a post I’d made about testing at microsoft. Here’s a couple excerpts from his comments.


So what I am trying to argue in the book is that testers are just as important as developers, in fact they may be more important, but it’s not because they are as good developers as the developers, it’s because they are good testers! That should be reason enough to respect them. In fact I make the claim that Microsoft started out in the “era of the developer”, moved to the “era of the program manager” around 1990 when Windows 3.0 shipped, and has now moved to the “era of the tester” — except nobody realizes this, and it is hurting the company.


Anyway you can read the book (it’s linked to online from my website if you don’t want to buy a copy) to get the full argument, it’s scattered throughout chapter 3, on pages 40-60. But please understand that I was trying to raise testers up to equality with dev and PM. I’m not sure what “He further rants negatively about testers especially in chapter 4” means but I don’t recall doing any negative ranting about testers. I rant negatively about the ATTITUDE towards testers. The quote on p. 50 about “for a variety of reasons they are viewed as lower on the pecking order than program managers and developers” is not me bragging about the way things should be, it’s me lamenting about the way they are.

Adam, thanks for clarifying your books points. I think that Adam and I are in agreement about the way things were, and that attitudes needed changing and understand that testing is currently a bottleneck in delivering quality software. I believe that the companies views have changed significantly in the last 2-3 years. Bill Gates quote from May 2002 : http://www.informationweek.com/story/IWK20020517S0011


INFORMATIONWEEK: When we were on Microsoft’s campus in March, it became clear that some people there think they’ve been developing quality software for years; in other words, it’s not a new concept to them. And we got a sense that a few people were a little bit defensive about it.


GATES: Well, let’s separate out quality from security. Microsoft in terms of this quality stuff–we have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We’re more of a testing, a quality software organization than we’re a software organization.


Now where Adam Barr and I may disagree: I want to hire people who are both great testers as well as being great developers; I don’t see these skills as mutually exclusive. I took my current job to start a new group 2 years ago. From day one we’ve worked to hire people who have strong testing experience or aptitude, and are skilled developers. Given the resource constraints, and the longevity of software servicing, as a manager, I must pursue 100% test automation as aggresively as possible, and have the automation completed as close to feature completion as possible. To build the framework necessary to deliver on this goal, you have to have strong developers. It’s simply not a simple task.


Think about it like this: students at school, or developers from other companies, have lots of knowledge on how to solve typical problems in software engineering. How to design a 2 tier or 3 tier data bound application; how to draw bezier curves; how to implement pixel shading support, etc, etc. There are books for every aspect of software engineering. Need a sort? Just look it up. How about jpeg compression? Again, just look it up. I’m not saying that everything developers do is regurgitate code. There is room for lots of creativity. But there just aren’t books or classes yet on how to write a test harness that can be shared by a development team and a qa team to run both unit tests and functional tests, that runs in process and out of process with respect to the application being tested. How about designing software that automates the installation of Operating Systems and other prerequisites? How about building a verification system that can test that your bezier curve is actually rendered correctly?


That’s why I believe that during the next decade, the hardest problems we have in software engineering are in the realm of testing. It may not be glamorous work; the code we write never gets put onto a single customer’s computer, yet quality is still the single most important feature we ship in every product. If it works as expected, people will have the ability to love it (of course, it also has to do something they care about for them to actually love it); if it is buggy, people will hate it, no matter how many killer features are in the box. Getting the product to market in a timely fashion at that quality bar, and putting the product in a position to be sustained without continually pulling resources away from the next version, unless you have unlimited resources (hah!), requires you to have quality test automation. To implement that, of course, brings me full circle back to my point on hiring people who are both great testers and great developers.

Comments (4)

  1. Adam Barr says:

    I am not saying testers should be not be good or really good or great developers. All I am saying is that they don’t need to be as good (as developers) as the SDEs. In an ideal world, all PMs would also be as good at development as developers are. Heck in a really ideal world all of marketing would be also. But in reality, Microsoft has thousands of open jobs, and can’t find enough people to hire. Given that, a top-notch developer would be steered towards SDE over SDET.

    Here’s a question: what is the ladder level of a typical entry-level SDE vs. SDET? What about a test manager vs. a dev manager? Until those are equal, SDET won’t truly be equal to SDE.

    (Also I’ll point out that in the timeframe I was discussing in my book, the SDET job was nonexistent or new, and I was really talking about STE vs. SDE. Those statements from recruiting and management about how testers were just as qualified, as developers, as SDEs were — those were being made about STEs, not SDETs.)

    Anyway, I was reading some of your other posts and there were a couple of interesting quotes (talking about what you look for in interview candidates):

    <I>For SDET, the bar goes up slightly in that I am looking for an additional skill. I look to see if they can test their own code. Sure, everyone tests their own code, right? It’s just part of the development process, right? Well, not exactly. Can they step back from the fact that they wrote the code, and look with a critical eye on their implementation?</I>

    <I>Ideally, we want to hire all #3 types [can test own code and other’s code] for SDE or SDET, but we really need them in SDET roles. Why? Well, for test code, there is no additional organization that test’s the test code to ensure that it is working correctly, so these people are the last line of defense between the bugs in their code that cause test code to not uncover flaws in the shipping code. #4s [can test own code, but not others’] are ok as SDE as well, they can test their own code, they just don’t help their peers that much via code reviews. Mostly, these people see one solution to a problem, and when people throw a different algorithm at them to solve a problem, they struggle grasping it, so reviewing someone else’s implementation is challenging for them.</I>

    What you are saying is that SDETs actually have a bigger skill set than SDEs (I didn’t see any mention of a skill that SDEs have that SDETs don’t). My first thought was "Nice try!" But it did make me think for a while. Is there an extra technical skill that SDETs have that SDEs don’t, beyond being able to tolerate obnoxious SDEs with a superiority complex? Nonetheless, I maintain my position that someone who would be a bad SDET (by lacking this skill, whatever it was) would also be a bad SDE. To be a good developer you have to be able to step back and analyze code. And you have to be able to consider different algorithms — if you can’t do that, you’re not a good developer, period. The only difference between testing your own code and someone else’s is that you might view it as beneath you to have to look at someone else’s code. I think THAT is the difference between #3 and #4. It’s an attitude thing, not a difference in technical skills. And I would say that the #4 people (of which Microsoft has plenty, certainly) ultimately make bad SDEs also.

    You used to see interview feedback (frowned upon by HR) that said "he’s not good enough to be an SDE, maybe try SDET?" Your post has given me hope that someday, you might actually see someone write "I don’t think he has the skills to be an SDET…maybe he could try for plain old SDE"?

    – adam

    P.S. Incidentally I am back at Microsoft, working as a PM (insert joke here). The single thing that most impresses me about my new group is the quality of the test team, AND the attitude of the dev team towards the test team. So maybe things are changing, slowly.

  2. I. M. Testy says:

    What is the difference between a software test engineer (STE) and a software design engineers in test…