Quality of Testing and Quality of Code

I see a good number of Tweets in my Twitter stream that just flash by without leaving any impression. Chatty comments between people I barely know (if at all), links to things that don’t really interest me, and other items that really don’t have a lifetime beyond a few minutes after they are sent. But enough them really do make me think or take action that it makes Twitter  pretty useful for me. Recently I saw the following Tweet from Eugene Wallingford.


Student developers do “interesting” things. For example many of them try to write the whole project before starting to test. Oh they may run the compile to check for syntax errors – as if a program that compiles cleanly will automatically work correctly. And then they start to test. This seldom goes well because there is just much too much to test all at once. Especially for a beginner finding logic errors in a (theoretically) complete program is just too complicated. What makes it worse than just the quantity of code (which let’s face it is not all that much compared to a professional program) but the number of code paths (i.e.. places for things to go wrong) that is just more than a beginner can trace in their heads. Yes, in fact, a beginner writing code this way means that the quality of the code may just not be good enough for easy testing. The result is a long and painful testing process.

Experienced programmers test small pieces of code. The fewer code paths the easier it is to design a set of test data to test them all. This, test design, is not something we tend to spend much time on in introductory courses. Or perhaps in anything other than a software engineering course which is a shame. I think we could save a lot of students a lot of heartache and perhaps reduce attrition (theory of mine) if we taught them how to test earlier.

We ask students to do a number of things when creating their own software. I tend to start with three large items:

  1. Understand the problem
  2. Design a solution (algorithm)
  3. Code up the solution

Many start the testing thinking during (or worse after) the code up the solution part. Not a good idea. At the latest we need to start thinking about testing no later than design a solution. In fact if we understand the problem we should already have a handle of testing. At that point we should know  - what is the right answer for various inputs? What values should be outside the range of what we want to handle and how should we deal with them? In short, testing data is all part of understanding the problem. This understanding should directly lead to test data. Much of this test data would should manually run through our algorithm before we code solutions. When we test our data with the code the results we achieved during manual testing of the algorithm should match.

Reducing the pain of testing requires several things of which having good code is just one. It’s one important piece of course but not the whole piece. Having good, solid testing data with known expectations of results is a piece. So is testing small pieces of code so that fewer things can go wrong. The fewer the places for things to go badly the easier it is to find and repair them. By coincidence (not) breaking ones code into smaller, more manageable and testable pieces almost invariably results in better code. Sections of code that are too long and that try to do too many things in one module are the huge run-on sentences of programming. In natural language writing run-on sentences are a “bad thing” and make understanding difficult for the reader. Modules of code with too many lines make testing and understanding difficult. They also make modification, expansion, reuse, maintenance and all sorts of other things difficult if not impossible.

The first programming course is a difficult course to design. Students have to learn a whole new way of thinking as well as a new language in which to express their ideas. That alone makes it difficult. And then there are so many things that we could make an argument belong in the first course. I could see a first course that is four years long if we tried to get it all in. So by necessity things get left out. I am thinking, at this point in my life, that testing is not one of those things we can afford to leave out though. Not that we spend forever on it of course.  I also don’t think it is a separate unit either. Rather I think it is something we talk about as we talk about other things. Include it is the discussion of understanding the problem. Include it with designing algorithms. And include it when we talk about writing the code as something more than just having the compiler check the syntax. It it means that something else waits until a second or third course so be it. On the other hand I wonder if doing better testing and including testing thinking in the whole process might just reduce the amount of time student projects take to complete. It’s a theory. What do you think?

Comments (7)

  1. David Klappholz says:

    Re "On the other hand I wonder if doing better testing and including testing thinking in the whole process might just reduce the amount of time student projects take to complete. It’s a theory. What do you think?"

    There's a huge amount of data that shows that in industrial software development, putting off validation and verification — including testing — until all the code is written can cause a project to take 5 or ten times as long to complete than if it's done at every step, starting with requirements elicitation.

  2. Dan W says:

    Unit testing, at a minimum, is super important to teach. It's not that hard to write a unit test if you are using VS2010. But I don't think your average college freshman will understand it, aside from introducing testing concepts… maybe CS2 is a good place for it though.

  3. Garth says:

    As those that read this blog are aware I teach high school programming.  Now considering I teach a whole bunch of other stuff and am the school’s only semi-trained computer/system tech I do not have a whole lot of time to design programming courses in the manner in which I would like to teach them (damn those wasted hours eating and sleeping).  So it would be nice if there was a text book that actually taught programming and not just coding.  Of Alfred’s big three, number three seems to be the one and only topic covered in a standard high school programming text.  I think 1 and 2 are much more important and number 3 should be shipped overseas.  The coding should be taught in snippets: write a small piece, test, paste it into the project.  The trouble is the kids are rarely taught how to design the project.  A good course would have an interesting project as a final goal (“interesting” is a key word here, a game is always a biggie, remember we are talking high school here) and the semester would be spent designing the project away from the code, then start coding the pieces of the game, testing the piece, build the next piece, test and assemble.  Kids should not be hand fed the needed code as most tutorials do.  The kids need to see they need a piece of code to do a particular task, then go look in the API, find a resource with piece of code to modify or, if all else fails, write something original.  I had better quit writing this and look at what I have to teach in Math II next period.

  4. G Andy says:

    First off make your page readable – decent size font and don't make people scroll left to right to read each line if they have to zoom it to read it.  

  5. AlfredTh says:

    G Andy, what screen resolution are you using? That would be helpful for me in trying to figure out better design because of course it is all pretty easy to read on the screens I use even though I need new reading glasses. Thanks.

  6. Mike Ferdinand says:

    2 initiatives come to mind: first, a course on designing test modules, test data, and test widgets that could exercise code inside and outside the development environment.  Second, include in each programming assignment some tasks that involve thinking about how it could be tested, or how some parts could be tested, and include that in part of the grading for the assignment.  

  7. Mad Bollie says:

    Jeez finally in all the net chit chat on how to be a better programmer someone has finally got an inkling of a clue!!!

    Just as developers should cross-train a little in electronics, communications and micro-computing, so they should also cross-train a little in the testing disciplines.  Back in the early '90s my BTEC was pretty good but didn't really prepare me for being a professional developer.

    Send your students onto a typing course – RSA and Pitmans – to get a basic standard of 30wpm.  Don't accept any code unless it is commented.  Teach this by making them write a comment line above every line of code they write for at least 6 weeks.  It'll soom become habit and then they won't even obsess about doing it later.

    Consider other aspects of development early on that would be necessary to make a better rounded developer from the start.  Don't just teach them some basic testing techniques, but all how to use a debugger.  Also teach them the alternative to debugging – logging techniques.  Teach them the 3 important aspects of fault finding are where variables are created/initialised, where variables change value and the entry and exit points of functions.  Teach them that understanding an unfamiliar algorithm goes hand in hand with understanding how a programs data is structured with in a program.

    Teach them the concept of unit testing and test harnesses.  Get your students used to the idea of isolating functions or subsystems to debug and testing independantly of a larger program.  Teach them source control and get them comfortable with using it.  Teach them how to document their work in terms of requirements, design and test plans which are really the critical development documents for any project.

    Teach them that files should have a comment banner with useful information like copyright notices, authors etc.  Teach them that a function/method/procedure/class should also have a banner giving the name, description, input and ouput parameters.

    When you want your students to work together to collectively produce a larger program as a project be sure to realise they won't have a clue what a project is and how to organise themselves into an effective team.   At this stage structure the work into sub-systems for the students and teach them the importance of defining and working to interface specifications.  That way if one student lets down the others in the group – someone always does – the others can always borrow the missing part from a successful group or from the tutor's official solution and complete their work to receive their grade.

    Also in the latter stages of a course you may want to provide an introduction on how to use a fault management systems like JIRA, the concept of project risk in software and provide practice in optimising code and dealing with crappy code that the student didn't write.

    Throughout the whole course get the students into the habit of estimating the time their work takes and review with them how accurate their estimates were, or not, and why.

    There's loads more, but I hope you are getting the idea now!

Skip to main content