The trouble with test questions

One of my students complained to me about the questions on a test once. The question in question came from a previous Advanced Placement Computer Science exam. The complaint was that the code in the question was bad. It was bad because no one in their right mind would write a program like it. It went about things in an unnecessarily complex way. Unnecessarily that is if your goal was efficient, easy to read and easy to maintain code. The student was right. He went on to state that the AP CS program was full of how not to do things. Again, in a way at least, he was right.

The problem is that often on a test you are trying to demonstrate something in a small number of lines of code or you are testing a student’s ability to understand the operation of a concept. Sometimes you have to bend the rules about good design to force things into a small but complete demonstration.

The problem for students is that they just see a small but complex program doing something that could be very simple to do. They want to jump in and “fix” the program. Teachers see the concept they are trying to teach but students just see artificial code. It’s a tough problem. It is also one of the reasons the AP CS curriculum has a complex case study – it lets students see a good design on a reasonably sized project. If only we could do more of that.

I was thinking about all of this while reading a blog by Dan Forhan the other day. He has an example of a Java puzzle that is some very convoluted code.  One would never write a program like that to produce the output that puzzle produces. Well not if you intended people to actually use it you wouldn’t. It’s just not the write way to do it.

At the same time you might very well right code that has 20 times as many lines but the same organization of  classes and methods all calling each other. You’d no doubt be trying to solve a more complex problem than a bunch of System.out.println statements of course. If a student can’t follow the simple (ok simple here is a relative term) example in the puzzle how will they ever follow a more complex problem?

One tool that they could use though is a good debugger that will let them step through the code statement by statement. I think that many times teachers ignore the value of a debugger as a means for demonstrating the path the computer takes through a program. Debuggers can show the program flow, how and where the values in variables are changed and many other things that are not always obvious when checking code on paper. I liked to show the steps through the debugger using an LCD projector on a white board or screen in class. Sometimes seeing is believing.