Advanced Placement Computer Science - Time for a Big Change?

OK let's start with the disclaimer first. These are my personal opinions and only my personal opinions. I am not speaking for any company or any other individual in full or in part.

I have been giving a lot of thought to the Advanced Placement Computer Science program lately. I'm spurred to try and start a discussion now in part because the time appears to be ripe for it. A week or so ago Chris Stephenson, Executive Director of the CSTA, wrote in her blog that there is  a "growing sense among both K-12 and university educators that the [AP] exam is in need of a significant review and revision. " At SIGCSE last week I heard a number of teachers complain to the AP Test Development committee that the exam was tied too closely to not only a language but a specific version of that language. My friend Dave Jacobus reports in his blog that a tool he has used for teaching AP CS is no longer usable because of changes (expansion) in the exam related to that specific version of the language. And in spite or perhaps because of this I hear university faculty members complaining regularly that the incoming students they get who have taken AP CS don't know much and appear not to have been well taught.

This is all happening at a time of crisis for computer science. A British academic asks "Is Computer Science Dead?" Enrollment in high school and university computer science programs is on a decline. While the AP CS exam took a small increase last year it is still below its peak. What in the world is going on and how do we fix it?

Let's take a look at the AP CS exam. In theory it is supposed to reproduce a college level CS course. The idea behind the A exam is  to be like a typical CS1 first programming course. The goal behind the AB exam is to include both CS1 and CS2 (data structures). ETS (the Educational Testing Service) surveys university faculty and asks them what should be in a CS1 and CS2 course as well as what they include in their CS1 and CS2 courses. I have heard from someone close to this process that often there is a disconnect between what faculty say should be in the course and what they actually include because of time constraints.

The test development committee is made up of both high school and university faculty and the chief reader or senior faculty advisor is a university CS professor. They take the results of these surveys very seriously. Unfortunately either because there is too much focus on what should be (rather than is) in the course or because enough people claim enough different things are actually in their course it appears that the test tries to make too many people happy by including too many things. I believe that the material that is actually covered, or attempted and tested, is probably more than all but a few very advanced universities actually offer. I do not believe it is an average CS1 and/or CS2 course. Note also that Java, while widely used in these courses, it is not exclusivity used. There are a good number of languages with widespread use in CS1 and/or CS2 courses.

So why is that a problem? Actually there are several problems. One is that this makes the course too difficult for too many students. Computer Science is widely perceived as a difficult subject already and the AP CS curriculum would clearly support that idea. The other problem I see is that the shear size of the curriculum means that many subjects have to be covered in too little depth. There is in effect teaching to the test and I believe this results in "learning" that is not retained. Too many students must relearn basic concepts in college because they didn't really grok it in high school.

This latter problem is exacerbated by the required case study and the amount of time that must be spent on it. Now I love the idea of students reading code and I do see a real need for students to understand larger programs. I'm just not convinced that CS1/2 is the place to look at a large body of code. I do believe that practice reading code and perhaps even a formal code review process would be useful in a first course. Our students spend too little time critically reading their own code and so learning this process could really help. But as it is the case study accounts for about a quarter of the test questions and so must be studied at length. This is far out of proportion with the value of the concepts being taught this way.

And then there is the choice of language. Are there any concepts in the AP test set that cannot be covered as well in Visual Basic, C++, C# or any of a number of other languages? I don't think so. And yet AP CS teachers, unlike university faculty, are forced to use Java no matter what they think of its suitability for the purpose. Perhaps worse still that there is an enforced importance of syntax over concepts. Now I have graded the AP CS exam so I know that there is some allowance around harmless syntax errors. One can get away with missing the occasional semi-colon for example. But there is still a lot of opportunity to get messed up by syntax. If we could get away form syntax a little it might open the door for more languages to be used for teaching. Perhaps.

When you talk to the AP Development team, and I have, you will come to understand that creating the test is not an easy process. Grading it is also not an easy process. Restricting the test to one programming language makes it all less painful and less expensive. But at the same time I think that part of the reason the language is at issue is that so many of the concepts tested require very different syntax in different languages. This is particularly true around class creation but there are some differences around generics and even loops between Java and other C family languages like  C++ and C#. I do believe that if the testable set were reduced to what is actually required and used by university CS1/CS2 courses the number of such items could be reduced.

The ideal in my opinion would be a language independent exam. My second choice would be to allow faculty to choose between several languages. If the set of concepts is really as basic as it should be than it shouldn't be that difficult to translate questions between languages. I would argue that is the concept cannot be expressed in all of Java, C++, C# and Visual Basic (or a generic BASIC) than one should ask if it even belongs in the AP subset.

Translating and printing up different tests adds something to the expense and logistical complexity but with modern printing techniques and good computer tracking software that shouldn't be a show stopper. The next argument against this would probably be grading the free response portion of the test.

The test is graded, or in AP parlance read, by a mix of university faculty and high school teachers.  A university professor worth his/her PhD should have at least a freshmen college level understanding of these languages. So I don't see a problem with them. I would like to think that we'd get the same mix of readers from high school that we would see in teachers using the various languages. That is something the chief reader could look for and ask about in the applications for the readers he appoints. If load balancing needs to be done I am sure that enough teachers are or will be familiar with multiple programming languages to move around.

All in all it seems as though language independence or at least expanded language choice really revolves around getting the right curriculum concepts and having the will to do it. Allowing teachers to pick the language they like best, right sizing the curriculum and getting some extra time for teaching by dropping the case study should improve the AP CS program in several ways.

Most importantly it should allow teachers to do a better job of teaching the most important concepts. This should improve acceptance of the results by universities as they should see better prepared students. Students should find the course more interesting as teachers use their increased flexibility to teach in ways that let them show of their creativity and enthusiasm for the material.

After that, or maybe instead of that as I see little chance for AP CS reform, we need to work on getting more computational thinking and more actual programming in schools well before AP CS. We also need to find some more and better ways to train the teachers who are teaching computer science. There is a lot to do here.

[BTW if you have a blog and an opinion on this topic (agree or disagree) I invite you to widen the discussion by posting comments on your blog and linking to this post. Let's build a community wide discussion.]

Technorati tags: apcs, collegeboard, csta, java, visual basic