Syntax, Semantics, Micronesian cults and Novice Programmers


I’ve had this idea in me for a long time now that I’ve been struggling with getting out into the blog space.  It has to do with the future of programming, declarative languages, Microsoft’s language and tools strategy, pedagogic factors for novice and experienced programmers, and a bunch of other stuff.  All these things are interrelated in some fairly complex ways.  I’ve come to the realization that I simply do not have time to organize these thoughts into one enormous essay that all hangs together and makes sense.  I’m going to do what blogs do best — write a bunch of (comparatively!) short articles each exploring one aspect of this idea.  If I’m redundant and prolix, so be it.

Today I want to blog a bit about novice programmers.  In future essays, I’ll try to tie that into some ideas about the future of pedagogic languages and languages in general. 

Novice programmers reading this: I’d appreciate your feedback on whether this makes sense or it’s a bunch of useless theoretical posturing.

Experienced programmers reading this:  I’d appreciate your feedback on what you think are the vital concepts that you had to grasp when you were learning to program, and what you stress when you mentor new programmers.

An intern at another company wrote me recently to say “I am working on a project for an internship that has lead me to some scripting in vbscript.  Basically I don’t know what I am doing and I was hoping you could help.”  The writer then included a chunk of script and a feature request.  I’ve gotten requests like this many times over the years; there are a lot of novice programmers who use script, for the obvious reason that we designed it to be appealing to novices.

Well, as I wrote last Thursday, there are times when you want to teach an intern to fish, and times when you want to give them a fish.  I could give you the line of code that implements the feature you want.  And then I could become the feature request server for every intern who doesn’t know what they’re doing…  nope.  Not going to happen.  Sorry.  Down that road lies cargo cult programming, and believe me, you want to avoid that road.

What’s cargo cult programming?  Let me digress for a moment.  The idea comes from a true story, which I will briefly summarize:

During the Second World War, the Americans set up airstrips on various tiny islands in the Pacific.  After the war was over and the Americans went home, the natives did a perfectly sensible thing — they dressed themselves up as ground traffic controllers and waved those sticks around.  They mistook cause and effect — they assumed that the guys waving the sticks were the ones making the planes full of supplies appear, and that if only they could get it right, they could pull the same trick.  From our perspective, we know that it’s the other way around — the guys with the sticks are there because the planes need them to land.  No planes, no guys. 

The cargo cultists had the unimportant surface elements right, but did not see enough of the whole picture to succeed. They understood the form but not the content.  There are lots of cargo cult programmers — programmers who understand what the code does, but not how it does it.  Therefore, they cannot make meaningful changes to the program.  They tend to proceed by making random changes, testing, and changing again until they manage to come up with something that works. 

(Incidentally, Richard Feynman wrote a great essay on cargo cult science.  Do a web search, you’ll find it.)

Beginner programmers: do not go there! Programming courses for beginners often concentrate heavily on getting the syntax right.  By “syntax” I mean the actual letters and numbers that make up the program, as opposed to “semantics”, which is the meaning of the program.  As an analogy, “syntax” is the set of grammar and spelling rules of English, “semantics” is what the sentences mean.  Now, obviously, you have to learn the syntax of the language — unsyntactic programs simply do not run. But what they don’t stress in these courses is that the syntax is the easy part.  The cargo cultists had the syntax — the formal outward appearance — of an airstrip down cold, but they sure got the semantics wrong.

To make some more analogies, it’s like playing chess.  Anyone can learn how the pieces legally move.  Playing a game where the strategy makes sense is the hard (and interesting) part.  You need to have a very clear idea of the semantics of the problem you’re trying to solve, then carefully implement those semantics.

Every VBScript statement has a meaning.  Understand what the meaning is.  Passing the right arguments in the right order will come with practice, but getting the meaning right requires thought.  You will eventually find that some programming languages have nice syntax and some have irritating syntax, but that it is largely irrelevant.  It doesn’t matter whether I’m writing a program in VBScript, C, Modula3 or Algol68 — all these languages have different syntaxes, but very similar semantics.  The semantics are the program.

You also need to understand and use abstraction.  High-level languages like VBScript already give you a huge amount of abstraction away from the underlying hardware and make it easy to do even more abstract things.

Beginner programmers often do not understand what abstraction is.  Here’s a silly example.  Suppose you needed for some reason to compute 1 + 2 + 3 + .. + n for some integer n.  You could write a program like this:

n = InputBox(“Enter an integer”)

Sum = 0
For i = 1 To n
      Sum = Sum + i
Next

MsgBox Sum

Now suppose you wanted to do this calculation many times.  You could replicate the middle four lines over and over again in your program, or you could abstract the lines into a named routine:

Function Sum(n)
      Sum = 0
      For i = 1 To n
            Sum = Sum + i
      Next
End Function

n = InputBox(“Enter an integer”)
MsgBox Sum(n)

That is convenient — you can write up routines that make your code look cleaner because you have less duplication.  But convenience is not the real power of abstraction.  The power of abstraction is that the implementation is now irrelevant to the caller.  One day you realize that your sum function is inefficient, and you can use Gauss’s formula instead.  You throw away your old implementation and replace it with the much faster:

Function Sum(n)
      Sum = n * (n + 1) / 2
End Function

The code which calls the function doesn’t need to be changed.  If you had not abstracted this operation away, you’d have to change all the places in your code that used the old algorithm.

A study of the history of programming languages reveals that we’ve been moving steadily towards languages which support more and more powerful abstractions.  Machine language abstracts the electrical signals in the machine, allowing you to program with numbers.  Assembly language abstracts the numbers into instructions.  C abstracts the instructions into higher concepts like variables, functions and loops.  C++ abstracts even farther by allowing variables to refer to classes which contain both data and functions that act on the data.  XAML abstracts away the notion of a class by providing a declarative syntax for object relationships.

To sum up, Eric’s advice for novice programmers is:

  • Don’t be a cargo cultist — understand the meaning and purpose of every line of code before you try to change it.
  • Understand abstraction, and use it appropriately.

The rest is just practice.

Comments (48)

  1. Mike Shaffer says:

    Wow! In the long line of highly decent blogs, this has to be the top of the top. Well done Eric! I’ve printed (gasp) a copy out and will refer to this many times in the future. To address your query: I fall into the "experienced" programmer category. I program all day (in JScript no less!) and teach Jr/Sr High kids programming at night. This is exactly what I stress: Syntax is the easy part, it’s just memorizing "ie before e except after c" type of stuff. While VB is syntactically different than C#, it has the same basic foundations, and it’s those that are critical for learning. I’ve known some brilliant (fill in the language) coders in my time and many of them knew their language dejour in and out. But how to program…that was a different question. Thank you for the great analogy (Cargo Cult programming) and your Function Sum example is exactly what I’ve been using to demo implementation for a number of years. Well done!

  2. Mike says:

    My next piece of advice would be: Learn to use your debugger.

    I see it so often on message boards where a novice’s code isn’t working right, and they have just run the code, looked at the output (which was wrong), and then been stuck on what to do next. Often it’s something simple like a reversed boolean test, or an uninitialized variable, stuff that would be immediately evident if they stepped through the code.

    After that, I’d reiterate the rule: Just because your code compiles, doesn’t mean the code is correct. If it compiles, but doesn’t produce the right output, then don’t just throw up your hands and say "what’s wrong? it compiles!" Use the debugger. If it compiles, that just means you matched up all the {} or begin/end pairs. It says nothing about the semantics.

  3. DJ: from Alabama says:

    You are so cool! I love this story!

  4. Jack Shainsky says:

    I agree with Mike about importance of knowing your debugger, but after learning to use the debugger many people, even very experienced, remain closely tied to it. When they find a bug, they immediately begin to debug it, passing lots of code just in order to get to the _suspected_ piece and find that the problem isn’t there. The continue the routine until they finally find the bug, then they fix it and feel proud of themselves.

    I think that the very important thing for novices (and not only for them) is: Learn to NOT use your debugger until you _absolutely_ have to do so. Reading logs (and putting them in necessary places in first place) and passing code in mind can give you enough information to fix the bug, and it will take tenth of time you would need to find the problem with debugger.

  5. Robert Sayre says:

    I have been programming since 1997, when I took Introduction to Computer Graphics in art school. I thought it would be Macromedia Director type stuff, and the school’s course catalog was a little confusing. It turned out that it was a semester of C followed by a semester of C with OpenGL. I ended up liking it more than art courses.

    I think I can still remember what that level of knowledge was like, and I wish it had been made clearer to me just how hard it is to be a good programmer. I think most novice courses lie about this difficulty, in order not to intimidate students.

    I wish someone had showed me this:

    Teach Yourself Programming in Ten Years

    http://norvig.com/21-days.html

  6. Erik says:

    Truthfully, I probably still fall into the "novice" category. I am less than a year out of college though, so that is acceptable in my eyes.

    I do see myself moving more towards "advanced" as the months pass. I spend less and less time throwing random print, debug, etc. statements into code trying to find the problem and more and more time analyzing the code trying to see the problem(s) before gutting it.

    Please continue these types of essays, I have been reading for a few weeks and you always have something interesting (even if some of it flies above my head the first read through).

  7. Michael Gardner says:

    As a professional programmer for many years (25 at least), I’ve been thru it all and back and all again. Your cargo story is right on the money. It all has to do with people’s abilities to abstract information and operational levels. I use several different analogies when telling what/how programming works. First of all, scripting IS programming. All the ego-driven drivel about compiled vs. scripted, whatever is just that…ego. It’s all code, it all AT SOME LEVEL controls a system in some fashion. Hell, I use Excel/Access macros every chance I get rather than writing VB(A), tho’ of course, I’d rather write Java for everything…but’s that’s just dumb. Learn programming well and you’ve learned it all. Languages just become a means of expression. Again, it’s the appropriate level of abstraction that counts first of all.

    Second…I’ve started seeing in college C.S. tracks an Introduction to Programming Concepts (or something or other) where the intent of the course is not only syntax but BEGINNING semantics i.e. how to convert a problem (usually in some form of pseudocode) into a structured block of syntactically correct code…the key here is that Structured Programming as defined in the late 70s – early 80s was NOT supplanted by object-oriented programming or by scripting but complemented it and made it easier to develop an application. Take that intro class. Get your company to pay for it.

    Next point: OOP definitely the landscape tho. I’ve seen the REVERSE problem: older programmers who’ve said to me, "Geez, you can put code just about anywhere!" which to old structured programmers appears to be some form of chaos…well, yes you can in a badly designed/written piece of OOP (whether it is script or not…everything has SOME sort of OO going on, regardless of how pure it is). Design becomes very critical with OO-based languages. ASP is a perfect example of that.

    Debuggers are important tools. Learn to use them, but learn to read/write code first. As another person said: you should know EXACTLY what every line of code does in terms of overall program behaviour. In fact, if you can’t tell why that code is THERE at THAT place in the code, you shouldn’t be touching it because you don’t know enough.

    So in many respects it’s both harder AND easier…but you have to make the first steps PAST the syntax and begin to understand the appropriate placement of lines of code into blocks that perform the FUNCTION you want.

    As I said to my son who was taking his first C++/programming class: it’s like an onion, you keep peeling layers to find more layers. And if you REALLY want to master it, you’ll keep peeling in spite of the tears.

    Another suggestion: READ CODE, DON’T WRITE IT!!! Everyone thinks the key to programming success is writing code. The secret to programming success is READING CODE, other people’s code, on all kinds of topics/functions/apps, etc…you learn from reading, not writing (this is the secret of the whole open source…honestly we don’t snitch code, we "borrow" ideas that we’ve read!)

    Programming in 10 years….that’s the right idea!

  8. Wiennat says:

    My math teacher always said "It’s not important to get the right answer. But it’s important to get the right solution". Is it the same?

  9. Eric Lippert says:

    Getting the right answer _is_ important. Very, very important!

    But I think I see what your teacher is getting at. If I could paraphrase, I’d say that what is important is that your logic be sound. If your logic is sound then you can’t help but get the right answer. Just having the right answer is not enough — you might have guessed, or cheated, or whatever. You must have a reliable method, because that’s what mathematics is all about — coming up with methods that work EVERY TIME.

    I think your teacher is making the same point that I’m making. It’s not enough to have an arithmetic technique that does the right thing. The whole point of learning mathematics is to UNDERSTAND why those techniques work, and to PROVE that they are reliable.

  10. mike says:

    Another fabulous adventure in blog-reading. 🙂 A couple of thoughts. Thought one: while no one can argue that the philosophy of "understand how it works" is right, there is also the type of programmer (let’s call them the programmer-by-accident, occasionally thought of as "Mort") who actually doesn’t want to become an advanced programmer; he or she has some job that needs doing. Perhaps we can call them the second-job programmer, whose first job is managing or crunching numbers or whatever. This type of programmer is probably the classic cut-n-paste programmer — "well, I found this on the Web, and it seems to work." (A nod here to Scott Hanselman’s recent comment on this practice.) Such a programmer can in fact learn in a slightly way from what you’re describing. Ok, here’s a piece of code that I got working. It’s not quite what I need. What if I change this? Aha. What if I change this? Oops, better change that back. Etc. You can learn a lot by dismantling things. (And by pestering your colleagues and fellow listserv members to help you get it working.)

    Thought two, which I believe is complementary to your thesis: programming language, eh. It’s _object models_ that need to be mastered. I have a friend who’s learning ASP.NET. He’s competent enough in C#; when he calls me for help, it’s because he’s flummoxed by ASP.NET or ADO.NET. Is that semantics, too?

  11. Wonderfull post. I will use the term "Cargo Cult Programmer" from now on. This is indeed something I had discovered – to a devastating hlarge percntage within the amount of professional developers (I would say 20% to 25% of all developers out there earning money are nothing more than cargo cultists), but to a much larger degree in training courses which I occasionally hold to keep my speaking skills intact.

    Here (in germany) there were a lot of retrainig programs (making an unemployment cook a programmer in 12 months), and heck – everyone I met there (with the unusual occasional exception) has been a cargu cultist, including the majority of the trainers.

    I would say we live in a bad time for IT – the problem is that IT professionals as a group do not show enough of a honour attitude. Wherever you go there is this "cheap and fast is best" thing, and most people can go through years of training and working without ever meeting any one developer who knows what he does. Cargo Cult grows on an understandig like "it is good enough to just get it working somehow".

    @mike:

    nice point. But copy/paste is not what makes a programme ra cargo cultist. I do copy/paste regularly, too – it is simpler. YOu want to get something special done, dig up a KB article with some sample source. Why not copy it in and use it as a starter point 🙂 This is not cargu cult. Cargo cult is copy / paste / clueless. THe clueless part is important here. These are the guys you meet regularly in newsgroups and message boards that have copy/pasted and now ask for the trivial modification (I foudn the sample to create a directory named "foo" – what do I need to change to create a drectory named "bar"), often saying "I get an error" (gues s which) and then being helpfull by posting some source (and the directory is created in a winform app in a dialog, and I have included the 200 pages of code – the form also handles some database interaction – so that you know what I am doing), often with a nice request for help (you can clean up the code if you want). THIS is a cargu cultist 🙂

    ::when he calls me for help, it’s because he’s

    ::flummoxed by ASP.NET or ADO.NET. Is that

    ::semantics, too?

    No, this is him being too lazy to read the documentation. Especially ADO.NET is trivial as object model. ASP.NET is not really complex wither, but it is always an advantage to know what happens there – means: for ADO.NET it really helps a lot to know what HTML is and how the web works (it is interesting how many people working with ASP.NET have no clue about how http does work).

  12. Rich C says:

    Perfect analogy. I’m neither novice or expert, but your article really spoke to me.

    I was once learning Pascal (long time ago) and at the same time assisted Fortran students – I didn’t know Fortran. For the most part, it’s just a different syntax for accomplishing a task, though.

  13. Centaur says:

    What you need to learn first is learn to learn. I sometimes say, “A programmer who does not learn is either sleeping or dead. And I’m not actually sure about sleeping”.

    Many people ask for my help in programming matters. By “help” they usually mean “do for me”. They don’t care about learning, they want a solution, to show it to their instructor and get a passing grade. I then try to see if they can solve it on their own with a little pointing in the right direction, and if they can’t and actively resist, well, too bad for them. “Sorry, can’t help you.”

  14. Tarjei T. Jensen says:

    Cargo cult programming seems to be what I call parroting. That is people who only can answer back with what you have already told them. They can’t use what they have learnt in one programming language with another programming language.

    A lot of understanding of programming comes from being thaught with the right tools. Getting the right error messages. It also helps that the teaching language reads reasonably well. Verbosity is a plus.

    That is why C/C++/Java are not good starter languages. It is the reason Ada/Eiffel/Delphi are good starting points. The latter programming languages are more productive. If you don’t belive that, it is because you are unaware of how you are spending your time: You forgot to factor in the time it took you to get the program to actually work.

    Countrary to popular belief, the effectiveness of a programming language is not inverse of the number of characters a program consists of. That is probably why Cobol is alive and well.

    greetings,

  15. Jonathan Perret says:

    How about starting education with the documentation ?

    As an experienced programmer who occasionnally has to (heck, wants to!) use scripting languages, the (human) language in which the scripting documentation is written consistently gets in my way when I’m looking for information on the _semantics_ of a scriptable API, by repeatedly addressing the syntax, probably in the assumption that a) scripting languages are for beginners and b) beginners tend to forget about the syntax.

    I’m probably not making myself clear at this point, so let’s look at an example : the documentation for Microsoft’s scripting runtime library. Just about any reference page will do, like for example http://msdn.microsoft.com/library/en-us/script56/html/jsmthCopy.asp

    (the Copy method of the FileSystemObject).

    The method is presented like this :

    object.Copy( destination[, overwrite] );

    Below, we find that "object" is an "argument" (in a very technical sense, I suppose it is) which is (must be ?) "Always the name of a File or Folder object."

    Now is that syntax or semantics ? Does that mean I can’t write :

    (new FileSystemObject()).Copy(…)

    Or :

    functionReturningAnFSO().Copy(…)

    ?

    I hope my point is clear now. I could go on for hours – in just this method’s description there are several more problems such as not specifying the type of the "destination" argument (is it OK if I pass a Folder object, then ? Why not ?), but there is also the Internet SDK that insists on segregating ‘properties’ from ‘objects’ and ‘collections’, hopelessly confusing the cargo cultist into thinking they are funamentally different to the parent object.

    Don’t you think improving the documentation should be the first priority ?

    Cheers,

    –Jonathan

  16. mike says:

    Thomas and Jonathan, interesting points. Thomas first:

    >No, this is him being too lazy to read the

    >documentation. Especially ADO.NET is trivial

    >as object model. ASP.NET is not really

    >complex wither, but it is always an advantage

    >to know what happens there – means: for

    >ADO.NET it really helps a lot to know what

    >HTML is and how the web works (it is

    >interesting how many people working with

    >ASP.NET have no clue about how http does

    >work).

    All true (well, the "trivial" part is up for discussion). But I think you are illustrating an interesting issue: in order to be effective with ASP.NET, you need to know HTML. But suppose I don’t really know HTML that well, but nonetheless I need to get this one thing working right now. In theory, I could go away and study HTML, etc. until I felt I really understood what’s going on. But I don’t have time! I need to get this thing done right now!

    ASP.NET + HTML is probably not the best example, but perhaps XML + XSLT is. You could spend months studying XSLT before you felt like you "understood" it, but in fact, you can get stuff working pretty rapidly by copy-and-tweak. Which leads to the kind of situation I was describing. Note that the person who is doing this IS learning XSLT, just on an incremental and JIT basis. It simply takes time to absorb all of these things, and it’s not always (in fact, rarely) realistic to spend a long time studying the background on something before you start working with it.

    Thomas: interesting point. I can tell you that different types of documentation is written in different ways. The example you cite is from a reference topic (description of a class or member). That type of documentation is traditionally written tersely because reference docs are to some extent written for the already-experienced programmer who (it is theorized) does not need a bunch of background information repeated when all they wanted was the syntax of method such-and-this. (You’ll note that almost all reference material for all languages — JScript, C#, Java, T-SQL — follows this philosophy.) The idea is that if you don’t already know what an object is or whatever, you should be able to read about that in a conceptual topic, which in previous years would have been the "programmer’s guide." Good reference material will cross-reference to the conceptual documentation that provides the background required to understand the reference material. A weak analogy, I suppose, is a dictionary. It defines the words, but it doesn’t tell you each time what a noun or adjective or verb is; it’s assumed you already know that or can look it up elsewhere.

    More germane to this discussion, reference documentation is not meant to be tutorial in nature. It’s not where a beginner should be learning the semantics of the language. (Again, is the theory.) To be clear, I think it’s possible to write good and bad reference documentation. I favor verbosity myself (obviously).

  17. Michael Gardner says:

    We’ve each restated many of the same things using other helpful analogies here about the educational process of becoming a programmer.

    Regarding "Mort" or the middle-level "experimenter/copy & paster" coder…certainly this person exists: the fact is that there is the spectrum from the Parroting Cargo/Cult Novice Coder to the Polished Pro-grammer (who can steal just about any code and know full well what to expect of it). But tread very carefully in the cut & paste world.

    It’s as if there are different "heuristic" as well as abstraction levels. A "Mort" will C&P and test/break the code. A Polished Pro knows this and anticipates doing this BUT is cautious in the "breaking"; "constants that vary; the variables that don’t" can cause all hell to break loose even in the experimental phase.

    Someone else said in a previous response "…learn to learn…": that is the key. Learning music, learning airplane maintenance, learning whatever…if you don’t start from principles (abstraction), you’re a Cargo Cult programmer. But learning by doing (heuristics) is part of the learning process…i.e. seeing principles AT WORK.

  18. Programming and Personas

  19. Jonathan Perret says:

    mike, I think you meant to reply to my post in the 2nd part of your post.

    Judging by your comment "[reference documentation] is traditionally written tersely" it seems I totally failed to illustrate my point with that example.

    What I wanted to say is that I find that sample topic to have, if anything, too much information. Too much syntactic garbage : what’s the benefit of repeating on every reference page that to call a method you need to put a dot to the right of an object value ? Particularly when instead of well-defined words such as ‘instance’, they use ‘object name’ which is totally wrong and ill-defined.

    Once you get rid of that garbage, there is ample room to specify the details that really make the difference such as the data types of the arguments.

    And I disagree that reference material for all languages (I was actually talking about API, not language, references but I see what you mean) is written in the same way. Look at the CLR or PSDK docs, which essentially target non-scripting programmers; while they have their own share of problems, they have at least the advantage of actually being _reference_ texts, in that they precisely and concisely define a specification for an API. The scripting runtime docs just leave too much to the imagination.

    Hoping this is clearer…

  20. I’m in your "expert" category, having been programming for 20 years and done everything from device drivers to business systems for Fortune 50 companies. On the side, I teach (C & Perl) and write books (Perl & Web), and I’d teach a lot more if I had the time.

    I think you’re right on the mark. When teaching (and writing) I’ve had tremendous success teaching in 4 steps to beginner and intermediate programmers.

    First, give an overview of a small problem (in plain English) with lots of handwaving and other gestures — but no code. Walk through how it’d be solved.

    Second, give a piece of code to solve the problem directly. Apply the walkthrough to the code, explaining at each step what’s going on (but not syntax!). If there’s a danger of Cargo-Culting, it’s here. What I do to combat this is vary the problem a bit, and show them ever-changing variations on the same theme. This re-inforces how the code is solving the problem, rather than what’s being solved.

    Third, either as an exercise or with a class solve a similar-but-not-quite-a-trivial-variation of the problem. That further re-inforces the problem solving pattern.

    Fourth (in class), take wrong examples that students have worked on or given up on and work them out in front of the group using classic beginner debugging techniques: lots of "print" statements, divide and conquer, simplification by hard-wiring logic, etc..

    The thing that makes some of this difficult, is that some languages are more prone to "idiomatic" structures than others. The idioms have to be taught one way or the other, but I’d rather re-construct the idioms by hand and have the student (or reader) pleasantly surprised later to find that their coding technique is canon.

  21. Shawn says:

    I am a novice ‘Programmer’..Having decided to formally take a programming course for the first time. (Its C++ by the way..)

    <snip>

    # Don’t be a cargo cultist — understand the meaning and purpose of every line of code before you try to change it.

    # Understand abstraction, and use it appropriately.

    <End snip>

    This article applies to many facets of life/work…

    It could as easily be restated..

    Dont be a CC – understand every LAW before you try to change it

    Dont be a CC – understand every <Medical Procedure on the Foot>|<type of carburator to put on a edlebrock>|<flavor of ice cream avail on the market>|….before you try to change it.

    This blog speaks to professionalism and the apparent lack of professionalism in the programming /IT world as perceived by the non-programming/IT world.

    Businesses look for and expect results – cut-n-paste with the resultant hope it works – is accepted/required. Produce code or loose your position.

    Is the cargo-cultist approach the reason beginning / basic programming is be ‘global-sourced’ / ‘ out-sourced’ / ‘off-shored’??

    Which came first – the chicken or the egg?

    The business world apparently perceives a lack of value in a professional programmer ( for the most part the companies want websiteX or embedded productY to just work ).

    Mind you I am not against free-trade – Micro|Macro Econ should be required before allowing anyone to graduate from University.

    Back to the blog – I perceive that a some amount of the population desires to know WHY – or HOW things work. A fair group however do not.

    Excellent articel and definitely thought provoking. It is rare that I read anything where I am required to look up a definition of a word prior to finishing the first paragraph.

    (prolix – tediously prolonged or tending to speak or write at great length).

    Thank you – I have just added a word to my vacabulary ( as I am guilty of it as well)

    -Cheers

  22. Eric Lippert says:

    Might as well add "sesquipedalian" while you’re at it.

    There’s no doubt that poor understanding of the nature of software engineering is behind many poor management decisions — but it is also the case that software engineering is not a mature enough discipline yet to consistently give good data to management!

    That’s a huge topic in itself, which I know little about.

  23. Peter Evans says:

    I think your entry has good merits for the novices in the audience. Definitely, at some point assertion of random code mimicry will halt progress toward some new concievable goal. Much like trying to find harmony on a piano by random sounding of keys.

    While I still consider myself a intermediate programmer. IMO programming (learning to program) is just like trying to write about unkown subjects or speak a foreign language. It boils down to a knowledge transfer problem. Of course knowledge transfer is a deep abstraction in and of itself and it requires many data exchanges between the communicating parties to begin a meaningful discussion and is the subject here.

    What novice programmers need is a primer that enables reflexive understanding of what programming is so that they can keep evolving their understanding of programming.

    By primer I mean something like the concept of numbers and a set of axioms in math; or an alphabet and the parts of speech in english; or notes and modality in music. I know of no such great such reference manual for programming. Maybe the tao "The art of programming", but I think that is rather complicated by the mathematical notation. And besides novices want practicle advise not conceptual advise.

    But beyond this primer, what also is needed is a good short book on how to solve computer problems similar to G. Polya’s "How to solve it" for math problems.

    The book "Design patterns" comes close here, but again it bogs down any novice with notation and syntax yet again. The book "Refactoring" might also be good but I believe has an even greater symbolic requirement.

    Alas this goal may be unreachable, because of the complexities associated with programming both mathematical and linguistical.

    I think what is needed is something like a very concise compendium of of "Design patterns", "Refactoring", "Feature Modeling", "Aspect Modeling" and excerpts of "The ‘New’ Touring omnibus", combined with a pratical backgrounder on syntax, grammar, and sematics in order to boot strap a novice. But that is too much to ask any new programmer!

    So… are we doomed to suffer the wrath of cargo culture programmers indefinitely? I think not. This is just a normal step in the advancement of how we communicate with each other. Computers are just a rich communication medium which allow us to interact and express ourselves. Like natural languages, programming is just a new and very young dialect yet to be codified into a succinct and intuitive manner like music, math or language.

  24. Excellent post, Eric! 🙂

    Just thought I’d mention that we’ve got a good thread about learning to program going on at the moment:

    http://www.sitepoint.com/forums/showthread.php?t=156802

  25. Greg Fox says:

    As a "novice" web developer for 4+ years, I am VERY guilty of cargo cult programming. Even after picking my way through the bits of code I have either written, or cut-and-pasted, I don’t feel any closer to actually understanding the semantics.

    In order to improve my ability to semanticize(?), I have searched high and low for any information regarding how do you break a problem down and apply the tools any given language has, to solve the problem? When do you know the problem can’t be broken down any farther? It’s that part that I need help with, but I think I am in the minority.

    And quite frankly, the example you gave regarding abstraction, is SO far over my head, I’m still spinning, but I will keep coming back, because it was a great article. Thanks.

  26. msp says:

    i’ve been a developer for 15 years coding all kinds of projects and i’ve gotta say there’s a fine line between abstraction and cargo cult programming.

    i’m guilty of ccp. often times you get a snippet, you know what’s it’s supposed to do, you can look at it and have a rough idea of what it’s doing where, but in reality, the deepest details of WHAT’S REALLY GOING ON ™, are totally magical.

    anybody that’s worked as a knowledgeable novice on a crunch project with high level APIs for occasionally deep and dark stuff like say heavy direct3d programming probably knows where i’m coming from. it’s due in week. nobody has a clue even on the newsgroups. "you’re trying to do what?!?!?" but nevertheless, you’re the crazy with the duct tape and debug stream waiting to see what happens.

    in the realm of rapid development, where you’re stuck with alpha or beta (if you’re lucky) APIs to produce the latest greatest feature, waving the mystical palm frawns and offering sacrifices is sometimes the best and only chance you have.

    yet some people might calmly think of that as black boxing. widget A draws a perfect circle and whatever it does underneath the hood is it’s business. to channel arthur c clarke, if you’re comfortable with it, it’s technology, if you’re not, it’s magic.

    after a generation or two of cargo culting, when nothing keeps showing up, you’d have to realize that what you’re doing isn’t working. it’s magic is gone. if you do actually have success, you’ll repeat that magic, but like a witch doctor or grandma, that home remedy really turns into technology.

    frankly, i wouldn’t recommend staring into the dark of ccp for new programmers. leave that to the old, fringe weirdo on the edge of town. he or she is mostly useless except for the most dire of situations. (aka wrangling windows security hassles and legacy programming insanity vs immediate deadlines.)

    m.

  27. Kevin Daly says:

    This reminds me a lot of what I have tended to refer to in the past as Voodoo Programming, especially in maintenance situations:]

    [Me]"No, don’t leave it at that"

    [Dev#2] "Why not, I don’t know why but it seems to work"

    [Me] "Because it doesn’t make any bloody sense! Find out what’s *really* going on".

    As for the vexed problem of the care and feeding of new programmers, there is a dirty little secret that nobody (especially in CS departments) wants to admit in public: you can take many perfectly normal, intelligent people and teach them how to write syntactically correct code, you can teach them design patterns up the wazoo so they can write code that handles many commonly occurring situations, but only a certain proportion of those people will ever really "get it" (the rest will generally end up as their managers and tell the others how to do what they never really understood themselves). That is because programming is only partly a matter of education, while it is also very importantly a matter of aptitude.

    We pretend otherwise because

    a) It helps justify academic CS budgets

    b) It suits corporate executives who think programming can and should be reduced to a factory activity.

  28. Jozo Bozo says:

    Kevin, you are right. There are different types of people in the world. I’ve met super-intelligent guys (actually had them tested at interivews) who simply weren’t able to think in abstract terms. This is all well and normal; they are not stupid for that, just differently wired.

    Furthermore some people learn by induction, other by deduction — someone needs to see 1.000 examples before realizing how to go about it, while others first have to understand the big picture first.

    <provocation>

    That being said, Eric isn’t your post a cargo cult type post: talking of mechanics of teaching the trade, but not understanding the big picture of how people learn?

    </provocation>

  29. Eric Lippert says:

    > Eric isn’t your post a cargo cult type post: talking of mechanics of teaching the trade, but not understanding the big picture of how people learn?

    Why do you think that? The post isn’t about how people learn at all, it’s about a pitfall that people fall into: confusing the form of a program with its content.

    I could just as easily have written the piece in the context of writing sonnets, not programs. Yes, you have to learn what the structure of a sonnet is before you can write one, but just because you have the structure right doesn’t mean you’ve written a GOOD sonnet, any more than just because you’ve moved the pieces legally means you can win at chess.

    That people have different learning styles is undoubtedly true, but I fail to see the relevance to this particular post.

  30. Eric Lippert says:

    > yet some people might calmly think of that as black boxing

    Clearly there is a spectrum of behaviour here.

    On the one end are the gurus who understand the intended and actual behaviour of every usage of a particular programming construct — the people who see through multiple layers of abstraction down to some underlying substrate like electrons, the hardware, the operating system, whatever is "basic" for a particular application.

    On the other end are the cargo cult programmers who don’t understand the code except that they know that right now, it does what they want, sorta.

    The "black boxers" are somewhere in between. Of course everyone is a black box programmer at one time or another. I don’t know how exactly GDI draws those circles, and nor do I care. I’m a GDI black boxer.

    But when I call a GDI method, you’d better believe that I know WHY I’m calling it, what every argument to the method means, what resources it consumes that need to be cleaned up later, all that stuff.

    In short: gurus understand how it works, whatit does and why it was designed that way. Black box programmers understand why they’re using it and how to use it and what it does, but not how it works. Cargo Cult Programmers sort of understand what it does but not how it works or why it works — they just know that it works.

    When I’m interviewing people, I’m looking at all those things. ("When would you use a delegate? What are delegates for? If delegates didn’t exist, how would you implement them?" — people who know what a delegate is for but can’t come up with how they might work are people who haven’t seen past the abstraction.)

  31. Reen says:

    I call cargo cultists "Mouse-ka-teers". Good spellers, every one (M-I-C… lol) It works well for copying code, but isn’t necessarily a good thing to do when defining your own variables. One of the first things I learned as a programmer; spell it wrong so it doesn’t match a reserved word. That was 30 years ago when a 64bit mainframe filled a big ice box of a room and you got your exercise lugging around cartons of punched cards.

    These days, I consider myself both an expert AND a novice when it comes to programming. In the micro world, things change so fast, I’ll always be a beginner. Just when you think you’ve mastered some language, you wake up the next morning!

    But you’re right about the semantics of programming. Once you understand the logic, using another language is just a matter of substitution. Sort of… At least for those primarily driven by innovation and creativity. Those truly fascinated by the art of programming. Us eggheads.

    I really enjoy reading your "blogs". I’ve read them all. You take such a lighthearted approach, it’s easy to understand, and quite entertaining. So… get to work! Lets have another fabulous adventure!!!!

  32. Tim says:

    Debugging is at least twice as hard as programming. If your code is as clever as you can possibly make it, then by definition you’re not smart enough to debug it.

    – Brian Kernighan

  33. XML-BLOG says:

    I was reading this post by Eric Lippert when I stumbled the statement that Software Engineering is an immature discipline. This triggered the stream of consciousness below: Software Engineering is an immature discipline. Is it? What does that mean? Does it mean software engineering is a new discipline? How long…

  34. Another 1 Bytes the Dust says:

    Hey, it makes sense to me but I don’t really know if I am a novice programmer or an experienced programmer. I do know that my computer isn’t made out of straw and I certainly don’t type away on a keyboard constructed out of wood, so I guess I’m not a ‘cargo cult programmer’ either.

    I’m a self taught programmer with no qualifications and that’s how I like it, it’s fun and it keeps the coffee industry going. I think a more appropriate term for ‘cargo cult programmers’ would be ‘google cult programmers’

    Good read, the cargo cult was even more interesting after I read about them.

  35. My colleague Mike , in a comment in yesterday’s entry , mentions "Mort". Who is this Mort guy? At Microsoft,

  36. Zachary G. Jensen says:

    I agree with your sentiments, with an anecdote to boot.

    My first exposure to programming was with Microsoft QBasic. I was about 8, and had trolled Prodigy for scripts, and came across many neat looking games. I dissected them with a fine-tooth comb for several years, and got pretty good at hacking them up, adding additional menus, moving graphics around, etc. (I was never a good artist, so I left the DATA instructions alone…).

    I couldn’t figure out why the numbers went from 0-F. I had no exposure to hex. With some guesswork, I usually managed to get what I was looking for. I didn’t find out until college what a hexadecimal number was. I ran into similar problems with non-square matrix division, among other things. Throughout, I understood the syntax quite easily (even polymorphism, function overriding, and streams came simply–possibly I had a good teacher). However, without understanding the external concepts I was dealing with, I was lost.

    I’m a pretty well-versed programmer. No genius, but not a "Mort", as your term seems to be. I don’t run into problems with languages as much as various toolkits, particularly (and unfortunately) Microsoft’s .Net Framework. Microsoft could well take your advice when it comes to MSDN, in my opinion. I’ve found tons of information on how .Net works, but not much on the concepts that implement it: The reasons, design decisions, and descriptions that come with a thorough understanding, presented in a clear manner.

    In effect, I run into problems pretty regularly on my projects. More than once, I’ve implemented something that .Net already provides, but I never knew about.

    My point in this, is that "cargo cult" programming, in my experience, is not merely about understanding the semantics of your language, but a cohesive picture of the tools available to you–the semantics of your toolkit, so to speak.

  37. Two additional quick notes about books: I am also pleased to announce the availability of the C# 3.0

  38. zack says:

    Hi, I have this vbscript but I want it to make me a sandwich. Can you write the code for me? It should know what I like on my sandwiches. Thanks in advance.

  39. Mort says:

    Hi, I have this javascript but I want it to make me a milkshake. Can you write the code for me? It should know what I like on my sandwiches. Thanks in advance.

  40. Paddy3118 says:

    “They tend to proceed by making random changes, testing, and changing again until they manage to come up with something that works. “

    Please tell me that you are exaggerating to make your point.

    Of course I’m not. — Eric

    It has been decades since I first taught myself to program and people like the above could not be called programmers of any sort. Maybe if exposed to a new type of programming language, I might peck around whilst learning it, but I would not consider myself able to program in it at that stage. Maybe you are being too kind to this type of person, to the detriment of all programmers, by tacking on the word programmers after cargo cult?

    There are lots of people who are paid to program computers who have no formal background in computer science and whose training is in the business domain, not in the programming language domain. Whether they meet your personal standards of what makes a Real Programmer is hardly relevant; they exist and their organizations buy tools from us.

    And even professional programmers — even compiler writers — sometimes act like cargo cult programmers. Every time I have to write a WMI script, that’s how I proceed. I do some web searches, find some samples, and hammer on them until they work. I’m totally a cargo cult WMI programmer, because I write WMI scripts two or three times a year, tops. If I did it every day, I’d invest in understanding how the thing actually works, but I don’t. — Eric

    – Paddy.

  41. Clapton says:

    Hey Eric, you’ve written (in the comment above) that people don’t need to have formal background or degrees to get a programming job. Is this true in organizations like Microsoft?

  42. Mr. Novice says:

    Excellent post Eric. I tried to teach myself programming, and many of the resources I found trained me to be a cargo cult programmer. By now I’ve developed a better approach and it has been working much better so far. The three things that have helped me the most so far are the following:

    1. Try to understand how the code works and why, not just ‘make it work’. That is, try not to be a cargo cult programmer.

    2. Read programming blogs and code that others have written.

    3. Try to develop my own applications to learn from, and get feedback from more experienced programmers. Then, refactor my code and make it better.

  43. Sandy says:

    Thanks Eric, for this blog. You had put it 8 year back but its true right now also.