What’s with the "programming test"!?


I was having a chat to a mate of mine this morning, who had just returned from a job interview where they had asked him to conduct a programming test. I asked him what the test entailed, and what more importantly was the point of said test. His answer flared up my career colon again, thus the resulting post.

So, the programming test was how to implement a file copy operation. I asked him if the interview process also included an Object Oriented design test, or an algorithms test…nope, just the programming test. Riiiighhht!

Why does this flare up my CC? Well, this goes towards my “Why I’m not surprised that software projects fail” manifesto.

I mean, firstly, I don’t store all the API and SDK objects, functions, method signatures, whatever in my head, that’s what the programmers guide is for. But more importantly, knowing how to implement a file copy function without an understanding of why you would implement a file copy function is where the red ink is set free. I’m appalled that companies still see the “programming test” as a viable way to vet talent. In fact, unless the job you’re going for involves writing software that directly talks to the file system as part of its core capabilities, then the correct response from a candidate should be, “Um, I’d just use a framework”. That’s why development communities far and wide have frameworks; it’s so you don’t have to give a toss about System.IO.File blah blah blah during a job interview!

And then there is the issue of why would someone care if I knew how to write a file copy function, ahead of what the difference between an interface and an abstract class was. Or when would I use encapsulation? Or in my design, how would I choose between polymorphism or inheritance? Or what pattern would I recommend for dealing with a general or specific problem?

It wreaks of a bigger problem, where organisations are focusing on the low level technical capabilities of people, rather than the overall software engineering capacity of people. The reality is, and I assert this to the peanut gallery, that we have people within organisations, such as Directors, Managers, generally non-Software Engineering folk, trying to ascertain whether an engineer is fit for the job. And what’s more, they are focusing on the most basic of tasks as the process for evaluation. Duh, can you program hello world? Duh, can you write me a form with a button? Jheesh, I mean, who cares if you know you’re way around an IDE or a language? Just because I can carve the Sunday roast doesn’t mean I should be applying for surgical positions at my closest hospital.

I’m really getting the feeling that the lunacy of the Software Engineering industry (and I use that term sparingly, as I don’t want to encompass the whole of IT, just the bit that involves designing, developing, deploying and maintaining software) is spiralling out of control. And that I will be forced to encounter more crazy capability and skills estimation tools such as programming tests, and interviews with recruiters (jeez, don’t even get me started on that…”Um, can you tell me what you know about the difference between .NET and Linux??” WTF!!? Um, one is a framework and the other is an OS!!! I should ask them if they can tell me the difference between a Toyota Hilux and the dark side of the moon!).

Am I completely off the dial here people, am I the only one who sees these things as not only completely crazy, but also diabolical! Or maybe I should just keep quiet, and save all my experiences for my own sitcom! Yeah, sitcom!


Comments (44)

  1. Me says:

    You are missing the point – the quality of the available talent pool is terrible.

    Requiring an understanding of how file copying can be implemented is an easy test to get the top 20% of candidates.

    Requiring an understanding of interfaces and polymorphism is a harder test to give, especially given that 99% of candidates will fail.

  2. Pieter says:

    I like this entry: Great read!

    Just one remark: Depending on the type (level/role/etc…) of person you’re interviewing it can be interesting to also ask low-level questions.

    (He can then always lead the company bbq ;-))

  3. davidlem says:

    Oh, trust me Me, I didn’t miss the point, it’s been poking my right butt cheek for the past 8 years! 😛

  4. akashra says:

    I got this so frequently while going through IBL (internship) interviews.  Frequently it was a sign of that typical snobbish, looking down at you attitude.  You get similar things with design patterns, they want to know specifics about certain patterns which proves nothing at all about you as a general developer on the larger scale.  By the forth of those interviews when I was going through them I started really taking a "who cares" attitude to many of the questions I got asked by some – so much so that I actually said so and spoke my mind, because I was getting sick of the pretentiousness.  Didn’t bother me because I have no desire to work in a company where I’ll be under such arrogant attitudes.  Similar thing with language specifics – it’s not the point!

  5. Luciano says:

    You seem to forget, that there are MANY programming job outside the PC area, where been able to write a filecopy function could mean programming the IO pin on a PIC processor in 2 K of ROM!!!

  6. AnkitDass says:

    Good one mate. Yeah it seems ridiculous in terms of interview questions they come up with. But obviously depending on the job profile of the person you’re interviewing it can be interesting to also ask low-level questions as well.

  7. Dave R. says:

    What annoys me more is opinionated idiots on blogs who can’t even bloody spell. It’s ‘reeks’ ffs, not ‘wreaks.’

  8. Anonymous says:

    I am a little confused.

    Was this question about opening one file and writing it’s contents to another (with appropriate locking, of course)?

    Or was it about writing kernel mode code, i.e. a proper FS implementation?

  9. KoolRaaga says:

    Nothing is far from what you have written.. Good THought.. Just add some more perpective.. You will get the answer.

  10. davkean says:

    David,

    Asking candidates to write code, quickly distinguishes between those who have simply studied for the interview and those who have actually written code.

    We (Microsoft), use these tests all the time – in fact of the 2 days and 10 interviews for my current job, over half of them were ‘programming tests’.

    Regards

    Dave

  11. PatriotB says:

    The first job I had out of college required me to take a written programming test–after "passing" the phone screen, they mailed a test to my public library where I took the test, and then upon passing that I was invited for in-person interviews.  The purpose of the test is to weed out people–the company would get over 20,000 applicants a year, of which it would hire a few hundred.  A test is an easy way to do that.  And as long as the test isn’t tedious or API/language-specific, there’s nothing wrong with using it.

  12. Rex says:

    I agree with the idea of a test itself, but it’s important that the programming test covers a large number of criteria (such as OOP, basic framework knowledge eg: .net, VCL, MFC etc).

    A test for a file-copy function is pointless and silly. But a test for the candidate to take a few hours and write a basic application which does something useful – that is a key difference.

    I’m an advocate of programming tests, and good candidates have to be well educated in all aspects of development – not just "file-copy operations"

    -x

  13. Garry Trinder says:

    Ahh yes. The question we should be asking all potential programmers is how to code a bubble-sort of million row array. That’ll sort the chaff from the wheat.

  14. Java Froth Mouth says:

    Perhaps are more revealing test would be to provide a problem with a partially working/designed solution, asking the candidate to complete/ehance/correct the solution.  

    This approach may then give the candidate the ability and freedom to be a little more creative than they might otherwise be, while also allowing the employer to assess their skills across a range of areas, ie design, coding style and general problem solving ability.

  15. David says:

    I’d just ask clarifying questions until he gave up or kicked me out :P. Check out an example: http://exold.com/article/stupid-interview-questions

  16. NL says:

    As someone from Microsoft pointed out, they use programming tests all the time.

    And now we know why so many glaringly obvious bugs and security holes keep showing up in their software.  This is probably why half their Win32 API is implemented in such a messed up way too.  They must hire technical writers the same way they do programmers, because their API documentation is almost as bad as the API implementation.

  17. mjs says:

    You are right. Often these tests are a used as substitute for judgment.

    That said…

    We use the programming test as a sanity check – does the guy/gal have the experience claimed.

    And our test includes an object hierarchy and asks the canidate to predict the execution result. And it asks the canidate to discuss the issues when given object foo is moved to a multi-threaded environment. So you can get tomore in depth question – you just can’t expect stellar performance. Certain tasks including great coding often don’t happen in a given hour

  18. Marko says:

    I’ve interviewed quite my share of developers and since I’ve just started a new company, I suspect I’m not done yet.

    At a loss when I hired my first employee over 8 years ago I was like what you described; a freshly appointed team leader in a company experiencing over 500% per year growth. Yes I’ve made those mistakes. So over the years I’ve learnt:

    It isn’t always about technical skills. It is just as important how the person fits in with the team. How does he think, how does he go about finding and implementing solutions? How does he communicate that to his peers? How does he feel about being second guessed and how does he deal with that? And yes, I want to know about technical skills and passions too.

    So what I found works really great is that after I talked to the candidate a bit and have an inkling that it might be a fit I invite them to join the team for a day.

    Who can root out nit-wits better then me? The team can. Who can better judge if this person’s talents and skills will be an asset to the team? Who can better judge if they’ll be able to spend half their waking day in close collaboration with this candidate?

    You get a mutual introduction and the candidate can make the same judgment himself. Of the people that I’ve hired in this way, all of them have become exquisite and outstanding team members. None of them have ever left.

  19. errno says:

    It weeds out the outright liars who say they can write foo language on bar platform when they can’t at all.

  20. MikeT says:

    void copyFile(String in, String out)

    {

    File source = new File(in);

    File target = new File(out);

    FileInputStream reader = new FileInputStream(source);

    FileOututStream writer = new FileOutputStream(target);

    byte[] buffer = new byte[1024];

    int bytesRead;

    while ( (bytesRead = reader.read(buffer)) != -1)

    writer.write(buffer);

    reader.close();

    writer.close();

    }

    Not too hard. It took me just just a minute to find the JavaDocs for FileInput/Output streams.

  21. Julian says:

    Though asking people to solve programming questions is a flawed way of measuring someone’s ability, it’s the best approach I’ve come across.

    (And by programming, I mean problem solving and algorithms, not factual knowledge you can look up.)

    It’s definitely more revealing than asking them is discuss their prior work.

  22. It may seem stupid to have such a simple test, but often candidates who can talk the talk turn out not to have programming experience. For someone who knows how to code, this should be a trivial job. Heck, with any decent programming language you don’t need to know any system API’s, just your basic libraries. Even in C++ it’s pretty straight forward to implement a basic copy operation using the standard libraries.

  23. ebola says:

    If I was asking the questions, I would be asking for solutions to open-ended problems and observing the candidates approach to solving the given problem through either an; oral walk-through, via pseudocode or even diagrams.

    As I feel it is more important to identify people who can think for themselves and solve real software engineering problems with a quality approach rather than someone who can regurgitate this or that algorithm?

  24. Cristian says:

    You’re perfectly right. I was thinking of this since 5yrs ago when I failed 1-2 programming tests. I don’t know syntax/API about a language but I’m sure I can do pretty well if I have a help next to me.

    Anyway, the whole idea with the job interviews should focus on what he achieved (in college, last job, etc), what he can do in a situation (let’s say: how would you implement a file copy? – without implementing) and what are his/her expectations, technically speaking.

    Basically: I did this, I started this, I learned this, etc. I want to do this, i want to work like this, etc. Based on what he has learned/done you can see if can learn/perform good in a new environment.

  25. Mike says:

    If you are not a Microsoft employee yet, apply. You are totally clueless, and thus a good fit.

  26. Mark L says:

    I think programming tests are useful.  It would surprise you how many "experienced" developers cannot implement a solution for a few simple problems like an algorithm like "reverse the string" in a reasonble (~30 minutes) time.

    Also specfic questions in an interview may be used to

    1) probe the breadth of you knowledge. An interviewer doesn’t expect you to have all the answers but if you say you did 5 years of Win32 development on your resume, you should a expect a few questions specific questions about Win32 API.

    2) See if you are bull-shi*tter when you don’t know the answer.

  27. Jman says:

    An aside: Please don’t every ask someone to write code on anything besides a computer (paper, whiteboard, etc.).  Psuedo code is fine.

    Even though I’m a competent programmer, I’ve been asked to write a "Hello World" program on paper in an interview . . . and I blanked!

    If I was in an IDE, it would have been no problem 🙂

  28. Shane says:

    I totally agree, they are useless. One interview i took involved me turning up and them saying "hi, heres a test". If a company doesnt want to ask any questions about me & my qualifications, ala normal interview, i doubt it is a very good place to work.

  29. Pierce Wetter says:

    Weird, I had two job interviews this week. One asked me tech questions, one knew of me so didn’t bother.

    Mostly, I use those sorts of questions as a chance to show off. In my case, I’d show off that the hard part about copying a file is not the bytes, but the error checking. Is there enough room for the new file? Is there enough room for the new file given the block size on the new device? Etc.

    However, my standard interview question for junior-level interviewees would be to ask them to write code to walk through a linked list.

    You’d be surprised how many recent CS graduates couldn’t do that…

  30. Bruce McLeod says:

    I make all cantidates write code in my interviews. The reason I do it is summed up in this quote HIRING A JUGGLER, Chapter 15 of ‘Peopleware’ by Tom DeMarco and Timothy Lister.

    Circus Manager: How long have you been juggling?

    Candidate: Oh about six years

    Manager: Can you handle three balls, four balls, and five balls?

    Candidate: Yes, yes and yes

    Manager: Do you work with flaming objects?

    Candidate: Sure.

    Manager: …knives, axes, open cigar boxes, floppy hats?

    Candidate: I can juggle anything

    Manager: Do you have a line in funny patter that goes with your juggling?

    Candidate: It’s hilarious

    Manager: Well that sounds fine. I guess you’re hired.

    Candidate: Umm… Don’t you want to see me juggle?

    Manager: Gee, I never thought of that.

    It would be ludicrous to think of hiring a juggler without first seeing him perform. That’s just common sense. Yet when you set out to hire an engineer or a designer or a programmer or a group manager, the rules of common sense are often suspended. You don’t ask to see a design or a program or anything. In fact, the interview is just talk.

  31. Steven Arnold says:

    My personal experience is that it’s shocking how many people can’t pass a simple programming test in their strongest language.  I agree the file copy is stupid.  I like to ask people to write a Pig Latin translator.  Does it prove you’re an excellent object-oriented architect?  No, it proves that you have basic beginning skills in your craft — it’s a minimal starting point and we continue from there.  If you can’t write a working Pig Latin translator in 20 minutes, you can’t work for my company; if you can, maybe you’re still not good enough, but you passed one hurdle.

    You know, I’ve never regretted hiring anyone who passed that test.  I did hire a couple people who didn’t pass it, but who seemed impressive in other ways, and those were mistakes.  I’d never omit it from a technical interview.

  32. Daryl S says:

    That is a good test if hiring a juggler and doing a programming test is good if hiring a programmer. Who hires programmers?  I want developers, A/Ps (whatever you want to call them) … I want someone who can solve problems, think, design, not someone who can merely cut code.

  33. PHP kitchen says:

    Here’s quite a good list of resources if you’re looking for programmers, or indeed if you are a programmer yourself looking for work. PHP devs seem to be very much in demand these days, at least that’s the case here in London, we’re talking almost lik

  34. Wow, just had a look at my control panel to free some comments, and realized I had hit 500 posts. So,