One reason why apps have security bugs -- because we developers were trained to focus on and typically only ever focused on how legitimate users will use the product -- we never used to have to think about misuse!
A couple of years ago I wrote up a little skit. It’s a software developer and a Quality Assurance (QA) guy, circa 1993. I play the developer and Steve Riley plays the QA guy.
QA Guy (imitating Newman from the TV show "Seinfeld"): Hello, Jerry.
Dev (imitating Jerry Seinfeld, with derision): Hello, Newman.
QA Guy: How are you today?
Dev: How am I? I was fine. But now I have a QA guy in my cubicle. The only clearer sign I could possibly receive that my life was about to take a downward turn would be to receive an invitation to appear on the Jerry Springer show.
QA Guy: Good guess! I found a serious bug in your program.
QA Guy: Yep. I found that if I entered a last name of 33 characters or more, the program crashes.
Dev: That’s serious?
QA Guy: Yes, it’s serious. The program crashes!
Dev: Yeah, but it will never happen. No one has a last name that long.
QA Guy: Someone could.
Dev: OK, I’ll make the buffer 40 characters instead. That should be more than enough.
QA Guy: Yeah, but then a 41 character name will crash it.
Dev: So put something in the manual that says that last names can’t be more than 40 characters! Jeez! It will never happen, OK? These edge cases don’t matter.
QA Guy: Of course they do – someone could crash it on purpose.
Dev: What? Why would anyone do that? First of all, our customers are paying good money for our product. Why would they then deliberately misuse it and make it crash on purpose? And second of all, it’s running on NetWare – it crashes all the time on its own anyway, without any help!
QA Guy: Well, that’s true – maybe we ought to switch to Windows NT. (laughs)
Dev: (laughs) Oh, yeah, Windows NT 3.1, there’s a pillar of stability.
QA Guy: Uh huh – and performance. (laughs) What a joke!
Dev: Yeah. But you know, I’m seeing more RFPs lately that specify that "the server platform must include Solitaire".
QA Guy: Ha ha! Windows NT will never amount to anything. A dozen years from now no one will even remember that it ever existed.
Dev: Yep. The future obviously belongs to NetWare.
OK, so are we done here with this so-called bug of yours? No one’s going to crash the server on purpose, OK?
QA Guy: Sure they would. They could take advantage of your buffer overrun to inject code into the system.
Dev: Oh, now this is an advance in computing that I was hitherto completely ignorant of. I’ve seen computers that ship with monitors, and keyboards, and even tape backup, but I had never heard of one that ships with a syringe!
QA Guy: It goes like this: The attacker sends more data than your buffer can hold, thus overwriting the return address on the stack with a pointer back into the buffer, which now contains malicious code sent by the attacker. The attacker now runs code of his choosing on your server.
Dev (stares at QA guy for a while, before replying): I think that hair dye has leached into your brain. Gee you know, it’s hard enough to get our code deployed – maybe we should use “code injection” instead. Look, Steve, we’ve got a ton of features we still have to implement and a ridiculous deadline. The server’s only got 2MB of RAM – we can’t afford the time to code up all these extra checks, and we can’t absorb the performance penalty of running them either. This isn’t a bug. The program works as designed. If you want to go find real bugs, why don’t you go after that new guy we hired, Jesper Jo-whatever-his-name-is.