When you have only 16KB of memory, you can’t afford to waste any of it on fluffy stuff


The original IBM PC came with 16KB of memory.

That’s not a lot of space in which to squeeze an operating system, especially since you had to leave enough memory for the user to actually get work done. A product of its time, the MS-DOS kernel is written entirely in assembly language, pretty much standard procedure for programs of the era. It also meant that the code takes all sorts of crazy shortcuts to shave a few bytes here, a few bytes there, in order to squeeze into as little memory as possible. For example, one very common trick was to have jump into the middle of an instruction, knowing that the second half of the instruction, when reinterpreted as the start of an instruction, performs the operation you wanted.

Anyway, this subject arose in response to my discussion of why a corrupted program sometimes results in a “Program too big to fit in memory” error, which prompted the question from commenter 8 wondering why the kernel didn’t simply reject .COM files bigger than 64KB?

Well, yeah, and that’s what it did: By complaining that it was too big to fit into memory. There’s no point adding a redundant test. (It appears that some people like to call these redundant tests basic sanity checking, but I consider sanity checking to be protecting against unreasonable values before they cause trouble. But in this case, they don’t cause trouble—the error is detected and reported even without the so-called sanity check.)

Consider:

bool SomeFunction(...)
{
 ...
 if (x == 3) return false;
 if (x < 10) return false;
 ...
}

The first test is redundant, because if x is three, then even without the test, the function will still fail because x is also less than ten.

And when you’re trying to squeeze your kernel into a few bytes as possible, you’re certainly not going to waste your time coding up a redundant test.

Comments (16)
  1. Miral says:

    Redundant checks like that are more useful when you can give more specific error messages (which of course you can’t in the context of a must-be-as-small-as-possible DOS kernel).

  2. porter says:

    > Redundant checks like that are more useful

    > when you can give more specific error

    > messages

    Such as "try again, 3 is still less than 10"

    When you only have 16Kb on a PC you realise you should have got a CP/M machine.

  3. sorry :) says:

    Contemporary IBM PC clones come with 2GB of memory.

    That’s not a lot of space in which to squeeze an operating system, especially since you had to leave enough memory for the user to actually get work done. A product of its time, the Vista userspace components are written entirely in high level languages, pretty much standard procedure for programs of the era. It also means that the code takes all sorts of crazy abstractions to waste a few bytes here, a few bytes there, in order to be in accordance with modern development practices as much as possible. For example, one very common change is to replace data stores that used to be plain text files or compact binary files with XML files and databases, respectively, knowing that they perform the operation you wanted, just at a higher performance cost.

  4. Josh Straub says:

    Well played, Mr. "sorry :)" on January 28, 2009 9:01 PM!!

  5. Anonymous Coward says:

    Sorry :-) has a point. XML was a mistake. And they had enough experience at the time to know it, that’s the sad thing.

  6. enjpm says:

    This post is related to a discussion I just had (actually, it is related to the original post but comments are off there)(and it’s only really tenuously related to that, so I’m sorry)(though not sorry enough to not actually post, just a bit less than that). On the original post, Raymond says to save your work before trying some random file as an .exe, and people ask what might happen.

    I’ve never programmed to the windows API directly, but have written lots of image processing code in C++ – and today we managed to white-screen XP reproducibly, the first time I’ve seen XP die so spectacularly. The question we discussed was:- is the white screen a result of an event within the deep dark bowels of windows that is raised after some particular bit of memory is accessed, which prompts that section of windows to do a quick check that things are still ok, then find that they’re not and goes to panic stations? We assumed that windows can’t differentiate between a process directly accessing memory it owns and memory it doesn’t, ie. checking each time that the user/process has the right credentials, as how could you possibly do this without it being unearthly slow?

    In that case then, we are manipulating pointers to image buffers directly all the time, and we could just get the line pitch wrong one day, jump out of the memory for our process and just dive straight into the windows kernel. Sure, it would die very soon afterwards but when exactly? And the damage could be pretty bad – what is the worst that could happen? Or am I just missing something fundamental?

  7. peterchen says:

    I am no stranger to blaming XML, but how does that relate to the part that executes MS-DOS style .COM programs?

    Why touch a part that is working, just to give a different error message? It’s not that windows was exactly designed for renaming arbitrary files to .com and running them.

  8. configurator says:

    @enjpm: The processor has ways to check what memory is and isn’t owned by the current process, though I’m not sure about the terminology.

  9. Dog says:

    Thanks for the history lession, Raymond, but that still doesn’t explain why (the) MS-DOS (emulator) doesn’t do the check in 2009.

    That’s the thing about hardware limitations, you can’t continue using them as an excuse when they no longer exist. Now in this particular case it’s probably not worth it, but Windows is filled with these sorts of things.

  10. enjpm says:

    Ah, so the cpu is in charge of such matters. Ok, thanks for the reply, that all makes sense. I knew there was something fundamental I was missing.

  11. RobertWrayUK says:

    Dog – backwards compatibility probably!

    There’s probably some program, somewhere (undoubtedly a LOB application run by a Fortune 500, or similar) that does something crazy that expects that specific error message.

    Stranger things have happened!

  12. Karellen says:

    "We assumed that windows can’t differentiate between a process directly accessing memory it owns and memory it doesn’t, ie. checking each time that the user/process has the right credentials, as how could you possibly do this without it being unearthly slow?"

    Windows, and all modern OSs, *can* do this, and do it with the help of the CPU. Read up on paging[0], page tables[1], page faults[2], the translation lookaside buffer[3], memory management[4] and the MMU[5]. Because the OS and the CPU need to maintain the process-virtual-memory to physical-memory mapping and translate between them on memory accesses, checking that the memory being accessed even *has* an entry in the mapping is a free side effect.

    The mapping also includes things like whether a page can be written to, whether it can be treated like code and executed, whether it is shared and needs copy-on-write semantics, and a whole bunch of other stuff.

    Because the CPU does nearly all of the work for the common case (process memory is paged in, and the TLB is up-to-date) and it’s just part of what the CPU does to access memory, there’s almost no slow-down at all compared to not using virtual memory. If you’re reading and writing large amounts of memory, the page checks are probably dwarfed by the time it takes to read the data across the memory bus.

    [0] http://en.wikipedia.org/wiki/Paging

    [1] http://en.wikipedia.org/wiki/Page_table

    [2] http://en.wikipedia.org/wiki/Page_fault

    [3] http://en.wikipedia.org/wiki/Translation_Lookaside_Buffer

    [4] http://en.wikipedia.org/wiki/Memory_management

    [5] http://en.wikipedia.org/wiki/Memory_management_unit

  13. dave says:

    but that still doesn’t explain why (the)

    MS-DOS (emulator) doesn’t do the check in 2009.

    Because there’s no need to change working code?  

    Especially obsolete working code like a compatibility layer for a 1980s operating system.

    The system reliably detects images whose size word is corrupted. What more is needed?

  14. Dog says:

    dave: Like I said, in this case it’s probably not worth it.

    The thing is, this is far from the only case where historical hardware limitations stiff affect Windows’ performance and usability (at both the UI and API levels).

    Seriously, a good 90%* of Raymond’s posts start with "In 16-bit Windows…" or "In Windows 95…". My point is that you can’t use history to explain why something is the way it is *now*. You can use history to explain why it was that way originally, but then you’ve got to fill in the gap (probably with "and we never bothered to change it").

    * Source: Made up.

  15. For anybody that was born after say 1980 it would do wonders to go back for a while and program some old box you buy on ebay for $50 and make it do something useful. Or do something with an Atmel or PIC chip. You’re certain to pick up a few tricks that you’ll use in your ‘regular’ programming.

  16. Gregory Kong says:

    Dog:

    "Seriously, a good 90%* of Raymond’s posts start with "In 16-bit Windows…" or "In Windows 95…". My point is that you can’t use history to explain why something is the way it is *now*. You can use history to explain why it was that way originally, but then you’ve got to fill in the gap (probably with "and we never bothered to change it")."

    Er, yes, that’s what Raymond’s blog is all about. The past. Backwards compatibility. The whole megillah. Why is something the way it is now? Because of the past, yes, but more importantly, it would *break* stuff – *very expensive and important* stuff – if they changed it. It’s not that they didn’t bother, it’s that they couldn’t.

    Consider the HUGE library of applications written for DOS, Extended DOS, Win16, Win32, and now .NET and Win64. Consider that MSFT’s core goal is to make money by selling software, most notably its cash cows Windows and Office. Consider that we have very obvious evidence showing us that making changes to the way apps work virtually guarantees the failure of a particular OS release.

    And then consider that the original C&C had better work under Windows Vista pretty much the same way as under DOS 6.xx – warts and all – or else some segment of the population (including me) would not upgrade to it.

Comments are closed.