Summary of the recent spate of /3GB articles


A table of contents now that the whole thing is over. I hope.

I'm not sure how successful this series has been, though, for it appears that even people who have read the articles continue to confuse virtual address space with physical address space. (Or maybe this person is merely mocking a faulty argument? I can't tell for sure.)

Comments (36)
  1. winden says:

    The confusion on some comments just highlights what we all know: memory management is hard to grasp and is easier to use when handled by the OS or the language.

  2. Bryce says:

    I think that the wording now reflects what I was thinking. I didn’t think that I was confused but now I’m not so sure. Help?

  3. For what it’s worth, I really liked this series – I always enjoy your blog, and it was particularly good to have such thorough and deep coverage of a topic.

    If some people have misunderstood certain fundamental points like the distinction between virtual and physical address space, then I don’t think that’s necessarily indicative of a problem with the series. I would say simply that it indicates that those readers weren’t really ready to read the articles.

    I suppose you could have assumed a different level of knowledge from your readers, but that would simply have made the articles useful to a different set of readers. It wouldn’t necessarily have made them better as such.

  4. mpz says:

    I have to say I’ve enjoyed these articles too. To top the series off, do you have any tidbits about the x86-64 Windows that you can share, especially memory related?

    Like somebody else commented in one of the articles – it’s a revolution waiting to happen. Once x86-64 Windows is out, there’s no going back.

  5. Carlos says:

    Myth:Virtual memory is system memory that is simulated by the hard drive.

    This definition comes from googling "what is virtual memory":

    http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=what+is+virtual+memory

    Surely developers aren’t this clueless?

    When interviewing I used to ask some questions about virtual memory. I never got good answers – the interviewees would have no idea or they would give me something similar to the definition above. Virtual address space was a mystery.

    I’m not even slightly surprised that people have trouble with /3GB, PAE, etc.

    The articles are excellent in themselves, but I think that the basic misunderstandings fuel the myths, so an introductory article would have helped.

  6. Norman Diamond says:

    Well, what the hardware does with virtual memory isn’t the entirety of the meaning of virtual memory. Sure all the rest of the baggage associated with virtual memory has part of the CPU’s hardware as its foundation, but the baggage is what users usally have to deal with. Plus in Windows the settings for page file sizes often say virtual memory instead of page file. When the abstraction doesn’t leak, no one has to remember that it’s founded on address translation units and stuff like that.

    Meanwhile, virtual memory isn’t alone in confusing people. Recently I had a run-in with someone who didn’t understand the possible benefit of increasing the physical memory in his notebook from 64MB to 128MB. He designs electronic circuits, and he uses computers (using both hardware and software), but I guess the design and meaning of computer hardware just never entered into his experience.

  7. Raymond Chen says:

    I’m not surprise that your average computer user has trouble with the virtual memory / physical memory / address space distinction. But your average computer user also doesn’t go spouting off about the /3GB switch.

  8. Frodo says:

    This is a very difficult subject for most people to understand, even programmers. I mean, this:

    >>Virtual memory is system memory that is simulated by the hard drive.

    This isn’t a bad explanation for a non-programmer. I assume Carlos is taking the position that this is wrong, but — conceptually — it works for laymen.

  9. If virtual memory is simulated by the hard drive, what are virtual disks simulated by?

    Hmmm…. :)

  10. Tito says:

    I sincerely thank Raymond for covering this topic. I don’t think I was alone in the confusion. My main incorrect notion was about the distiction between Virtual Memory and Virtual Address Space. (I had no idea there even was a distiction.)

    However, it is really unfortunate that on msdn, the functions that were mentioned here (CreateFileMapping,(Un)MapViewOfFile) aren’t mentioned anywhere in the "memory management" section:

    http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtualalloc.asp

    That the "virutal memory" functions listed there actually handle both the allocation AND the mapping is surely the cause of much of this confusion (it was for me).

  11. David Candy says:

    It seems to me the confusion is per machine vs per process. Raymond is talking per process and people are thinking per machine and interpreting it from that perspective.

    Larry O was talking about VM from the machine perspective (page file size + physical memory + (I presume) code segments in exe and dlls) which is called commit charge.

    http://weblogs.asp.net/larryosterman/archive/2004/03/18/92010.aspx (and look at surrounding days).

    With raymond, he is talking about things that are normally done for system configuration (editing boot.ini) but is talking about the effect on a process not the box.

    The person you linked too is also thinking you are talking about system configuration (eg setting the page file size).

  12. David Candy says:

    Although in 3.1 enhanced mode it was said that a swap file made the system run quicker, even if it wasn’t used as it wqas designed to swap and was slower for ????????????? reason if it couldn’t.

    Still changing LRU sweep frequency et al never made any detectable difference.

  13. Andrew Feldstein says:

    Thank you for the jump table.

  14. Ben Cooke says:

    I’ve been quite interested in this stuff because I rarely encounter these kinds of problems in my day-to-day programming and so I’ve never encounted this 3GB switch nor had to worry about distinctions between memory types.

    Of course, as a programmer I can’t help but know what virtual address space is. However, I don’t recall anyone ever saying specifically what virtual *memory* is, as a concrete definition. I have some hazy ideas in my head about it, but I don’t feel confident that I could actually give an accurate definition in English. Can anyone offer one?

  15. David,

    In general, having a paging file makes systems run faster – that’s a corrolary to my post (btw, I wish you’d also included a reference to the correction: http://weblogs.asp.net/larryosterman/archive/2004/05/05/126532.aspx – while my post was roughly correct, there were certain critical details I missed).

  16. Peter Evans says:

    Raymond,

    Is this series a personal rant? (An early entry mentioned something about having to live with it for six years.)

    I am puzzled as to how these posts relates to your other programming posts in your blog. Well there were those few discussion about lock free approaches.

    Hmm. Maybe you are just leading up to details of 64-bit pointers issues and other memory management issues and design issues with regard to 64-bit hardware and drivers.

    Haven’t read the each entry in the series yet. Been catching up. Good to see a little coverage of these poorly understood things.

    Will the series continue to cover other aspects of memory management such as the secured memory concepts of .Net and Non-executable memory regions.

  17. At work, there’s been some debate on how best to structure our J2EE servers to maximise utilisation of a scarce resource (per-CPU licenses!). The crux of the debate centered, for some reason, on how we can allocate more JVM instances…

  18. Back when Windows Server 2003 SP1 shipped, HTTP.sys hit an important milestone, WOW64 support.What is…

  19. 32bit Windows is Not Limited to 2GB/4GB

  20. I started programming in the old times of 16-bit MS-DOS, where you had only 640 KB of memory, and you

  21. I started programming on x86 machines during a period of large and rapid change in the memory management

Comments are closed.