Why is the page size on ia64 8K?


On x86 machines, Windows chooses a page size of 4K because that was the only page size supported by that architecture at the time the operating system was designed. (4MB pages were added to the CPU later, in the Pentium as I recall, but clearly that is too large for everyday use.)

For the ia64, Windows chose a page size of 8K. Why 8K?

It's a balance between two competing objectives. Large page sizes allow more efficient I/O since you are reading twice as much data at one go. However large page sizes also increase the likelihood that the extra I/O you perform is wasted because of poor locality.

Experiments were run on the ia64 with various page sizes (even with 64K pages, which were seriously considered at one point), and 8K provided the best balance.

Note that changing the page size creates all sorts of problems for compatibility. There are large numbers of programs out there that blindly assume that the page size is 4K. Boy are they in for a surprise.

Comments (31)
  1. IIRC, the page size on the 32 bit alpha port was also 8K.

  2. mschaef says:

    "There are large numbers of programs out there that blindly assume that the page size is 4K."

    Can you provide an example of how the page size might matter to a program? (other than a debugger, or some kind of system tool, etc.)

  3. Nicholas Allen says:

    Yeah, the Alpha port had an 8K page size (like the x86 I think it was the only supported size for the platform). The R4000 had an adjustable page size from 4K-64K.

  4. Eric Wilson says:

    I’ve heard conflicting reports above whether the OS uses 4MB pages or not. Some of the reports mention that Win2k will use 4mb pages for some OS structures and code if it is availabe. Out of curiousity, is this true?

  5. Raymond Chen says:

    mschaef: Consider the function in http://weblogs.asp.net/oldnewthing/archive/2004/08/10/211890.aspx and how it could easily have been written to assume 4K pages.

    (I don’t know the answer to the 4MB page question.)

  6. Matt Povey says:

    Nicholas,

    re: the adjustable page size. Why only on the R4000? It strikes me (as a non-developer so excuse stupidity) that a run-time adjustable page size would make sense in terms of being able to configure for certain loads.

    For example, would MSSQL, Oracle and Exchange benefit from an adjustable page size for different kinds of loads?

    Also, out of interest, I guess the page size on AMD64 is still 4K for compatability? Or has it changed?

  7. Barry Tannenbaum says:

    re: run-time adjustable page size

    Remember what the page size is used for. The size of your page effects the number of page table entries that are maintained by the OS. Since pages are a fixed size, the virtual memory system can make a simple calculation to convert from a virtual address to a physical address.

    This isn’t something that can be changed on a per-process basis.

    Unless we go back to segmented architectures, which require more complicated VM systems.

  8. Ben Hutchings says:

    Raymond: That’s dependent on the allocation granularity, not the page size. The same applies, though, since allocation granularity is also at least theoretically variable from the normal 64KB.

  9. Matt Povey says:

    Barry,

    Sorry, by run-time, I meant the OS (i.e. configurable at OS startup). I wasn’t being at all clear there :).

  10. Miles Archer says:

    IIRC, VAX/VMS had an AutoGEN program that would run to determine all sorts of OS parameters such as working set size. I thought it had the page size as a boot time variable.

  11. Nicholas Allen says:

    Matt-

    The OS is only one part of the equation in most modern virtual memory systems. A lot of the actual work is done by the processor itself. In this case, the page size is limited to the sizes the processor gives support for. Multiple page sizes are not a high demand feature so processor manufacturers generally didn’t spend the silicon to support them.

  12. Nicholas Allen says:

    re: the 4 MB page question

    I also don’t know the level of OS support for these. But I believe the 4M page support actually started with the i860 and not the Pentium. Given NT’s special connection there, anything is possible. I don’t know if it was the original i860 (the N10) or the i860 XP (the N11) that debuted jumbo pages though.

  13. Barry Tannenbaum says:

    re: VAX page size.

    Nope. The VAX had a fixed (anemic) 512 byte pagesize. Even when the VAX was introduced in 1977 it was considered small.

    It’s been a long time, but I vaguely remember that the OS would bring in a "cluster" of pages on a pagefault. Perhaps that’s what you’re referring too.

    There were lots of things that you could change with AUTOGEN which it was generally healthier to leave alone. My favorite was the one which was specified in units of Furlongs Per Fortnight. When asked why he specified such an odd unit, the developer replied "If you have to ask, you shouldn’t be mucking with it."

  14. Filip says:

    Since W2K the OS really uses 4MB pages for the kernel if the computer has more then 127MB (IIRC) of the RAM. You can configure it with LargePageMinimum registry setting (Google for details) to force using it only with more RAM (you still can’t go under 127MB) or disabling it at all. In WinXP+ the minimum was raised to 255MB (IIRC).

  15. Ulf Vatter says:

    Hmm, I guess one of the most important pro points for larger page sizes is still missing: the page table walks become faster and (most importantly) the TLB foot print becomes smaller. This can deliver surprising big speedups on some systems/workloads/etc.

    To guess about the 4MB-Pages-in-Win-Question: Of any modern kernel on the i386 arch, I expect it to use superpages to map certain parts of the kernel area of the virtual address spaces — it saves page tables, is faster installed than a comparable mapping using 4k pages, plus the advantages above. Since adding support for this is not difficult (uh, well, or at least should not really be difficult if your kernel is designed halfway sensible), I think it is almost safe to say that a modern Win2000 kernel uses superpages.

    To use it to map the user area is more difficult and questionable. The idea to mix multiple page sizes is old (I heard a rumor, that already Multics did that and wasn’t happy with it) — but unfortunately many problems creep up. As an example:

    Upon using a mapping (i.o.w. accessing memory), the cpu (well, most of them) set reference bits. These are evaluated by the OS to decide which page is least likely to be used in the future again. This will be the first page to be swapped out under memory pressure. If you use many large pages, the statistic might become very blurry — the wrong pages might get paged out, the overall performance of the system might get severely degraded.

    I’m not aware of a OS actually using multiple page sizes for user space (well, since Multics anyway). However, recently this was published:

    http://www.cs.rice.edu/~druschel/publications/superpages.pdf

    Didn’t read it, but was told it was good. Have fun.

  16. Skywing says:

    Actually you can use large pages in user space with Windows Server 2003; pass in MEM_LARGE_PAGE for VirtualAlloc.

  17. One other consideration is that using large pages causes the OS to use up fewer PTEs. This was the motivation for the 4MB page stuff AFAIK.

    Also, the nonpaged pool is mapped out of large pages, in addition to the kernel and HAL, as filip said.

  18. Nicholas Allen says:

    Is there even such a thing as PTEs in the 4MB page world? I thought the PDEs went straight to the page frames. Page tables as such didn’t even exist.

  19. Raymond Chen says:

    Oh right that was allocation granularity. But you can easily imagine analogous code to do page computations for VirtualAlloc.

  20. nikita says:

    I don’t understand why file system IO unit has to coincide with the page. At the very least this would make transferring file system images between machines with different page sizes impossible.

    In Linux, for example, page size is 4K on x86, but granularity of file system IO depends on a file system type, requests can be smaller or larger than page.

  21. asdf says:

    A few docs on MSDN still say 64k when they really mean SYSTEM_INFO::dwAllocationGranularity (take VirtualAlloc for instance). I reported it a while ago and it still hasn’t been fixed.

  22. Tony Cox [MS] says:

    nikita: This isn’t about unbuffered file I/O, it’s about how hard page faults are serviced. When a page is hard-faulted in, that results in I/O – the OS has to go to the pagefile and bring in the data to physical memory so the application can work with it. Obviously the whole page has to be filled in such a case. So the page size sets a lower bound on the I/O size used for servicing page faults. Make sense?

  23. The reason that page size tends to match I/O transfer size is because of the way the cache manager works. Because it’s a virtual block cache (as opposed to the logical block cache setup used in most cache systems), images are swapped into and out of the cache in page size increments. Also, when an executable image is loaded, it’s actually just mapped into the cache and faulted into memory page-by-page.

    Not all I/O needs to be page-based, and in fact there is some work being done to make it work better (see a recent thread from Neal Christensen on NTFSD), but regardless, you can see why page size makes such an impact.

    FWIW, this is also relevant to network redirectors – if you map a file over a redirector through the system cache, it’ll get faulted in and written out a page at a time. 4K, 8K, etc., might not be an optimal minimum transaction size on many networks.

  24. Paul Walker says:

    Note that changing the page size creates all sorts of

    >problems for compatibility. There are large numbers of

    >programs out there that blindly assume that the page

    >size is 4K. Boy are they in for a surprise.

    You mean you actually broke compatibility? Wow. :-) What happened to the bending-over-backwards previously done to make everything (including buggy apps) continue working?

    (Not meant in a snarky way, I’m genuinely curious as to why this particular decision came out this way)

  25. Because we didn’t break compatibility.

    x86 is still 4K pages. That hasn’t changed. But when you’re porting to a new processor architecture, lots of things change, among them the page size.

    There are ALWAYS bugs found in applications when they are ported. But the key word here is "PORTED". The app’s source code is cracked open for testing on the new platform, and bugs found are fixed in the application.

    But for each platform, compatibility doesn’t change.

  26. Ben Hutchings says:

    Larry: x86 applications run on IA64 under emulation. Can the emulation layer simulate 4K pages for them?

  27. nikita says:

    nikita: This isn’t about unbuffered file I/O,

    > it’s about how hard page faults are serviced.

    > When a page is hard-faulted in, that results in

    > I/O – the OS has to go to the pagefile and bring

    > in the data to physical memory so the application

    > can work with it. Obviously the whole page has to

    > be filled in such a case. So the page size sets a

    > lower bound on the I/O size used for servicing

    > page faults. Make sense?

    A lot of. For page faults this makes perfect sense.

  28. The OS does use 4MB pages for mapping the kernel, but only when it’s sure there’s enough memory. I think the cut-off is very low however, any more than 24MB or 32MB of memory ensures 4MB pages are used.

  29. Ben: I believe so. But I don’t know how the x86 emulation works on IA64. I do know that for Alpha, there was a x86 VM emulation layer (WOW32) that emulated the x86 instruction set, including the 4K pages.

  30. Nathan Moore says:

    7-max will supposedly let you use 4mb pages on Windows XP/2000 for your user space apps.

    http://www.7-max.com/

  31. Large Page size increases internal frwagmentation. Variable block sizes, variable this and that (like variable speed drives, for the mac’s) just makes life harder.. The only reason pages exist is to allow for easier memory protection.. Note, I got this in CS courses, and I don’t do any systems programming..

Comments are closed.