Part 3: Understanding !PTE - Non-PAE and X64

  • Article

Hello, Ryan Mangipano (ryanman) again with part three of my series on understanding the output of the !PTE command. In this last installment I’ll continue our manual conversion of Virtual Addresses by converting a Non-PAE VA. Afterwards I’ll convert a VA from X64 Long Mode. Then I’ll discuss the TLB. If you haven’t read part one and two of this series I recommend taking a looking before jumping into the rest of this article.

Non-PAE system VA to Physical Address conversion

In part 1 and part 2 I discussed using windbg to manually x86 PAE virtual to physical address conversion with 4KB and 2MB pages. Now I’ll mention the same thing for Non-PAE systems. The official processor manuals explain how systems not using PAE have only two level of tables instead of the three used by PAE. This fact is because PAE added the Page Directory Pointer Table. This means the CR3 register will contain a pointer to the Page Directory Table. Also note the size of the table entries will be 32-bit, not the 64-bit size observed in the PAE tables. PAE expanded the table size to allow for more bits for the purpose of allowing the addressing of more physical memory.

The PAE bit is bit number five, which is the sixth bit due to bit numbering starting at zero. You can see PAE is not enabled on this system.

       0: kd> .formats @cr4

  Binary: 00000000 00000000 00000110 11010001

 Another method of checking this:

               1: kd> j ((@cr4 & 0y00000000000000000000000000100000) != 0) '.echo PAE flag Enabled';'.echo PAE flag Disabled'

PAE flag Enabled

 Now I'll use the following random virtual address which is valid:

 

0: kd> !pte f72c5c00

F72C5C00 - PDE at C0300F70 PTE at C03DCB14

          contains 01014963 contains 06CE7963

        pfn 1014 G-DA--KWV pfn 6ce7 G-DA—KWV

 

0: kd> .formats f72c5c00

  Binary: 11110111 00101100 01011100 00000000

 Let’s break it down:

PD offset 11110111 00

Page Table Offset  101100 0101

Offset in the Physical Page1100 00000000 (since it’s a even 12 bits, just refer to it in hex as c00)

 

0: kd> !dd @cr3 + (0y1111011100 * @@(sizeof(nt!HARDWARE_PTE)))L1

# a07df70 01014963

 

0: kd> !dd 1014000 + (0y1011000101 * @@(sizeof(nt!HARDWARE_PTE)))L1

# 1014b14 06ce7963

 Now that I have the physical page base, I'll place the last 3 hex digits (c00) from the Virtual Address onto the address base.

 

0: kd> !dd 6ce7c00 L4

# 6ce7c00 00000001 c0000005 00000000 00000000

 

0: kd> dd f72c5c00 L4

f72c5c00 00000001 c0000005 00000000 00000000

X64 VA to Physical Address Conversion

Just as PAE added a third level to the non-PAE two-level system, x64 Long mode adds a fourth level to the hierarchy. This table is called the Page-Map Level-4 (PML4 table). AMD refers to the entries in this table as PML4E (Page-Map Level-4 Entry). Intel refers to each entry as PML4-Table Entry. Internally we refer to this as the eXtended Page directory Entry (PXE). Regardless of how you refer to these entries they contain indexes into the PDP table (Page Directory Pointer Table).

 Here is the output of the !pte command against a 64-bit address:

               7: kd> !pte fffffade`c24eb7c0

                                 VA fffffadec24eb7c0

PXE @ FFFFF6FB7DBEDFA8 PPE at FFFFF6FB7DBF5BD8 PDE at FFFFF6FB7EB7B090 PTE at FFFFF6FD6F612758

contains 0000000111800863 contains 0000000119826863 contains 0000000119839963 contains 0000000001FF6121

pfn 111800 ---DA--KWEV pfn 119826 ---DA--KWEV pfn 119839 -G-DA--KWEV pfn 1ff6 -G--A—KREV

I'll break it down in binary and use data from the processor manuals to separate the bits

7: kd> .formats fffffade`c24eb7c0

  Binary: 11111111 11111111 11111010 11011110 11000010 01001110 10110111 11000000

 

Sign extend 11111111 11111111

PML4 offset 11111010 1

PDP offset 1011110 11

PD offset 000010 010

Page-Table offset 01110 1011

Physical Page Offset 0111 11000000

 

Now that I have the numbers, I'll plug them in and find the physical address. If you are having problems following along, refer to part one of this blog and the AMD x64 System Programming manual. You should be comparing the output below to the !pte output above

 

7: kd> !dq @cr3 + ( 0y111110101 * @@(sizeof(ntkrnlmp!HARDWARE_PTE))) L1

# 147fa8 00000001`11800863

 

7: kd> !dq 0x00111800000 + ( 0y101111011 * @@(sizeof(ntkrnlmp!HARDWARE_PTE))) L1

#111800bd8 00000001`19826863

 

7: kd> !dq 0x119826000 + ( 0y000010010 * @@(sizeof(ntkrnlmp!HARDWARE_PTE))) L1

#119826090 00000001`19839963

 

7: kd> !dq 0x119839000 + ( 0y011101011 * @@(sizeof(ntkrnlmp!HARDWARE_PTE))) L1

#119839758 00000000`01ff6121

 

7: kd> !dc 1ff67c0 L4

# 1ff67c0 5085ff48 48000005 68244c8b 04a8f633 H..P...H.L$h3...

 

7: kd> dc fffffade`c24eb7c0 L4

fffffade`c24eb7c0  5085ff48 48000005 68244c8b 04a8f633 H..P...H.L$h3...

TLB- Translation Lookaside Buffer and Conclusion

The CPU’s memory management unit performs these operations to translate virtual addresses to physical. Wouldn’t it be great if we could cache the virtual address to physical page information in a location that can be accessed very quickly so that the CPU doesn’t have to look this up for future references to this page? That is just what the Translation Lookaside Buffer (TLB) does. Hopefully this will shed some light on some basic memory structures like Large Pages, Flags, and the TLB so I encourage you to read more about these topics from the following sources-

How PAE x86 works (on MSDN): https://technet.microsoft.com/en-us/library/cc736309(WS.10).aspx

Intel & AMD processor manuals: https://www.intel.com/products/processor/manuals/index.htm and https://developer.amd.com/documentation/guides/Pages/default.aspx#manuals

“Windows Internals, 5th Edition” Mark E. Russinovich and David A. Solomon with Alex Ionescu -Chapter 9: Memory Management