A look at the WoW64 layer in the debugger

[1st edit 2 Jan 2007- updated to make the exercise slightly harder] 

When I first started using Windows Server 2003 x64 as my day to day operating system of choice, I did a bit of digging to understand how the WoW64 layer provides support for 32-bit processes to run in the 64-bit world.

Here is a walkthrough of what I found, you should be able to follow the steps yourself if you are running an x64 version of Windows.

Download and install the 64 bit version of Debugging Tools for Windows appropriate for your platform from:

www.microsoft.com/whdc/devtools/debugging/install64bit.mspx

Now run the 32 bit version of notepad:

Start->Run, enter "%systemroot%syswow64notepad.exe" (without the quotes) and click Ok.

Select Start->Programs->Debugging Tools for Windows 64-bit->WinDBG

In task manager, locate the 32 bit instance of notepad.exe on the "Processes" tab. This is displayed as notepad.exe*32

Ensure the PID (process ID) column is visible. (If it is not choose "Select Columns" from the View menu and check the checkbox for "PID (Process identifier)"). Note the process ID for the notepad.exe*32 process.

In WinDBG, on the File menu select "Attach to process". Use the process ID you noted above to select the 32 bit notepad instance in the list of processes and click OK.

If a message box saying "Save workspace?" appears click "No".

If there is no MDI child window titled "Command" visible then select "Command" on the "View" menu and one should appear.

You should see output something like this:

Microsoft (R) Windows Debugger Version 6.4.0007.2

Copyright (c) Microsoft Corporation. All rights reserved.

*** wait with pending attach

Symbol search path is: srv*D:symbolssymcache*\symbolssymbols

Executable search path is:

ModLoad: 00000000`01000000 00000000`01014000 E:WINDOWSsyswow64notepad.exe

ModLoad: 00000000`78ec0000 00000000`78ff9000 E:WINDOWSsystem32ntdll.dll

ModLoad: 00000000`78be0000 00000000`78c26000 E:WINDOWSsystem32wow64.dll

ModLoad: 00000000`78b90000 00000000`78bda000 E:WINDOWSsystem32wow64win.dll

ModLoad: 00000000`78b80000 00000000`78b89000 E:WINDOWSsystem32wow64cpu.dll

ModLoad: 00000000`7d600000 00000000`7d6f0000 ntdll.dll

ModLoad: 00000000`7d4c0000 00000000`7d5f0000 KERNEL32.dll

ModLoad: 00000000`762b0000 00000000`762fa000 comdlg32.dll

ModLoad: 00000000`77f50000 00000000`77fec000 ADVAPI32.dll

ModLoad: 00000000`7da20000 00000000`7db00000 RPCRT4.dll

ModLoad: 00000000`7dbd0000 00000000`7dcd3000 COMCTL32.dll

ModLoad: 00000000`77ba0000 00000000`77bfa000 msvcrt.dll

ModLoad: 00000000`7d800000 00000000`7d890000 GDI32.dll

ModLoad: 00000000`7d930000 00000000`7da00000 USER32.dll

ModLoad: 00000000`77da0000 00000000`77df2000 SHLWAPI.dll

ModLoad: 00000000`7c8d0000 00000000`7d0d3000 SHELL32.dll

ModLoad: 00000000`73070000 00000000`73097000 WINSPOOL.DRV

ModLoad: 00000000`4b8d0000 00000000`4b921000 MSCTF.dll

ModLoad: 00000000`7df50000 00000000`7dfc0000 UxTheme.dll

ModLoad: 00000000`77d00000 00000000`77d8c000 OLEAUT32.dll

ModLoad: 00000000`77670000 00000000`777a4000 ole32.dll

(8e0.e70): Break instruction exception - code 80000003 (first chance)

ntdll!DbgBreakPoint:

00000000`78ef3320 cc int 3

In the command area at the bottom of the command window enter the following command:

.symfix +c:symcache

This tells the debugger to download needed symbols from Microsoft and to cache them locally in c:symcache so they are not downloaded a second time.

Notepad is currently stopped by the debugger at a breakpoint. Notepad will not respond to user interaction.

In the command area, enter the following command:

g

Notepad should now be responsive again.

From the Format menu of Notepad select "Font". The font dialogue should appear.

Switch back to WinDBG and select "Break" from the Debug menu.

This interupts Notepad at a breakpoint on thread 1 which in this case is the "debugger thread", a special thread that the debugger injects into the debuggee (notepad in this case).

Now enter the following command to switch the debugger context to thread 0:

Thread 0 is the only other thread in this process and is the thread on which the main user interface of Notepad is running. This is the thread that has made a system call to display the Font dialog and is awaiting input.

You should see output like the following:

0:001> ~0s

SYMSRV: +c:symcache*msdl.microsoft.com/download/symbols needs a downstream store

*** ERROR: Symbol file could not be found. Defaulted to export symbols for E:WINDOWSsystem32wow64cpu.dll -

wow64cpu!TurboDispatchJumpAddressEnd+0x684:

00000000`78b842d9 c3 ret

Enter the following command to output the call stack of thread 0:

kp

You should see output like the following:

0:000> kp

Child-SP RetAddr Call Site

00000000`0007ede8 00000000`78b8428e wow64cpu!TurboDispatchJumpAddressEnd+0x684

*** ERROR: Symbol file could not be found. Defaulted to export symbols for E:WINDOWSsystem32wow64.dll -

00000000`0007edf0 00000000`78be6a5a wow64cpu!TurboDispatchJumpAddressEnd+0x639

00000000`0007ee60 00000000`78be5e0d wow64!Wow64SystemServiceEx+0x2ca

00000000`0007ee90 00000000`78ed8501 wow64!Wow64LdrpInitialize+0x2ed

00000000`0007f6c0 00000000`78ed6416 ntdll!LdrpInitializeProcess(struct _CONTEXT * Context = 0x00000000`00000000, void * SystemDllBase = 0x00000000`00000000)+0x17d9

00000000`0007f9d0 00000000`78ef3925 ntdll!_LdrpInitialize(struct _CONTEXT * Context = 0x00000000`00000000, void * SystemArgument1 = 0x00000000`00000000, void * SystemArgument2 = 0x00000000`00000000)+0x18f

00000000`0007fab0 00000000`78d59630 ntdll!KiUserApcDispatch(void)+0x15

00000000`0007ffa8 00000000`00000000 0x78d59630

*** ERROR: Module load completed but symbols could not be loaded for E:WINDOWSsyswow64notepad.exe

00000000`0007ffb0 00000000`00000000 0x0

00000000`0007ffb8 00000000`00000000 0x0

00000000`0007ffc0 00000000`00000000 0x0

00000000`0007ffc8 00000000`00000000 0x0

00000000`0007ffd0 00000000`00000000 0x0

00000000`0007ffd8 00000000`00000000 0x0

00000000`0007ffe0 00000000`00000000 0x0

00000000`0007ffe8 00000000`00000000 0x0

00000000`0007fff0 00000000`00000000 0x0

00000000`0007fff8 00000000`00000000 0x0

00000000`00080000 00000020`78746341 0x0

00000000`00080008 00005370`00000001 0x20`78746341

Note that the addresses in the Child-SP and RetAddr columns are 64-bit addresses, like "00000000`0007ede8"

Note also in the Call Site column that the functions nearest the top of the call stack are functions in the wow64 module and the wow64cpu module. The lower part of the call stack is not correctly resolve due to a lack of symbols in this case.

However that is not important. The main point to observe here is that at a system level, the 32 bit process is actually a 64 bit process that uses the Wow64 layer to accept 32 bit system calls from the program's 32-bit EXE and DLLs and forward on those calls to the 64-bit operating system.

Now enter the following command to list the modules loaded in the process:

lm

You should see output like the following:

0:000> lm

start end module name

00000000`01000000 00000000`01014000 notepad (deferred)

00000000`4b8d0000 00000000`4b921000 MSCTF (deferred)

00000000`73070000 00000000`73097000 WINSPOOL (deferred)

00000000`762b0000 00000000`762fa000 comdlg32 (deferred)

00000000`77670000 00000000`777a4000 ole32 (deferred)

00000000`77ba0000 00000000`77bfa000 msvcrt (deferred)

00000000`77d00000 00000000`77d8c000 OLEAUT32 (deferred)

00000000`77da0000 00000000`77df2000 SHLWAPI (deferred)

00000000`77f50000 00000000`77fec000 ADVAPI32 (deferred)

00000000`78b80000 00000000`78b89000 wow64cpu (export symbols) E:WINDOWSsystem32wow64cpu.dll

00000000`78b90000 00000000`78bda000 wow64win (deferred)

00000000`78be0000 00000000`78c26000 wow64 (deferred)

00000000`78ec0000 00000000`78ff9000 ntdll (deferred)

00000000`7c8d0000 00000000`7d0d3000 SHELL32 (deferred)

00000000`7d4c0000 00000000`7d5f0000 KERNEL32 (deferred)

00000000`7d600000 00000000`7d6f0000 ntdll_7d600000 (deferred)

00000000`7d800000 00000000`7d890000 GDI32 (deferred)

00000000`7d930000 00000000`7da00000 USER32 (deferred)

00000000`7da20000 00000000`7db00000 RPCRT4 (deferred)

00000000`7dbd0000 00000000`7dcd3000 COMCTL32 (deferred)

00000000`7df50000 00000000`7dfc0000 UxTheme (deferred)

Note that there are two copies of NTDLL loaded in the process:

00000000`78ec0000 00000000`78ff9000 ntdll (deferred)

00000000`7d600000 00000000`7d6f0000 ntdll_7d600000 (deferred)

This is quite unusual, to have two copies of a module loaded in the process.

It happens here because we have loaded both the 32 and 64 bit versions of NTDLL.DLL

Exercise (answer at the end of the post):

How can you tell which NTDLL is which?

 

Wow64Exts Debugger Extension

You are now going to use a special debugger extension to look at the 32-bit process as if you were using a 32-bit debugger. Enter the following command to load the debugger extension:

!load wow64exts

Enter the following command to tell the debugger extension to switch to x86 (32-bit) mode:

!sw

You should see output like the following:

0:000> !sw

Switched to x86 mode.

Enter the following command to dump the call stack again. Note it is the same command that you used before:

kp

You should see output similar to the following:

0:000:x86> kp

*** ERROR: Symbol file could not be found. Defaulted to export symbols for USER32.dll -

ChildEBP RetAddr

WARNING: Stack unwind information not available. Following frames may be wrong.

0016fa58 7d957625 USER32!WaitMessage+0x15

0016fa84 7d957b9b USER32!DrawStateW+0x92d

*** ERROR: Symbol file could not be found. Defaulted to export symbols for comdlg32.dll -

0016faa4 762be42a USER32!DialogBoxIndirectParamAorW+0x36

0016fadc 762be305 comdlg32!ChooseFontW+0x14e

0016fb1c 01003100 comdlg32!ChooseFontW+0x29

0016fd94 01003947 notepad+0x3100

0016fdb8 7d9472f8 notepad+0x3947

0016fde4 7d9475e3 USER32!WindowFromDC+0x3b

0016fe5c 7d947816 USER32!WindowFromDC+0x326

0016fed4 7d947858 USER32!WindowFromDC+0x559

0016fee4 01002a32 USER32!DispatchMessageW+0xf

0016ff1c 0100752b notepad+0x2a32

*** ERROR: Symbol file could not be found. Defaulted to export symbols for KERNEL32.dll -

0016ffc0 7d4e6e1a notepad+0x752b

0016fff0 00000000 KERNEL32!BaseProcessInitPostImport+0x8d

Note that in the ChildEBP and RetAddr columns, the addresses are now 32-bit, like "0016fa58".

Note also that the call stack looks very different. In particular the wow64 and wow64cpu module are no longer present.

This demonstrates how, from the point of view of the 32-bit code, the WoW64 layer is transparent, as if it did not exist.

If you attached a 32-bit debugger to this an instance of the 32-bit version of notepad you would see the same call stack.

The wow64exts debugger extension allows you to switch between x86 and x64 (or IA64) modes and debug the 32-bit process with a 64-bit debugger. This could be useful if you are trying to troubleshoot problem with a particular 32-bit process and you suspect the problem are to do with the behaviour of the WoW64 layer within that process.

The debugger extension also has special commands to allow you to do things such as set breakpoints in the 32-bit code.

Reference

WinHEC Presentation about the Wow64 architecture:

download.microsoft.com/download/1/8/f/18f8cee2-0b64-41f2-893d-a6f2295b40c8/DW04040_WINHEC2004.ppt

(Look at slides 5 and 6 in particular)

Cheers

Doug

Answer to the exercise (how can you tell which NTDLL is which):

If you calculate the size you can tell.

For the one listed as ntdll, 78ff9000 is the upper address of the mapped region of the module. 78ec0000 is the lower end. Use calc.exe to work out the difference. Check this against the sizes of the two versions of ntdll.dll on your system. Now check the size of the other one just to confirm.

The one shown as ntdll is the 64-bit one from windowssystem32.

The one shown as ntdll_7d600000 is the 32-bit one from windowssyswow64.

(the fact the debugger appends the _7d600000 to the one from syswow64 indicates it loaded second after the 64-bit one from system32)