Hi, welcome back,
I’ve been dealing these days with an issue about a Custom Authentication Package which was crashing LSASS.exe process even before we had the opportunity to log on the machine. So, how can I debug the package/LSASS process with my favorite debugger, WinDbg (Debugging Tools for Windows), to know what’s going on there?
To make things easier I use two machines: a Windows XP machine running on Virtual PC and my Windows Vista machine. Virtual PC makes my life much easier as I can recover the machine very easily if I “break” it. It also makes kernel debugging easier as I don’t need any cables to connect the machines involved. The target package/LSASS process will run on WinXP, and WinDbg will run on Vista.
1) Let’s configure LSASS on WinXP to load and use the problematic package (i.e. MyPackage.dll):
1.1) Copy MyPackage.dll to %SystemRoot%\system32.
1.2) Add “MyPackage” to the Authentication Packages list in the \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa registry key.
2) Let’s configure WinXP to run LSASS under a user mode debugger like NTSD.exe (Debugging Tools for Windows). We won’t be able to use this debugger directly, but we will be able to use it through WinDbg working as kernel mode debugger on Vista.
2.a) Set Debugger value to REG_SZ “ntsd -d” in HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\lsass.exe.
2.b) Instead of manipulating the registry directly, we may use GFlags.exe (Debugging Tools for Windows) to make these changes: go to Image File tab, enter “LSASS.exe” in Image field, press TAB to refresh, click on Debugger checkbox and enter “ntsd -d” as the debugger.
After these changes NTSD will break in LSASS execution just when LSASS is about to start (thx to -d parameter), so we can debug it from the very beginning of its execution. Now we’ll need a kernel debugger (WinDbg in our case) to control NTSD remotely.
3) Let’s configure WinXP to start on debug mode and be able to attach a kernel debugger to it:
3.1) Run msconfig.exe (System Configuration utility), go to BOOT.INI tab, click on Advanced Options… button, and select /DEBUG, select & set /DEBUGPORT= to COM1:, and select & set /BAUDRATE= to 115200.
3.2) On Virtual PC go to Edit > Settings menu option, select COM1 and set it to a Named pipe called i.e. \\.\pipe\machinename.
Now our WinXP is prepared to be debugged. We can restart it now. We will see that WinXP hangs before showing the logon screen. NTSD has broken in LSASS, and because LSASS is not executing, WinXP won’t start either. NTSD is now waiting for our debugging commands, but we will need WinDbg to send those commands to it.
4) Let’s configure WinDbg on Vista to do kernel debugging on WinXP:
On WinDbg go to File > Kernel Debug… menu option, select COM tab, check Pipe checkbox, set Baud Rate to 115200 and Port to \\.\pipe\machinename (the same we used on Virtual PC). When we click OK WinDbg will connect to WinXP as its kernel debugger.
When WinDbg connects to WinXP and WinLogon is about to start, WinDbg shows the prompt of NTSD (“Input>“). We will now be able to send commands to NTSD, like bp to set a breakpoint, etc.
Please take a look to “Controlling the User-Mode Debugger from the Kernel Debugger” topic on WinDbg’s help for more info on how to control NTSD from WinDbg.
We will now be able to do user mode (thx to NTSD) and kernel mode debugging at the same time with WinDbg.
Note: I had some issues to load the right symbols (.pdb files), because I couldn’t set .sympath on NTSD to c:\symbols, for instance. So I did the following: I took a dump of LSASS process with .dump command, I opened the dump and loaded all the symbols I needed (i.e. lsasrv.pdb, lsass.pdb, kernel32.pdb, ntdll.pdb and MyPackage.pdb). I copied all those symbols to System32 folder on XP, and after that NTSD was able to find them without setting its .sympath.
Note: We may debug CSRSS and WinLogon processes following the same principles. Take a look to “Debugging CSRSS with NTSD” and “Debugging WinLogon with NTSD” topics on WinDbg’s help.
I hope this helps.