Using Kudu to capture First-Chance dumps for Windows Azure Web Sites


 

Last time, I introduced about how to use Kudu to capture a hang dump. This time, using the feature provided by Kudu, we can capture First-Chance dump for worker process which is very helpful for analysis application crash/exception issues.

In this article, I will show you how to get the dump step by step. I use a slow web site hosted at http://hk1test.azurewebsites.net/ as a sample.

Upload the tools

Upload ProcDump

       1. Download ProcDump tool from Microsoft site, and extract the tool from the zip package.

ProcDump

http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx

 

2.     2.  Logon to the site FTP and upload the tool. For example, I created a folder named dump and put the tool into the dump folder.

Upload AdPlus

1. Download and install Debugging Tools for Windows

ProcDump generates a dump for every first chance exception. This is not only a performance overhead. For Windows Azure Websites scenario, this can eat up all the disk quota easily.

ProcDump has an option -f to filter the exception. Another option -n to limit the number of dumps generated. However, this is still not enough if there are too many first chance exception.

Debugging Tools for Windows, which allows us control on the behavior after first chance exception happened. The tool can be download here.

 http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx

2.      2.  Upload the tool

After installed Debugging Tools for Windows on your local machine, please go to the folder where the tool was installed. And upload follow binaries:

·         Adplus.exe

·         Adplusext.dll

·         Cdb.exe

·         Dbgeng.dll

·         Dbghelp.dll

 

By default, Websites runs a 32bit W3WP.EXE on 64bit system. Using a 32bit adplus.exe is recommended. If you are running site with 64bit process (Available for standard mode, can be enabled on the portal.), you must upload 64bit version of binaries listed above.

 

Open the Kudu Console

        1.  Open IE and browser to the SCM site, for this sample, it is https://hk1test.scm.azurewebsites.net

2.       2. And you will see a credential prompt, please give your deployment user and password.

a.       You can find the deployment user on the portal. For my site it is wzhaoFtpUser(not case sensitive)

b.      Key in the user name and password, like this.

 

3.       3. Now, you should see the Kudu site, like this

    4. Click the “Launch” to open the Kudu Diagnostics Console.

Capture a first chance dump using ProcDump

   1.  Run “powershell.exe -Command Get-Process” to list the process (es). Here is the sample output. You should able see php-cgi.exe or node.exe if you are running PHP or node.js application. The process ID for w3wp.exe is 21680.

2.       2. Go to the folder where ProcDump/adplus.exe was uploaded to. For this sample, it is “site\wwwroot\debugger”. Select the tool which is suitable for your scenario.

 

3.       3. Using ProcDump to capture the first chance exception dump

Run “ProcDump –accepteula –e 1 -ma 21680” to create the dump whenever a first chance exception occurred. You will see the dump was created.

C:\DWASFiles\Sites\HK1Test\VirtualDirectory0\site\wwwroot\debugger>procdump -accepteula -e 1 -mp 21680

ProcDump v6.00 - Writes process dump files

Copyright (C) 2009-2013 Mark Russinovich

Sysinternals - www.sysinternals.com

With contributions from Andrew Richards

 

Process:               w3wp.exe (21680)

CPU threshold:         n/a

Performance counter:   n/a

Commit threshold:      n/a

Threshold seconds:     n/a

Number of dumps:       1

Hung window check:     Disabled

Exception monitor:     First Chance+Unhandled

Exception filter:      *

Terminate monitor:     Disabled

Dump file:             C:\DWASFiles\Sites\HK1Test\VirtualDirectory0\site\wwwroot\debugger\w3wp_YYMMDD_HHMMSS.dmp

 

 

Press Ctrl-C to end monitoring without terminating the process.

 

CLR Version: v4.0.30319

 

[09:20:02] Exception: E0434F4D.System.ArgumentNullException ("Value cannot be null.")

 

First-chance Exception.

Writing dump file C:\DWASFiles\Sites\HK1Test\VirtualDirectory0\site\wwwroot\debugger\w3wp_131105_092002.dmp ...

Dump written.

 

Dump count reached.

Note

1.     Here are Procdump options related

-e Write a dump when the process encounters an unhandled exception. Include the 1 to create dump on first chance exceptions.

-f Filter the first chance exceptions. Wildcards (*) are supported. To just display the names without dumping, use a blank ("") filter.

-g Only capture native exceptions in a managed process (no interop).

-ma Write a dump file with all process memory. The default dump format only includes thread and handle information.

-mp Write a dump file with thread and handle information, and all read/write process memory. To minimize dump size, memory areas larger than 512MB are searched for, and if found, the largest area is excluded. A memory area is the collection of same sized memory allocation areas. The removal of this (cache) memory reduces Exchange and SQL Server dumps by over 90%.

-n Number of dumps to write before exiting.

-o Overwrite an existing dump file.

More control on first chance exception using AdPlus

There are many scenarios which you don’t want a dump for first chance exception. Instead, a call stack for the exception is enough. Compare to create a dump, this saves disk space and also performance overhead.

The follow sample demos how to log call stacks when a first chance exception occurred. If it is a CLR exception, the managed stack will be logged as well.

1.       1. Create a configuration file with follow content and upload to the debugger folder. For this sample, I named it antares.cfg.

<ADPlus Version='2'>

    <!-- Configuring ADPlus to log all first chance exceptions -->

    <!--  -->

<Settings>

      <RunMode> CRASH </RunMode>

      <OutputDir> c:\debuggers </OutputDir>

      <Option> NoDumpOnFirst </Option>

      <OutputDir> .\ </OutputDir>

</Settings>

 

<!-- For .Net 2.0, change to ".loadby sos mscorwks" -->

 

<KeyWords>

   <KeyWord Name="LoadClrExt"> .loadby sos clr </KeyWord>

   <KeyWord Name="ClrStack"> !clrstack </KeyWord>

</KeyWords>

 

<PreCommands>

     <DebugActions> LoadClrExt </DebugActions>

</PreCommands>

     

<Exceptions>

     <AllExceptions>

       <Actions1> Log;stack </Actions1>

     </AllExceptions>

 

     <Exception Code='clr'>

                     <Actions1> Log;stack;ClrStack </Actions1>

                     <ReturnAction1> GN </ReturnAction1>

     </Exception>

</Exceptions>

 

</ADPlus>

 

2.      2. In the Kudu Console, Run “Powershell.exe -Command Get-Process” to get the PID of W3WP.EXE. For this demo, it is 17128.

3.      3.  Run the adplus.exe with follow arguments.

adplus.exe -c antares.cfg -p 17128

       4. Now, folder with current time will be created under the debugger folder.

5.       5. Go to the folder highlighted, and there will be a log file named like ADPlus_log_2354_2013-11-06_01-52-10-149.log. This log contains the exception info. After you reproduced the problem, you can find the information interested in this file.

6.      6.  Here is a sample exception stack in the log file.

 

Reference:

Kudu

https://github.com/projectkudu/kudu/wiki/Process-list-and-minidump

Dump Type

http://msdn.microsoft.com/en-us/library/windows/desktop/ms680519.aspx

ProcDump

http://technet.microsoft.com/en-us/sysinternals/dd996900.aspx

Debugging Tools for Windows

http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx

 

See you next time,

Wei

Comments (0)

Skip to main content