While running Customer’s .NET/PHP/ Node processes in Azure Web App environment, it may intermittently crashes due to code or performance issues. It’s important to capture the crash dump when such crash/exception happen automatically for further investigation.
This article is to introduce the new CrashDiag Site Extension, which can easily help us to capture the necessary data when intermittent unhandled exception happens. Now it can work for these scenarios:
Capture Dump for Exceptions
1. Unhandled Exception happening
2. Specific first chance exception happening based on filtering parameter
3. Monitoring new launched processes continuously
4. Configure total dump file numbers and debugger process instances
5. Native App, Managed App, Web Job with 32bit/64bit
6. Safe Stop at any time without terminating target process
Capture Dump for Performance (slow, high CPU, high Memory)
1. Capture Hang Dump on Demand
To capture dump for exceptions, follow below steps:
2. Configure (1st chance or 2nd chance exception handling):
1. Open Azure Portal, navigate to App Services -> your Web App -> Tools -> Extensions.
2. Click Add, and choose Crash Diagnoser to install it.
3. Please make sure the web app configuration is “AlwaysOn” because the Crash Diagnoser runs as a Continuous web job. Refer to: http://blog.amitapple.com/post/73574681678/git-deploy-console-app/#.VpcHaPl96Ul
1. After installation, select it from the Installed Web App Extension blade, click Browse:
2. Now we can configure it based on our troubleshooting scenarios (1st chance or 2nd chance exception handling):
2.1 Application Crashed (unexpected terminated due to 2nd chance unhandled exception)
2.1.1 Click the 2nd Chance Unhandled Exception tab, choose your target process name. The default options include w3wp, node, php-cgi.
If you want to monitor other process, such as mywebjob.exe, just type the process name mywebjob manually in the field.
2.2 Application didn’t crash, but has specific exceptions generated. We need to filter 1st chance exception and perform analysis.
2.2.1 Click the 1st Chance Exception tab, set the exception code and process name.
Besides the default options of exception code, you can type other specific code which occurred on the application. For example, if the application reported FileNotFound exception, you can type FileNotFound or NotFound in the Exception Code field.
To monitor all 1st chance exceptions, type *
But this may generate other kind of dump files other than you expected.
3. Click the Advanced Settings tab to make sure the settings are properly for you. The current default setting are suitable for common scenarios, you can tune them as you wish.
Consider one dump can be hundreds MB size, you may want to set the Maximum dump file number less than 10.
1. After configuration, click the Start button to start monitoring. If your application is running, around 10 seconds, you will see the CrashDiag status changed from “Stopped on Instance name” to “Monitoring Processes on Instance name”:
2. If the application crashed, you will see the *.dmp files are generated in the output path. In this sample, it is in d:\home\logfiles\crashdiag. Then You can open Kudu portal and further investigate the dump file.
1. After troubleshooting, click SafeStop to stop the Crash Diagnoser. This Stop action will not terminate the target process and can be executed at any time.
If the Instance is running a Crash Diag, and you click Start again, the portal will reminder you to safe stop it first:
To capture dump for performance issues, follow below steps:
1. Click the Process tab, click Load List to show current running processes
2. Click FullDump, the dump will be created and shows in the Dump File List section
Question & Answer
1. After install the Site Extension, will the Web App be restarted?
Yes. It will be restarted as applicationhost.xdt is used in the package. Without restarting, the route to CrashDiag cannot be found.
2. Can it analyze dump automatically?
Not Yet. Its current purpose is to capture intermittent exception data as dump files for root cause analysis.
3. If I set target process as W3WP, will it monitor the Kudu site W3WP process as well?
No. It will not capture processes which is running for Kudu, DaaSConsole, and DaaSRunner
4. If the site extension is updated, will it be auto-updated?
Yes. It will be auto-updated. A popup message will remind you for this:
Your web app will not be restarted during the auto-update.
5. If I don’t know which kind of exception I’m facing, how to configure it?
Please check if there is Process Terminated or Exit logs in the D:\home\LogFiles\eventlog.xml when the issue happened. If yes, then can use 2nd chance exception configure as above.
Normally 1st chance exception will not cause application crash. If you need to capture dump for it, please understand the performance impact on the target process can be a little higher than the 2nd chance exception capture. This is because the debugger needs to filer all 1st exceptions happen in the target process.
6. Will it monitor all Web Instances?
Through this UI, it will only monitor the current Instance in the current Kudu site (https://appname.scm.azurewebsites.net/crashdiag) instead of all instances.
The SafeStop option will stop all monitoring tasks on all instances.
7. How to download the dump files?
Click the file hyper link in the Dump File List section.
8. How to Analyze the dump files?
Start from V1.3, you can click the Analyze button directly in Crash Diagnoser and perform online analysis.
More Information Regarding Crash Diagnoser
-Freist Li from APGC DSI Team