WAD is Built on ETW

Windows Azure Diagnostics (WAD) is built upon Event Tracing for Windows (ETW).  To get the most out of WAD and future Azure product offerings you need to understand and use ETW.  Here is a little ETW primer and how it relates to WAD:

1. ETW is the efficient kernel-level tracing facility built into Windows.  Microsoft products leverage ETW extensively including Windows Azure Diagnostics.   Here is ETW’s architecture:

[Image from https://msdn.microsoft.com/en-us/library/aa363668(v=VS.85).aspx]

2. .NET has a legacy tracing capability from .NET 1.1 within the System.Diagnostics namespace implemented in the Trace, Debug, Switch, TraceListener, and derived classes. This legacy capability was built without ETW integration and has some performance issues.   Do not use Trace or Debug for tracing; use one of the newer approaches described below.

3. To improve tracing performance .NET 2.0 enhanced the Diagnostics namespace by adding the TraceSource and SourceSwitch classes.

4. To integrate .NET tracing with ETW, .NET 3.5 added the System.Diagnostics.Eventing namespace with the EventProvider and EventProviderTraceListener classes. The EventProvider class is an implementation of an ETW Provider shown in the diagram above.  The EventProviderTraceListener is the gateway from .NET tracing to the EventProvider class and hence ETW’s efficient and high performance tracing.

5. .NET 4.5 added the System.Diagnostic.Tracing namespace.  Classes derived from its EventSource and EventListener classes are now preferred for tracing.  Vance Morrison's blog has excellent articles about developing with these new classes.  EventSource is now the gateway from .NET to ETW so use EventSource instead of TraceSource or EventProvider.

Note:  For codebases prior to .NET 4.5, one of Vance Morrison's blog posts contains EventSource.dll which provides a 'stand-alone' implementation of the EventSource class.  He says the dll provides an easy way to use the new approach with older code then simply unreferenced the dll when you upgrade to .NET 4.5.

6. Here is WAD’s architecture:

[Image from https://msdn.microsoft.com/en-us/magazine/ff714589.aspx; a must read article]

7. The yellow and red portions of the WAD diagram are the .NET tracing discussed in #2 and #3 above. The Microsoft product teams have already added many trace sources within their framework code and you should do the same within your code.  The recommendation is to create at least one unique EventSource derived class per application or assembly.

8. The black arrows coming out of the red trace source portions of the WAD diagram are configured in web.config / app.config. The arrows go to trace listeners.  Out of the box .NET provides around ten trace listeners.  Enterprise Library adds a dozen more in its Logging.TraceListeners and Logging.Data namespaces.  The various trace listeners have different performance characteristics so pick wisely given your requirements and prefer ones that use ETW.

Note that these trace listeners are different than the .NET 4.5 introduced System.Diagnostics.Tracing.EventListener class.

9. WAD implements a trace listener called DiagnosticMonitorTraceListener which is shown in the purple box on the diagram above.  The listener writes events to WADLogsTable in Azure Table Storage.  DiagnosticsMonitorTraceListener is typically wired up in the web.config / app.config as show:

   <system.diagnostics>

   <sharedListeners>

   <add name="azureDiagnosticsTraceListener" type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">

   <filter type="" />

   </add>

   </sharedListeners>

   <sources>

   <!-- TraceSource.* related settings -->

   <source name="WAD_Example" switchName="logSwitch">

   <listeners>

   <add name="azureDiagnosticsTraceListener" />

   </listeners>

   </source>

   </sources>

   <switches>

   <add name="logSwitch" value="Information" />

   <!-- see System.Diagnostis.SourceLevels -->

   </switches>

   <trace>

   <listeners>

   <add name="azureDiagnosticsTraceListener" />

   </listeners>

   </trace>

   </system.diagnostics>

10. Additionally WAD can read from the Windows Event Logs, event counters, custom files, and ETW directly.  WAD is one of the ETW Consumers shown in the diagram in #1 above.

11. WAD uses an xpath type expression, which you specify, to filter the Windows Event Logs prior to WAD writing the events to WADWindowsEventLogsTable in Azure Table Storage.  If you need to write to the Windows Event Log, the preferred approach is to add an EventLogTraceListener to the config shown above instead of writing to it directly using EventLog.WriteEntry().

For more posts in my WAD series:

https://blogs.msdn.com/b/davidhardin/archive/tags/wad/