CloudFx Tracing and Diagnostics

Today I'm investigating how CloudFx tracing and diagnostics works.  I want to control the verbosity of output.  I found an answer from Valery M here to a question titled "Annoying diagnostics traces for queue storage", which got me started.

As of the CloudFx 2.0.0.0 release on NuGet, there is a class called TraceManager which contains the entry point into the CloudFx's tracing components.  TraceManager is a static class which contains a cache of trace providers which implement CloudFx's ITraceEventProvider interface.

CloudFx provides an ITraceEventProvider implementation called HybridTraceEventProvider.  This provider contains logic in its constructor that knows if it is running in Azure, the Compute Emulator, or outside of the cloud environment, as is done during unit testing.  The class's documentation describes how it works so I will not repeat it here.

While deployed in Azure or the Emulator, the verbosity configured for Windows Azure Diagnostics (WAD) controls which events are transferred to Azure Storage.  I use the diagnostics.wadcfg file to control the level via the scheduledTransferLogLevelFilter value in the Logs element, as in:  

   <!-- Configure the capture and persistence of basic log from the WAD trace listener-->
  <Logs bufferQuotaInMB="100" scheduledTransferLogLevelFilter="Verbose" scheduledTransferPeriod="PT5M" />

While unit testing a switch in the app.config file named CloudFx.Diagnostics.DefaultTraceSwitch controls verbosity, as in:

   <system.diagnostics>
    <switches>
      <add name="CloudFx.Diagnostics.DefaultTraceSwitch" value="Info" />
    </switches>
  </system.diagnostics>

The HybridTraceEventProvider code determines where events are traced and additional elements in <system.diagnostics>, such as an additional listener, are ignored.

As far as how CloudFx uses HybridTraceEventProvider, TraceManager creates ten different HybridTraceEventProvider instances; each pertains to a different area within the CloudFx code.  When the CloudFx code needs to trace it makes a call like TraceManager.CloudServiceComponent.TraceError(ex).

It is interesting that ITraceEventProvider derives from both IObservable<TraceEvent> and ISubject<Exception>.  This is a significant departure from normal tracing classes.  This means that ITraceEventProvider implementations can participate in Reactive Extensions (Rx) expressions.

From the looks of the HybridTraceEventProvider code, all tracing is observable by subscribing to the provider via its IObservable<TraceEvent> implementation.  This includes exceptions traced through the TraceError() overloads.

The same exceptions that flow through the IObservable<TraceEvent> implementation are also observable by subscribing to the provider's ISubject<Exception> implementation.  Additionally, the ISubject<Exception> methods are public so code outside of the ITraceEventProvider implementation could submit exception instances to the stream; I don't see a reason to do that though.  I talked to the CloudFx team and they might change the ISubject<Exception> aspect of the code in the future.

Overall CloudFx contains a nice reusable diagnostic class which supports Rx.  I look forward to seeing how it can be combines with EntLib's Semantic Logging.