How to consume ETW events from C#


In my previous post I explained how to collect ETW events from URL Rewrite (or any other IIS provider) and then display those structured events in the Event Viewer. Now I want to show you how to collect ETW events using C#.

The .NET Framework 3.5 provides a new namespace System.Diagnostics.Eventing.Reader where you can find useful classes for publishing ETW events, but doesn’t provide a mechanism for consuming, so I had to write a class EventTraceWatcher for simplify things.

I want to use this class for tracking, in real time, all the URL Rewrite Events.

Setup Event Trace Session

The first thing to do is to setup the session, open “Reliability and Performance Monitor”, go to Event Trace Sessions and add a new Data Collector Set named “Rewrite”; my previous post has more detailed steps, once you create the collector set, go to its properties and add the provider “IIS: WWW Server” and the Keyword 0x400 (Rewrite) and set the Level to 5. Here is how it should be:

Trace Providers

Use stream mode “Real Time”

By default the Data Collector Set will write the collected events in the file system. Change it from File to “Real Time”. Your .NET Application will be listening those real time events. Open the data collector properties and in the Trace Session tab change this setting.

Trace Session

Make sure to start the Rewrite Collector once you are finish with this settings.

EventTraceWatcher class

The EventTraceWatcher class is very trivial to use, you need to provide the name of the Data Collector Set to it’s constructor (“Rewrite” in our example), hook the event “EventArrived” in your code and then just set the property Enabled to true to start the asynchronous processing of the ETW Events.

using System;
using Microsoft.Iis.Samples.Eventing;

class Program {
   
static void Main() {
       
try {
           
new Program().Run();
       
}
       
catch (Exception ex) {
           
Console.Error.WriteLine(ex);
       
}
   
}

    private void Run() {
       
Guid RewriteProviderId = new Guid(“0469abfa-1bb2-466a-b645-e3e15a02f38b”);

        using (EventTraceWatcher watcher = new EventTraceWatcher(“Rewrite”)) {

            watcher.EventArrived += delegate(object sender, EventArrivedEventArgs e) {

                if (e.EventException != null) {
                   
// Handle the exception
                    Console.Error.WriteLine(e.EventException);
                   
Environment.Exit(-1);
               
}

                // Process only URL Rewrite events
                if (e.ProviderId != RewriteProviderId) {
                   
return;
               
}

                // Dump the event name (e.g. URL_REWRITE_START, ABORT_REQUEST_ACTION, etc).
                Console.WriteLine(“Event Name: ” + e.EventName);

                // Dump properties (e.g. RewriteURL, Pattern, etc).
                foreach (var p in e.Properties) {
                   
Console.WriteLine(“\t” + p.Key + ” — ” + p.Value);
               
}
               
Console.WriteLine();
           
};

            // Start listening
            watcher.Enabled = true;

            // Listen events until user press <Enter>
           
Console.WriteLine(“Press <Enter> to exit”);
           
Console.ReadLine();
       
}
   
}
}

With this code you can write tools to help you to filter in real time information from IIS. Download the EventTraceWatcher class from Visual Studio > NuGet:

PM> Install-Package EventTraceWatcher

https://www.nuget.org/packages/EventTraceWatcher/

The package includes the source code in the ‘src’ folder within the ZIP file:

https://www.nuget.org/api/v2/package/EventTraceWatcher/1.0.0

This class was previously available at code.msdn.microsoft.com

http://code.msdn.microsoft.com/EventTraceWatcher

Comments (16)

  1. logus2k says:

    Daniel,

    Thank you for this article.

    I’m trying to use EventProviderTraceListener to publish to ETW infrastructure and then read those events from managed code. I still couldn’t run your code to test if could use it to a general purpose publisher (instead of only IIS), it always gives me an "access denied" message when I try to run it.

    I’m also having some difficulty understanding what can/should I do programmatically to receive the events from an ETW session )and how can I create one from code).

    I understand there is no current official support to reading ETW events from managed code…? Is there something else I can use?

    Thanks in advance for your help,

    António Cruz

  2. Daniel Vasquez Lopez says:

    I updated the post to add another example that includes support for custom events.

    This is only for Vista/Windows 2008. Make sure to the consumer application with Administrative privileges.

    1. Compile everything.

    2. Run Producer.exe (it uses EventProviderTraceListener)

    3. From command line (as Admin), run ‘logman start MySession -p {13D5F7EF-9404-47ea-AF13-85484F09F2A7} -ets’

    4. Open ‘Reliability and Performance Monitor’ and change the stream mode, in MySession’s properties, from File to Real Time.

    5. Run Consume.exe

    6. Type something in the Producer.exe and press <Enter>

    Expected: Consume[r].exe should echo what you typed in Producer.exe

    When you are done, run ‘logman stop MySession -ets’ to stop the custom ETW session.

    The events produced by EventProviderTraceListener doesn’t have any schema, so what I did was to take the UserData ‘as is’ and convert it to string (be aware that this is just a guess, the data could not be a string).

    Hope this helps

  3. Adam Machanic says:

    Thanks so much for sharing this library!  

    I’m using it to consume ETW output from SQL Server 2008 Extended Events, and have hit a couple of issues:

    A) The event names do not appear to be populating.  If I dump the ETL file they are there, so I assume it would be in the real time output as well?

    B) Any chance it can be extended to be a bit more flexible with regard to non-top level properties?  For example the trace I’m working with right now sends back 11 top level properties and two non-top level properties (I’m not sure of the right term for "non-top level") — these are just Unicode strings, so no difficulties with maps, etc.  I tried to modify the code to consume them and wound up with all sorts of garbage data; I suspect the pointers are not correct…

    I’m happy to send you an example of how to set this up if you’re interested in seeing what I’m talking about.  Drop me a line at amachanic [at] gmail [dot] com … it would be great to get this thing fully working so I hope so :-)

    Either way, thanks again!

  4. walkerc says:

    Hello Daniel,

    I am using the EventTraceWater class to consume KernelTrace Events.

    I backed the class into .Net 2.0 by replacing ReaderWriterLockSlim with ReaderWriterLock.

    The class functions flawlessly on Vista64, on Windows7 (32 bit, build 7000) it … , on XP 32 bit,

    int error = NativeMethods.ProcessTrace(array, 1, IntPtr.Zero, IntPtr.Zero); in ProcessTraceInBackground fails.

    I have not been able to find a documented reason for this failure since ProcessTrace is supported on XP.

    http://msdn.microsoft.com/en-us/library/aa364093(VS.85).aspx

    Any ideas?

    I was also considering unwrapping ProcessTraceInBackground out of a delegate and using the entire class from a background thread in order to simplify the code and perhaps reduce failure modes that way. Your thoughts?

    Again thanks for the class, hopefully this functionality will make it to the framework sooner rather than later.

  5. walkerc says:

    Sorry, I forgot I left a statement hanging:

    on Windows7 (32 bit, build 7000) it …

    I was waiting for the error to occur again to be more specific, the class runs for a while and then throws a NullReference exception, just FYI. I don’t have a dev environment on Win 7 yet to debug.

  6. walkerc says:

    One final question, really I promise.

    I noticed that

    _logFile.EventRecordCallback = EventRecordCallback;

    in StartTracing, was not using a delegate. Would using a delegate here provide some benefit?

    I apologize if this seems ignorant, even though I have been programming for 30 years, my C# is pretty shabby.

  7. walkerc says:

    OK, new years resolution, become more familar with code before commenting on it…

    1) _logFile.EventRecordCallback = EventRecordCallback; is using a delegate.

    2) int error = NativeMethods.ProcessTrace(array, 1, IntPtr.Zero, IntPtr.Zero); in ProcessTraceInBackground  

    blocks.

    It does not matter if EventTraceWatcher.enabled (and in turn, StartTracing and NativeMethods.ProcessTrace is called from a separate external thread or as originally implemented.

    3) ProcessTrace still fails in XP and reason is still a mystery to me.

  8. dixi says:

    An extraordinary and very useful piece of work. Congrats Daniel. I noticed that some people are having troubles with the class in Win7. Actually, I have been troubles with ETW in Win7. It happens that the same .NET 3.5 code I developed works seamlessly in WS2008 but not in Win7. The Beta I gues? Anyway, still trying…

  9. Daniel Vasquez Lopez says:

    Thank you dixi for your comments, I haven’t test it in Windows 7, but I will give it a try as soon as I can.

    WalkerC, unfortunately this code won’t work for XP, I’m using the field ProcessTraceMode and callback EventRecordCallback http://msdn.microsoft.com/en-us/library/aa363780(VS.85).aspx that are not available prior Windows Vista.

    Thank you.

  10. Eok's Blog says:

    Event Tracing for Windows (ETW) is wonderful mechanism to monitor, log and trouble shoot of your application

  11. jamlou2 says:

    really great tool, much appreciated.

    i wanted to go further than that, what should be done so that this library would also consume WPP messages sent through DoTraceMessage()

    considering the event session is already created, and there is an application running and sending WPP messages, what should be changed in the implementation so that it would successfully consume those messages?

  12. Mukesh Kumar says:

    Hi Daniel,

    I am getting the following error while starting the consumer.exe on Windows 7 machine:

    E:EventTraceWatcherConsumerbinDebug>Consumer.exe

    System.ComponentModel.Win32Exception: The filename, directory name, or volume la

    bel syntax is incorrect

      at Samples.Eventing.EventTraceWatcher.StartTracing() in E:EventTraceWatcher

    Samples.EventingEventTraceWatcher.cs:line 252

      at Samples.Eventing.EventTraceWatcher.SetEnabled(Boolean value) in E:EventTr

    aceWatcherSamples.EventingEventTraceWatcher.cs:line 173

      at Samples.Eventing.EventTraceWatcher.Start() in E:EventTraceWatcherSamples

    .EventingEventTraceWatcher.cs:line 183

      at Program.Run() in E:EventTraceWatcherConsumerProgram.cs:line 38

      at Program.Main() in E:EventTraceWatcherConsumerProgram.cs:line 11

    Can somebody let me know what am i missing here?

  13. Witali says:

    Hi Daniel,

    many thanks for such a great work.

    I am currently trying to use your library for consuming real-time events generated by the Microsoft-Windows-Offline provider.

    While everything goes well with normal informational events (event id 2002 for instance), I cannot properly read warnings, that is event id 2006.  Unfortunately, the UserData section somehow gets cropped and I only get to see a small portion of it, such as "ss denied" as opposed to "Access denied" error message along with additional fields <ResultCode>, <Operations> to name but a few.

    Have you got any idea why this is happening?

    Thank you very much in advance for your help.

    Kind regards,

    Witali

  14. Eamonn J. Casey says:

    Any chance you could post some code that finds the machine & user name? I think that it is in the UserContext IntPtr, but I can't find how large it is and how to parse it.

    Thanks,

    Eamonn J.

  15. Inopia says:

    You use eventRecord.EventHeader.EventDescriptor.Opcode as a key to cache TraceEventInfoWrapper instances. This breaks for a lot of providers (i.e. microsoft-windows-wininet). I had to bypass this caching feature to make it work for me.

  16. Hi Daniel,

    I want to get the 'UserData' section of Ax Dynamics event and do not know where I can get the event format for that section. Can you help with some c# code to get UserData?