Version 1.1.24 of the EventSource NuGet Package marked as STABLE


As some of you may know, since last August, there has been a Prelease version of EventSource 1.1 available on Nuget.   It has been stable for the last 5 months or so, so we have gone ahead and marked the 1.1.24 version as stable.

For those of you unfamiliar with EventSource, it is a logging system built into the .NET Framework.   You can learn more about it in my other blog posts about it

As a reminder, however, there are TWO versions of EventSource

  1. System.Diagnostics.Tracing.EventSource – This is built into the .NET Framework, and you should use it whenever you can.   
  2. Microsoft.Diagnsotics.Tracing.EvenceSource – This is what is implemented in the Nuget Package.   It is effectively a stand-alone version of the code built into the .Net Framework.    Its only purpose is to allow you to use new features (like the ones I am about to describe) on OLDER version so the .NET framework.    Everything in V1.1 of this package (what was just released) is available in the V4.6 version of the .NET Framework (which is the version that Windows 10 includes).   Thus if your application only needs to run on windows 10, then you don’t need this package.   

Version 1.1 of the Nuget package (or 4.6 of the framework) has the following new features:

  • DynamicEvents – Allows events defined ‘on the fly’ by without creating a ‘event methods
  • RichPayloads – Allows specially attributed classes and arrays as well as primitive types to be passed as a payload.
  • ActivityTracking – Causes Start and Stop events to tag events between them with ID that represents all currently active activities.

I will be blogging about each of these in the near future, as each of the three new features are complex enough that some explanation and guidance is needed.     

 

Vance

Comments (40)

  1. Martin Liversage says:

    I am happy to see your continued effort to update this package. However, I just upgraded to 1.1.24 in my project and had a small issue. One of my events was simply named "Start" and with the new activity tracking functionality events having the prefix "Start" and "Stop" gets special handling. In my case there was nothing after the prefix "Start" so I got an ETW manifest compilation error: "MSXML Schema Validation Error 0xc00ce174. At Line=14, Column=58, Value cannot be empty.". Changing the name from "Start" to something different fixed the problem.

  2. @Martin Liversage:  you can also use the Event(,ActivityOptions=EventActivityOptions.Disable) to force any event to not have special handling (however in general, it is probably better to simply rename the event so that it does not collide with conventions).

  3. schlumpfmarkus says:

    Hi Vance, thanx for your effort and the update. I am curious to give the new features a try…

    I also had a small issue, after I updated my projects to use 1.1.24. As I always use the EventSource attribute to provide a good name, I simply use the class name 'EventSource' for my event sources. This was no problem with 1.0.26, but the WriteEvent method of current version throws an IndexOutOfBoundException,  when the event source has the class name 'EventSource'. Funny detail, right!?

    Maybe the EventRegister tool should check for this naming issue?

    Anyway, thanx for the update and your great work!

    Kind regards,

    Markus

  4. @Markus

    This sounds like a bug.   'EventSource' is a legal name (not a good one, but a legal one).  It is likely something else about your EventSource that is triggering the exception.    If you could send me a stack trace to vancem@microsoft.com I may be able to diagnose it from that.

  5. schlumpfmarkus says:

    No problem. I just sent the stack trace to vancem@microsoft.com.

    Hope that helps! Have a nice weekend,

    Markus

  6. schlumpfmarkus says:

    A short question regarding the new dynamic events: Is there a way to provide a channel, so that they can appear in the Windows event log?

    Thanx,

    Markus

  7. @Markus:  Channels are not supported by dynamic events at this time.  

  8. schlumpfmarkus says:

    Another question regarding EventListener: Is it by design, that enabling a source with a certain keyword will also accept events without any keyword!? I expected that only matching keywords would pass…

    Thanx,

    Markus

  9. ranta says:

    @Markus: I guess it's related to this sentence in the documentation of EnableTraceEx <msdn.microsoft.com/…/aa363711%28v=vs.85%29.aspx>: "If an event's keyword is zero, the provider will write the event to the session, regardless of the MatchAnyKeyword and MatchAllKeyword masks."

  10. @Markus  As ranta indicates the keyword value of 0 when attached to an event means 'ignore keyword mask'.   EventSource inherited this from ETW.   If you think about it, the 'natural' default for 0 would be 'never fire' which is not very useful (and given that 0 is the natural default, a major stumbling block).  

    Note that 0 also means something special when ETW controllers use it.  Technically speaking is means 'provider default' but in practice it means 'all keywords'.   I don't recommend relying on this however (if you want all keywords, send -1 as the keyword vector.  

  11. schlumpfmarkus says:

    @Vance @ranta: Thanx for the good explanations! This does makes sense to me, as on a flagged enum, the HasFlag() method returns always true, if the flags you ask for are zero. So this is always true: keywords.HasFlag((EventKeywords) 0)

    Saying that, I found that the behavior is not consistent: EventListeners using EventSources WITHOUT channels act as described above. Strange but true: EventListeners using EventSources WITH channels do also filter out events without any keyword!

    My suspicion is, that this is due to the fact, that channels are internally somehow added to the keyword value (if  I interpreted the source code right).

    From the viewpoint of the user of the APIs, this is not a good thing. My question now is: Can I rely on this (strange) behavior of events using channels? Then it would be OK for me, as we always use channels anyway…

    Thanx, Markus

  12. ranta says:

    EventRegister 1.1.25 still doesn't let me define multiple event methods with the same EventAttribute.EventId but different EventAttribute.Version. I need to preserve the old versions of event definitions so that Event Viewer can use the most recent resources to display events logged by previous versions of the software. I have worked around this limitation by assigning new IDs to the new versions, but changing event IDs in this way will cause difficulty if anyone ever wants to define triggers or custom views that refer to the event IDs.

    In the next service I implement, I think it will be easiest to edit the manifest for Event Viewer as XML and integrate mc.exe -css to the build system, instead of using EventRegister.  I might still use the EventRegister and EventSource packages for debug output, though.

  13. @ranta – The goal of EventAttribute.Version is NOT to allow you to have multiple versions of an event in the same version of your code.  It is only there so that you can DECLARE what version the event is to anyone consuming the event.    Thus for any given event ID there should be at most one method.  As you add new arguments (ONLY TO THE END OF THE METHOD), you can increment the version number so that clients of the event know the version is different (bumping the version is not STRICTLY needed as as clients can 'tell' by the fact that there are more arguments, but it is good practice).

    I am assuming you know about using EventRegister being used in the build to generate the manifest you need to register.  You are of course free to create your build system in any way you like including, considering the manifest 'source' (you have to be very careful to keep the EventSource and the manifest in sync).  I never do this myself, but it is your code base…  

  14. ranta says:

    In an earlier project, I did use mc.exe and have multiple versions of the same event ID.  The software only emitted the latest version, but I kept the earlier ones in the manifest.  The earlier versions had event/@notLogged="true", and they did not have any event/@symbol, so mc.exe did not generate any C# code for them.  Because mc.exe generated a C# class for emitting the latest versions of events, there was never any risk of the C# going out of sync with the manifest.

    mc.exe generated the following message IDs in the MESSAGETABLE resource, among others:

    – 0xb0000001 from provider[1]/events/event[@value="1" and not(@version)]

    – 0xb0010001 from provider[1]/events/event[@value="1" and @version="1"]

    – 0xb0000002 from provider[1]/events/event[@value="2" and not(@version)]

    – 0xb0010002 from provider[1]/events/event[@value="2" and @version="1"]

    – 0xb1000001 from provider[2]/events/event[@value="1" and not(@version)]

    – 0xb1010001 from provider[2]/events/event[@value="1" and @version="1"]

    So, it clearly supports having multiple versions of the same event in the same manifest and defining a separate format string for each version.

    What you write about adding parameters only to the end makes sense, but I don't see how that would disallow having multiple versions in the same manifest.  On the contrary, when old versions are preserved there, tools could easily verify that the templates are indeed backward compatible.

    Anyway, how would you support multiple versions in Event Viewer then?  Would you change the event ID in each version?

  15. @rata.   The simple answer is that EventSource is not the same as ETW/Window Event Log.   While EventSource was designed to minimize the semantic mismatch between it and ETW, there was never the goal to expose the underlying functionality (and complexity), of the windows logging system.  

    In a common case, when your EventSource only exists in a single component, you never need to have multiple versions of an event in your manifest.   Either the provider was registered, (in which case it there is only one version on the system, and the manifest declares it and it is accurate), or it is not even present (since it is not installed).  

    Multiple version in the manifest only make sense if there is the possibility of different version of the code existing on the same machine both of which are logging events.   This implies two different installers trying to register the manifest.  Since only the newer version can possibly create a manifest that describes the old and new events, the installers must carefully determine whether their manifest is the most complete before proceeding.  You can see that this is subtle, and arguably a bad idea (each of the users of your module should have a MSI for YOUR ONE VERSION and again you are in the 'good case' were there can't be two versions fo your code on the same machine simultaneously.    

    If you in this rare case of building a component that ships COPIES in multiple, independently installed components (and you actually trust those components to do the above installation correctly) AND you don't wish to simply use the scheme of using new EventIDs (you discovered above), then you are likely to have to keep a 'historicalManifest' as source code and update when you add to your EventSource.  

  16. ranta says:

    Yes, we run different versions of the same software side by side on the same servers.  But these servers are ours; we do not distribute this software to other parties.  So although the installation is a bit tricky, it hasn't been a real problem yet.

    When we used the older Event Logging API, we could easily define multiple event sources in the Registry and point them to copies of the same EventMessageFile, which we could then upgrade independently of each other.  Can we do anything like that with Windows Event Log: edit the manifest at install time so that it wevtutil will install it as a separate instance, and then just pass the correct GUID to the API?  I suspect it isn't that easy, because the WEVT_TEMPLATE resource includes the names of the channels and would presumably have to be updated too.  Because the format of the resource is undocumented, the installer would then have to run mc.exe, which might lead to license problems.

  17. ranta says:

    With EventRegister 1.1.25 on .NET Framework 4.5.2 and Windows 7 SP1, the generated manifest contains a localization/resources element for only one culture, even though there is a resource assembly for another culture and the manifest should thus contain two localization/resources elements.  I have found two workarounds:

    (a) Change the build system to copy the resource assembly to a culture-specific subdirectory of the directory that contains eventRegister.exe.  This workaround may cause problems if the same project is built in parallel for multiple configurations or platforms.

    (b) Set the environment variable "COMPLUS_relativeBindForResources=1".  This workaround seems a bit harder to integrate to the build system than workaround (a).

    I had the same problem with EventRegister 1.0.26 on .NET Framework 4.0.  Workaround (a) worked then too.  I did not try workaround (b) then.

    I also tried adding the resource assembly to the -ReferencePath option of eventRegister.exe, but that did not help.

  18. @randa 6/17 issue:   My general recommendation if you have control over your system is to not run with different versions of the same EventSource simultaneously.    You should have an EventSource for every independently versioned component on your system.   Thus if things version at different rates, you should have an EventSource for each.   Now versioning gets very simple because there is no sharing (or potential for conflict).    That said, in the end, EventSource mostly just calls the EventWrite API, and mostly does not care about the manifest.  Thus you have the opportunity to hack the manifest before you register it.   With sufficient care this could probably be made to work, but again I don't advise it because frankly it is complex, fragile and subtle, and thus likely to break in ways that your support group will find very frustrating.    It is MUCH better if you can avoid the sharing (and thus the potential for conflict).    For what it is worth….

    @randa 6/18 issue: As to the second issue, I have ask for a bug to be logged about the localization issue, as I believe it should work better than what you describe, but it will require some time that we don't have to devote to it right away.   It sounds like you have a work-around in the mean time.

  19. Peter Palotas says:

    I'm looking to use the EventSource of .NET 4.6, but how can I generate the dll/man files that I need to send to wevtutil in that case? The NuGet EventRegister tool seems to still perform validations of requirements that from what I've understood are no longer required in 4.6, for example that the class must be sealed or abstract?

  20. ranta says:

    I get a NullReferenceException within EventSource.CreateManifestAndDescriptors, when I call EventSource.GenerateManifest(Type, string) using the 1.1.25 NuGet packages with Visual Studio 2010 SP1 on .NET Framework 4.5.2 and Windows 7 SP1, and the event-source class defines a Stop event.  I compared the x64 disassembly to the coreclr source on GitHub, and I think the eventData variable is null when it is used in eventData[startEventId].  If I instead call EventSource.GenerateManifest(Type, string, EventManifestOptions) and specify EventManifestOptions.Strict, then the NullReferenceException does not occur.

    Should I report such problems on Connect, or what is the proper channel?

    Is there any documentation on what changed between 1.1.24 and 1.1.25?  I didn't find it at nuget.org.

    Does Microsoft publish debug symbols and exact sources for the Microsoft.Diagnostics.Tracing.EventSource.Redist package?  I tried enabling .NET Framework source stepping but that did not find the sources.

  21. @ranta In order understand your bug, we would very likely need a repro (that is the source code of the EventSource that causes the failure.     Normally you report such things through 'Contact Owners' link associated with the package on nuget.org.      

    There is no documentation on the changes between .24 and .25.  They where minor, targeted bug fixes,

    We currently do not publish the source code or symbols for EventSource.   We are planning on releasing it as open source later this year, but until that time it is not available.  

  22. ranta says:

    @Vance, here's a simple repro.  Just add the 1.1.25 NuGet packages.

    namespace EventSourceBugRepro

    {

       using Microsoft.Diagnostics.Tracing;

       [EventSource]

       internal sealed class ReproEventSource : EventSource

       {

           [NonEvent]

           private static void Main()

           {

               // This works OK.

               EventSource.GenerateManifest(

                   typeof(ReproEventSource),

                   assemblyPathToIncludeInManifest: null,

                   flags: EventManifestOptions.Strict);

               // This throws NullReferenceException in EventSource.CreateManifestAndDescriptors.

               EventSource.GenerateManifest(

                   typeof(ReproEventSource),

                   assemblyPathToIncludeInManifest: null);

           }

           [Event(1)]

           void ReproStart()

           {

               this.WriteEvent(1);

           }

           [Event(2)]

           void ReproStop()

           {

               this.WriteEvent(2);

           }

       }

    }

  23. Should be very easy to find and fix in our next release.    Thanks

  24. ranta says:

    The NuGet package Microsoft.Diagnostics.Tracing.EventSource.Redist 1.1.25 contains versions of Microsoft.Diagnostics.Tracing.EventSource.dll built for three frameworks: net35, net40, and portable-win8+wpa81. All three have 1.1.25.0 as both assembly version and file version. Each has a different TargetFrameworkAttribute though.

    Is this a best practice for versioning a library for multiple frameworks? I had to backport a library of ours to .NET 3.5 this week, so I'd appreciate any guidance on this topic. Is there a risk that the wrong version is loaded if these DLLs are installed in the GAC? I played a bit with gacutil /i and procmon on Windows 7 SP1, and it seems the net35 version goes under %SystemRoot%assembly but the net40 and portable-win8+wpa81 versions both go under %SystemRoot%Microsoft.NETassembly. I have not tested whether Windows 8 would place all three in separate directories.

    If I build an assembly with .NET Framework 3.5 and the net35 version of Microsoft.Diagnostics.Tracing.EventSource.dll, but then run it with .NET Framework 4.0 and the net40 version of Microsoft.Diagnostics.Tracing.EventSource.dll, is it expected to work?

  25. @ranta.   Nuget packages are not intended to be place in the GAC.   They are intended to be app-local.  This is true for the EventSource Nuget package as well.    

    EventSource is GENERALLY meant to be backward compatible so there is a good chance that code compiled against a V3.5 version will work against a later version.   However part of the value of Nuget (and app-local deployment), is that you don't HAVE to have a strict compatibility model, so if you intend to do things like that, and really MUST make it work, then you have to test before you rely on it.   (We don't try to break compatibility (unless forced to), but we also don't do extensive compatibility testing).  

  26. ranta says:

    Okay, I won't install the assembly to the GAC. But I was concerned that someone else might do that and break our application. "How the Runtime Locates Assemblies" msdn.microsoft.com/…/yx7xezcf%28v=vs.110%29.aspx says the runtime checks the GAC before probing for app-local files.

    ".NET 4.5.1 Supports Microsoft Security Updates for .NET NuGet Libraries" blogs.msdn.com/…/net-4-5-1-supports-microsoft-security-updates-for-net-nuget-libraries.aspx says that, if there is a serious security vulnerability in a NuGet package serviced by Microsoft, then Microsoft Update will install a corrected version and publisher policy to the GAC. I hope this will not cause any conflict between the net40 and portable-win8+wpa81 versions. Although I'm curious about the solution, I understand that the details do not affect how I should use the NuGet packages.

  27. @ranta  It is true that people could put things in the GAC and thus override things, but anyone who does has to believe that what they put there is 100% compatible or they will break other applications.    Frankly the only one who should be doing this is the owner of the assembly, and we will never do this short of a security fix, and in that case we will be careful about being as 100% compatible as possible.   In short, ignore the GAC.  

  28. ranta says:

    Regarding side-by-side versions of the same component ("6/17 issue"):

    I have now removed the Windows Event Log integration from our event sources. The event methods do not have EventAttribute.Channel, the build system does not generate a manifest, and no manifest is registered with wevtutil.exe. Instead, an EventListener in the same appdomain collects the events and writes them to a custom logging system, or to the console window if debugging. Because the manifest is not  registered, there should be no problems with side-by-side versions. This loses the internationalization support of Windows Event Log but that is not a problem for our in-house applications. The events still have structure and are available to ETW if desired, and I hope that ETW consumers can filter the different versions by process ID.

    I noticed something bizarre though:

    1. Build the project in Visual Studio.  The build system runs eventRegister.exe -DumpRegDlls, but eventRegister.exe notices there are no channels defined, so it does not write any *.man or *.dll files.

    2. Manually run eventRegister.exe -DumpRegDlls -ForceAll. It writes *.etwManifest.man and *.etwManifest.dll files.

    3. Edit EventAttribute.Message of one event method in Visual Studio and build the project again.

    4. According to the build log and time stamps, eventRegister.exe -DumpRegDlls compiled the *.etwManifest.man file to *.etwManifest.dll, even though -ForceAll was not specified during this build and the *.etwManifest.man file does not contain the EventAttribute.Message change.

    In my opinion, eventRegister.exe -DumpRegDlls should compile only those *.etwManifest.man files that it generated during the current invocation. The current behavior is particularly annoying if the *.etwManifest.man file contains an error that causes eventRegister.exe to reject it; then the build will fail regardless of whether the error has already been fixed in the C# sources.

  29. Lou says:

    I'm very interested in learning more about the Activity Tracking support in this latest version of the EventSource package. Can you recommend some reference material that walks through these capabilities?

  30. I am intending to blog about the Activity Support very soon (I really wanted to have already done it by now).  I have a pre-release copy of the docs here:

    onedrive.live.com/redir

  31. Lou says:

    Just what I needed. Thanks!

  32. Peter Cabus says:

    I just glanced through the pre-release doc on activity tracing and it looks very informative.  I did not have the time to experiment with it but I have one question.  What happens to ETW Events that already have activity IDs.  For example the WCF ones (Microsoft-WindowsApplication Server-Applications).  Are the activity IDs in those events changed by this feature?

    Thanks,

    Peter

  33. There is only one activity on a given thread at a given time, so 'last one to update' wins.    However there is a convention (which the Activity feature obeys), that when you issue a 'start' that you put the previous activity ID in the 'relatedActivityID' field.   Thus you for a linked list of activities (you update the ActivityID slot but you can remember the previous one).   Thus if you already have a WCF activity on your thread, the 'Start' will log both the new and the old one (so you know one caused the other).   If WCF follows this convention (I *think* it does) then it works the other way as well.  

    Thus if everyone plays by these rules, it all works.   The fact that people might not realize that they should play by these rules is one of the big reasons we really don't really want people trying to do this on their own…

  34. ajhuddymsdn says:

    Vance,

    Can you please advice how I can get a more accurate time an event was fired through the EventListener?  Capturing the current timestamp in the EventListener's OnEventWritten virtual method is not accurate enough for me, I require the timestamp at which "WriteEvent(…)" was called into the framework.

  35. EventListeners get called back AT THE LOGGING point, and thus are basically AT THE RIGHT TIME (or as close as you can get).    Thus you can't do better than simply taking the time.

    However what is true is that normal DateTime has only 16 msec granularity, so that is probably your problem.   You can use Stopwatch as a high resolution clock (start it at the beginning of the program and keep calling Elapsed on it to get a timestamp).  

  36. ajhuddymsdn says:

    Vance,

    Thank you for the insight.  I am indeed using a stopwatch.  I'm going to put this concern to bed now that I've heard it from the horse's mouth.

  37. ajhuddymsdn says:

    My main concern was raised by looking at the call stack for OnEventWritten.  The stack entries starting with "…" are framework methods.  I'm concerned about the timing of this stack.

    –> OnEventWritten()

    … DispatchToAllListeners

    … WriteToAllListeners(int, Guid*, object[])

    … WriteToAllListeners(int, Guid*, int, EventData*)

    … WriteEventWithRelatedActivityId(int, Guid*, int, EventData*)

    … WriteEvent(int, int)

    –>MyEvent()

  38. Yes, that is the stack you would expect.   It is true you are a few hundred nsec later than when your Write method was called, but that is as close as you can get.  

  39. ranta says:

    Does the following entry in ".NET Framework 4.6.1 list of changes" refer to the bug for which I posted a repro here on Jun 30 2015?

    * No null reference exception when calling EventSource.GenerateManifest(Type, string) when an event method name ends with "Stop" [129244]

  40. @ranta  yes, we believe we have fixed the bug associated with events ending with 'Stop'.  However experimental validation is always better than anything that a human will tell you about what *should* be true.  

Skip to main content