Instrumentation in Enterprise Library


Edit: This content applies only to the 1.x (January 2005 and June 2005) releases of Enterprise Library. Instrumentation behaves differently in the 2.x (January 2006) release.


Much of this has already been discussed in other forums, but there seems to be a fair bit of confusion about the instrumentation code in Enterprise Library, so I thought it would be worthwhile for me to touch on this too. I’ll try to clear up how it currently works, why we chose to do it that way, and what we are thinking of doing to improve instrumentation in vNext.


One of the primary goals of Enterprise Library is to showcase best practices for enterprise .NET development. We try to do this in multiple domains, including architecture, code standards, unit testing and operations. This last one is especially important, as it’s something that developers often ignore. As a developer it’s very tempting to fall into the mindset that once it compiles and meets functional requirements, it’s somebody else’s problem. Then one day someone from your organization’s operations team will call you at 11pm on a Friday night (probably when you’re at the pub with your mates) and declare that your app is broken and your help is needed to fix it. And unless the application is properly instrumented (including logging errors, trace/diagnostic messages and health monitoring) it can be really, really hard to work out what’s going on (and whose fault it is – because as soon as you can prove it’s not to do with your app, you can go back to the pub!).


So instrumentation is very important, which is why we made sure we included it in the blocks – including event log messages, WMI events and performance counters. Of course you also need to instrument your applications as well, and we make a lot of this easier with the Logging & Instrumentation Application Block. However most kinds of instrumentation come at a cost – you generally need to run some kind of installation script and/or have administrator rights. In most cases this is something that can be dealt with, but we are aware that it is sometimes not an option – for example when deploying apps to hosted environments where you have limited rights and don’t have the ability to run scripts.


So we tried to support both of these scenarios with Enterprise Library. Instrumentation is enabled by default, but to make sure everything is registered you’ll need to run the Install Services script from the Start Menu, or run installutil over each assembly (possibly as a part of your own MSIs). When you install Enterprise Library with default settings, all of the code will be automatically compiled, but unfortunately we didn’t run the Install Services script for you. This was an unfortunate outcome – but if you remember to run the script yourself then everything should work well.


If you are deploying to an environment where the instrumentation cannot be used, it’s pretty easy to disable it, but you will need to recompile the code. Luckily all the instrumentation code is wrapped around conditional compilation directives, so you won’t need to edit any source files directly. Just go into the Project Properties dialog for the Common project, and under Configuration Properties\Build, find the Conditional Compilation Properties property and remove ;USEWMI;USEEVENTLOG;USEPERFORMANCECOUNTER (or any combination of these that you don’t want). Once you recompile, the relevant instrumentation code will be disabled. Of course, it is still possible to configure the Logging & Instrumentation Application Block to use WMI or Event Log, so make sure you also choose appropriate settings for your environment if you are using that block.


So, how are we thinking about improving the instrumentation? We’re still in planning mode so we’re open to suggestions, but at the top of the list are:



  • Make it possible to change the instrumentation behavior (turn it on and off) through configuration, rather than by recompiling the code
  • Run the Install Services script by default as a part of the installation process
  • Make the instrumentation more fine-grained, for example have separate performance counter instances for each Enterprise Library item such as a Database or Cache Manager

I hope this helps clarify things – please keep the suggestions coming!


This posting is provided “AS IS” with no warranties, and confers no rights.


Comments (58)
  1. Alex says:

    I wonder why configuration through web.config wasn’t introduced in the first place. When I read that you need to recompile the app, I immideately asked myself "what on earth for?".

    What is gained by having instruments conditioned with directives vs. web.config?

  2. Tom says:

    It was a scope/timing thing. However it’s important to understand that the instrumentation we are talking about isn’t the kind of thing that is designed to be turned off and on after deployment (unlike diagnostic logging, for example). The assumption is that this type of logging instrumentation should always be turned on, but we provide the conditional compile options to disable this if it’s a deployment blocker.

    Tom

  3. Probably some experienced programmers want to turn it off because it could increase performance. Because this could be possible, I think you should make it possible to turn off instrument through configuration. It’s also a good idea to make the instrumenting more fine-grained. It could make it possible for us to watch the behavior on each block with the Performance monitor.

  4. What’s the proper way of using Tracing in an ASP.NET application?

    This doesn’t work:

    using (new TraceContext(this.Context))

    {

    Logger.Write("Hello world");

    }

    I get a "Cannot implicitly convert type ‘System.Web.TraceContext’ to ‘System.IDisposable’

    Thanks for any pointers.

  5. Larry says:

    Better deployment instructions. I was able to get asp.net logging via a flat file and event log working on my development machine, but getting it to work in a production environment is vague (actually I’ve been working on it a couple hours and still have no clue)

  6. Tom says:

    Adam –

    You should use

    using new(Tracer())

    {

    Logger.Write("Hello world");

    }

  7. Erik says:

    There is a problem when turning the USEPERFORMANCECOUNTER off (removing it from the project). When you try to CacheFactory.GetCacheManager(… , you get a runtime error in:

    Microsoft.Practices.EnterpriseLibrary.Caching.Instrumentation.CachingServiceItemTurnoverEvent.SetTotal(long totalItems)

    The CachingServiceItemTurnoverEvent.totalEntriesInstances is null. It’s not hard to fix, but can you give a hint how much you tested without USEPERFORMANCECOUNTER, can I expect more runtime errors when removing theese properties?

    Erik

  8. Mel says:

    Hmmm…maybe people want to turn it off because the application you wrote on your development environment won’t work in a production server? That’s a pretty good reason.

    I can’t believe that it wasn’t made blatantly obvious. "Oh, btw, your stuff won’t work anywhere other than your machine unless you recompile the Enterprise Library Common project with some objuscated settings removed. K? Thx."

  9. jasper pons says:

    The caching application block throws a runtime error on my deployment machines. The error message is not clear but I think the caching application block is trying to create an event log but the logged in user does not have admin permissions so this fails. I dont want the caching block to create event logs, is there a way to turn this off in the caching config file? documentation is a bit thin on the supported switches for caching ( btw I am not using the instrumentation block).

  10. jasper pons says:

    I took out those switches in the project properties as mentioned above, however, it then threw an error. I had no option but to modify the code and put a try with an empty catch block around it. The error is:

    An unhandled exception of type ‘System.NullReferenceException’ occurred in microsoft.practices.enterpriselibrary.caching.dll Object reference not set to an instance of an object.

    in class CachingServiceItemTurnoverEvent.cs in function SetItemsTotal at line cachingEvent.SetTotal(totalItems);

  11. Tom says:

    Mel: The code will work on a production machine if you have sufficient rights to run the installers. If you don’t have these rights, you should know this well in advance of deployment.

    Tom

  12. Steve says:

    I get unhandled exceptions, I have yet to get this data access blocks to run. The sqlhelper code worked well in the past, now I have to redo all my code for the new blocks.

    I don’t mind it writing to the event logs, I just wish you’d get an error message or some instructions on what to do beside ‘recompile the code’.

    I’m not impressed by unhandled exception handling code by MS.

  13. Steve says:

    "sufficient rights" – how about some clarification?

    I’m an administrator of the box – wouldn’t that be enough rights???

    Or do I have to give the rights to ASPNET and compromise my security?

    Confused with lack of documentation and vague responses online.

  14. Tom says:

    Steve – you don’t need to give additional rights to your ASPNET account, but if you want the instrumentation on then you should install the services using an admin account.

    Regarding the error message, it’s hard to help without more specifics. It’s better to post these types of questions to the community site though since more people are able to help out.

    Finally, if DAAB 2 works well for your app, don’t feel obliged to move to Enterprise Library. We added a bunch of stuff that many people find useful, but if you don’t need it, stick with what works!

    Tom

  15. Tom Whitner says:

    You mention that you intend to run the "Install Services" script as part of the install process. However, you are not distributing binaries. How will that be possible?

  16. Tom says:

    Tom – the installer already has the option to compile the code during installation. So we’d just need to add this as an additional step after the compilation.

  17. Mel says:

    Tom,

    The issue i think is that the installservices.bat file requires the use of the installutil.exe application. The script also needs Visual Studio installed on the production machine so that it can run vsvars32.bat so that it can call installutil.exe.

    It’s probably not a good idea to install Visual Studio on a production machine. Plus, for some people, that’s not even an option.

  18. Justin says:

    Searching for hours finally led me to find that these won’t work on a production server unless Visual Studio 2003 is installed on the server (why would you have it installed on a production server anyway?!?).

    A top priority item would be to create an installer that web developers can run on our web server to setup the logs so as to not get the security errors.

    I have all permissions on my server and after hours of searching I can’t figure out how to make it work manually.

  19. Mei says:

    There is a know bug in the CachingServiceItemTurnoverEvent in that when the UsePerformanceCounter switch is turned off, the perf. counter instance will be null. If you want to recompile to get rid of the use of the perf counters, you need to modify the code like below:

    private void FireEvent(long count)

    {

    if (turnOverInstances != null)

    {

    turnOverInstances.IncrementBy(count);

    }

    }

    private void SetTotal(long totalItems)

    {

    if (totalEntriesInstances != null)

    {

    totalEntriesInstances.RawValue(totalItems);

    }

    }

    Another issue I want to address is that when the user is not part of the local machine’s administrator’s group, firing WMI event will cause internal exception to be thrown and caught the first time the event is fired. After the InstrumentedEvent instance knows that the event failed to fire, it will not attempt to do that again in the singleton object’s lifetime.

  20. Shrinivas says:

    I am trying to use logging/instrumentation. If I work with winform application this works fine. But how I can use the same with class library where I need to incorporate logging/instrumentation.

    Should I follow the same steps?? The main problem I am facing is, it seems EL required app.config or web.config, none of which are available for class library.

    Thanks in advance.

  21. Tom says:

    Shrinivas –

    Yes you should follow the same steps to install the instrumentation.

    Using blocks from class libraries is discussed in the Enterprise Library FAQ here: http://www.gotdotnet.com/workspaces/customization/uploadedhtmlpage.aspx?FileID=ded67339-a081-489a-8d63-817323f31104&id=295a464a-6072-4e25-94e2-91be63527327

    Basically the DLL will inherit its class library from the exe or web app which calls it. If you don’t want to do this, you can manually bootstrap the block with config from another file.

    Tom

  22. Drew says:

    Tom,

    I encountered this error once before and it solved itself overnight on a dev box… Now I have deployed an app to a production server and I get this error….

    Exception Details: System.Security.SecurityException: Requested registry access is not allowed.

    [SecurityException: Requested registry access is not allowed.]

    Microsoft.Win32.RegistryKey.OpenSubKey(String name, Boolean writable) +473

    System.Diagnostics.EventLog.CreateEventSource(String source, String logName, String machineName, Boolean useMutex) +445

    System.Diagnostics.EventLog.CreateEventSource(String source, String logName, String machineName) +11

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.PerformanceCounterInstances.ReportCounterFailure(String message)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.PerformanceCounterInstances..ctor(String categoryName, String counterName, Boolean createNewInstance)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.InstrumentedEvent.AddPerformanceCounter(String category, String[] counterNames, Boolean createNewInstance)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.InstrumentedEvent.Initialize(String counterCategory, String[] counterNames, Boolean createNewInstance, String eventLogSource, EventLogIdentifier[] eventIds)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.InstrumentedEvent..ctor(String counterCategory, String[] counterNames, Boolean createNewInstance)

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataServiceEvent..ctor(String[] counterNames)

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataCommandFailedEvent..ctor(String[] counterNames)

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataCommandFailedEvent..cctor()

    The last time I got this error on my dev box I traced it through and it was on the attempt to fire a WMI event…

    When commenting out that line of code it worked flawlessly…

    Obviously on the web server I do not have VS 2003 installed so I am unable to step through the code and see what exact line is failing – but I have pasted the trace info above.

    This is running on a Windows 2000 Server SP4.

    I realize that I could possibly fix this by turning the instrumentation section off… BUT, why does it work on some servers and not all. I have run the exact same app on a Win 2003 box, Win XP Pro, as well as an additional Win 2K Server SP4 and they all work flawlessly.

    You say above that you should run with instrumentation turned on… should it not be able to handle when it’s own exceptions?

    I can’t believe that the solution to the error is not to cause it. Surely there is an explanation as to why this is causing an error.

    I would appreciate any insite you could share…

    Thanks for the great work.

  23. Tom says:

    Drew – did you run the Install Services script or installutil over the EntLib assemblies? Also WMI is very particular about where the assemblies are located, so you’ll need to run the script over the exact assemblies that your app is loading.

    Tom

  24. Drew says:

    Tom,

    That is a quick response…

    1) installed the Entlib – same error

    – installed to e:Microsoft Enterprise Library

    2) ran Install Services – same error

  25. Drew says:

    I was just looking in the Install Services Batch file and it appears to be pointing to VS 2003…

    What would I have to change to point it at the correct assemblies?

  26. Tom says:

    The stuff in InstallServices.bat that points to VS2003 is basically there to find the location of installutil.exe. However installutil can also be found in other locations, such as C:WINDOWSMicrosoft.NETFrameworkv1.1.4322.

    On a production machine without VS2003, rather than run InstallServices.bat it is probably safer to run Installutil.exe directly over the EntLib assemblies, eg:

    installutil Microsoft.Practices.EnterpriseLibrary.Common.dll

    installutil Microsoft.Practices.EnterpriseLibrary.Configuration.dll

    … and so on over whatever assemblies your application is using.

  27. Drew says:

    Thanks Tom

    I will try this later today and post the results.

    Two final questions –

    1) Should this be run on the Enterprise Library assemblies in the installation folder or on the assemblies packaged with the app in the bin folder?

    I am assuming that if I run it on the versions in the EntLib installation folder that will suffice.

    2) When recompiling the enterprise library with patches and such do I need to rerun the installutil?

    Thanks again

  28. Drew says:

    TOMMY – YOU THE MAN!!!

    For anyone that gets this issue – feel free to send me the cheque for the time Tom just saved you.

    I spent hours going through posts that suggested changing the registry and looking at permission levels for ASPNET and IUSER acocunts etc…

    All you have to do is like Tom said run Install Services or installutil from the framework directory.

    <b>I tested this on a Win 2003 Server machine without Ent Library installed.</b>

    <i>Note: the site is primarily using the Data Block</i>

    I was going to install them one by one starting with the common library and document

    1) Installed the website: Copied the files and setup the site in IIS

    a) received same error

    2) Ran installutil <path>Microsoft.Practices.EnterpriseLibrary.Common.dll

    a) Site works flawlessly!!

    Have a great day

  29. Steve says:

    I made it to a point where it then asked for a disk while running the installutil

    I also would like the option to turn this off, I’m having a tough time getting this to install on my servers.

  30. David says:

    I had the same problems with the services so tried to remove them as indicated above. The solution will only compile in DEBUG, when trying to compile in RELEASE or RELEASEFINAL the compile fails on 17 of the 38 assemblies.

    Has anyone else had this problem and what should they be compiled under, RELEASEFINAL presumably?

  31. Chuck says:

    Tom – I’m still having issues with getting the assemblies to work on my production server.

    They work fine on my dev workstation.

    I then deployed it to my staging/test server and it works fine (I have a page that pulls back a value from a db table and displays it in a label control). I do not have VS.NET installed on the staging server so I ran the installutil against the assemblies in my bin directory.

    I moved all of the assemblies to my bin folder in production, ran the installutil against them. When I try to load my test page, the page seems to just hang. I’ve let it run for 10 min or so and I don’t get anything. I don’t get any errors in the event log or on the page. I assume that it installed properly b/c there are perf counters for the data application block in the perfmonitor on the production machine.

    I know that this is pretty vauge but any help is appreciated.

    Thanks.

  32. Chuck says:

    Update….After waiting on the page, I’m getting an access denied error. When I compare the install logs from my staging server to my production server I do not see the ensuring that namespace/class exists entries on the production machine. I also used the installer package to set up the EL on the server (something I did not do before) and still no luck. I think I’m missing something small…..

  33. Shrinivas says:

    Hi Tom,

    I tried solution to load config file programmatically as well explained here:

    http://blogs.msdn.com/scottdensmore/archive/2005/03/01/382650.aspx

    I tried with LogWriter and was able to log it, but I cannot trace it with "Tracer". It fails to load the configuration file, and when I quickwatch "configFile" for Current.Builder (ConfigurationManager.GetCurrentContext), it is pointing to machine.config.

    Am I doing right thing?

    This blog by scott also talks about

    "Each block has a factory class that accepts a ConfigurationContext in it’s constructor"

    –> which is this factory class for logging block?

    Thanks in advance. Let me know if this requires further explanation?

  34. Shrinivas says:

    Continuation of my earlier thread:

    I think I need to explain this more clearly. This is how exactly I create LogWriter:

    context = ConfigurationManager.CreateContext( @"E:ELPROJECTSEL-BLOCK-CONFIGSLOG-INSTRU-DALDAL-LOG-INSTRUMENTATION.config" );

    LogWriter writer = new LogWriter( context );

    Unfortunately, after CreateContext, ConfigurationManager does not maintain this and also there is no way you can set Current Context, and therefore when I say ConfigurationManager.GetCurrentContext, it returns me the context which reads configuration from machine.config and not from my config file.

    Hope this helps to understand issue more clearly.

    Regards,

    Shrini

  35. Shrinivas says:

    Upon looking at Tracer code, it has a constructor which accepts context as a part of signature, but is internal, so I wont be able to use it.

    I am not sure why this constructor was made "internal"? Any idea? I think it is very nice to have something like that public.

    BTW, I am very anxious to know the solution for this issue.

    Regards,

    Shrini

  36. Shrinivas says:

    Upon looking at Tracer code, it has a constructor which accepts context as a part of signature, but is internal, so I wont be able to use it.

    I am not sure why this constructor was made "internal"? Any idea? I think it is very nice to have something like that public.

    BTW, I am very anxious to know the solution for this issue.

    Regards,

    Shrini

  37. Mark Schultheiss says:

    Tom,

    You said above: "Basically the DLL will inherit its class library from the exe or web app which calls it. If you don’t want to do this, you can manually bootstrap the block with config from another file." well, not really sure how to do that last part but…

    OK so I cannot see where ANYONE would want to do this for ASPNET user (inherit permissions for WMI)…I am probably missing the obvious somewhere on setting this up for ASPNET. It seems the WMI would be most useful actually for web environments but permissions seem to be getting in the way.

    I ran the service installs using the menu – it is my dev machine, and was still getting errors. (I am an admin on the box) I am attempting to use as an ASP.NET user.

    I had to set ASPNET user to be a member of

    the Administrators group or I continued to get WMI errors in the stack trace. (see below)

    Not ideal and will not work on production servers at all.

    Seems to have been a low level of attention to ASP.NET users of the block in general IMHO.

    Choice would seem to be to cripple the WMI by taking it out/recompile or to set as above which it seems would compromise security.

    FYI,

    I get the error on WMI when I try to use the Data block with:

    <b>dsDb = db.ExecuteDataSet(dbCommandWrapper); ></b>

    Add ASPNET to Administators group, restart IIS and it goes away. Take it back out of the Administrators group, restart IIS and it is back.

    Here is the error which it seems is not really accurate since the event log is EMPTY, it is just a permissions issue.

    ——————

    The event log file is full

    Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

    Exception Details: System.ComponentModel.Win32Exception: The event log file is full

    Source Error:

    —————————-

    Here is the stack trace…

    Stack Trace:

    [Win32Exception (0x80004005): The event log file is full]

    System.Diagnostics.EventLog.WriteEvent(Int32 eventID, Int16 category, EventLogEntryType type, String[] strings, Byte[] rawData) +606

    System.Diagnostics.EventLog.WriteEntry(String message, EventLogEntryType type, Int32 eventID, Int16 category, Byte[] rawData) +462

    System.Diagnostics.EventLog.WriteEntry(String message, EventLogEntryType type, Int32 eventID, Int16 category) +21

    System.Diagnostics.EventLog.WriteEntry(String message, EventLogEntryType type, Int32 eventID) +15

    System.Diagnostics.EventLog.WriteEntry(String message, EventLogEntryType type) +11

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.PerformanceCounterInstances.ReportCounterFailure(String message)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.PerformanceCounterInstances..ctor(String categoryName, String counterName, Boolean createNewInstance)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.InstrumentedEvent.AddPerformanceCounter(String category, String[] counterNames, Boolean createNewInstance)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.InstrumentedEvent.Initialize(String counterCategory, String[] counterNames, Boolean createNewInstance, String eventLogSource, EventLogIdentifier[] eventIds)

    Microsoft.Practices.EnterpriseLibrary.Common.Instrumentation.InstrumentedEvent..ctor(String counterCategory, String[] counterNames, Boolean createNewInstance)

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataServiceEvent..ctor(String[] counterNames)

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataCommandFailedEvent..ctor(String[] counterNames)

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataCommandFailedEvent..cctor()

    [TypeInitializationException: The type initializer for "Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataCommandFailedEvent" threw an exception.]

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataCommandFailedEvent.Fire(String commandText, String connectionString) +0

    Microsoft.Practices.EnterpriseLibrary.Data.Instrumentation.DataInstrumentationFacade.CommandFailed(String commandText, String connectionString)

    Microsoft.Practices.EnterpriseLibrary.Data.Database.DoLoadDataSet(DBCommandWrapper command, DataSet dataSet, String[] tableNames)

    Microsoft.Practices.EnterpriseLibrary.Data.Database.LoadDataSet(DBCommandWrapper command, DataSet dataSet, String[] tableNames)

    Microsoft.Practices.EnterpriseLibrary.Data.Database.LoadDataSet(DBCommandWrapper command, DataSet dataSet, String tableName)

    Microsoft.Practices.EnterpriseLibrary.Data.Database.ExecuteDataSet(DBCommandWrapper command)

    DynamicGrid.uc_dyngrid.BindReports() in c:inetpubwwwrootpagetemplatedynamicgriduc_dyngrid.ascx.cs:85

    DynamicGrid.uc_dyngrid.Page_Load(Object sender, EventArgs e) in c:inetpubwwwrootpagetemplatedynamicgriduc_dyngrid.ascx.cs:55

    System.Web.UI.Control.OnLoad(EventArgs e) +67

    System.Web.UI.Control.LoadRecursive() +35

    System.Web.UI.Control.LoadRecursive() +98

    System.Web.UI.Control.LoadRecursive() +98

    System.Web.UI.Control.LoadRecursive() +98

    System.Web.UI.Page.ProcessRequestMain() +750

  38. Many of us encountered a lot of problems when installing Microsoft Enterprise Library&amp;nbsp;on production…

  39. In my last post, Enterprise Library 2.0 : Used by ASP.NET Applications on Shared Server?, I asked the…

  40. Responding&amp;nbsp;to a comment on my last posting&amp;nbsp;regarding the signing of Enterprise Library, I was…

  41. What an utterpile of wank. I’ve spent DAYS on this one.I was using .NET Tiers (nettiers) with .NET 2

  42. Many of us encountered a lot of problems when installing Microsoft Enterprise Library on production servers,

Comments are closed.

Skip to main content