CLR SPY and Customer Debug Probes: The Object Not Kept Alive and Buffer Overrun Probes (A Quiz)

The Object Not Kept Alive and Buffer Overrun probes are unlike any other CDPs, because they do not output any messages to report bugs in your code.  Instead, they change general CLR behavior with the goal of forcing non-deterministic bugs that can be almost impossible to reproduce into bugs that will happen every time you run your application.  As a developer or tester, that’s obviously a good thing.  The lack of a specific probe message might make such bugs somewhat harder to diagnose, but once you understand the probes’ behavior, you can learn to be successful with them.

The Object Not Kept Alive probe forces a garbage collection (then waits at most a second for pending finalizers) immediately before every transition from managed to unmanaged code.  This is done to surface problems that could occur if a managed object happens to be collected during the small time window when transitioning to unmanaged code.  I find this to be one of the most useful of all the CDPs.

The Buffer Overrun probe forces a garbage collection (then waits at most a second for pending finalizers) immediately after every transition from managed to unmanaged code.  The intent of this is to catch buffer overruns, which unmanaged code has the power to cause.  (Managed code does too — even verifiable managed code if it uses methods of the Marshal class, which require unmanaged code permission.)  If unmanaged code writes into memory that it’s not supposed to, it could potentially corrupt the GC heap.  The next time the garbage collector compresses the heap (as part of a collection), it’s likely to run into an access violation.  So by forcing a collection after every call into unmanaged code, the corruption can hopefully be noticed at the source of the problem.

Due to the large number of collections forced by these probes, they will significantly slow down most applications.

Now consider the following harmless-looking C# code that uses the Microsoft.Win32.RegistryKey class:

  // This code has a bug!

  public static void Main ()


    // Open the key for read-only access

    RegistryKey key = Registry.LocalMachine.OpenSubKey(


    Console.WriteLine(“Install root: ” +



Most likely, this code will run fine if you don’t enable any probes.  If you enable the Object Not Kept Alive debug probe and compile this code with debugging information (“csc /debug ObjectNotKeptAlive.cs” from a command prompt, or the Debug solution configuration in Visual Studio .NET), you’ll still get the output that you’d expect when running the program.  For example:

  Install root: c:\windows\\framework

But if you compile it without debugging information (“csc ObjectNotKeptAlive.cs” from a command prompt, or the Release solution configuration in Visual Studio .NET) with the Object Not Kept Alive probe still enabled, you get surprising output:

  Install root:

Weird, huh?

Do you see any bugs in the code?  Who can tell me why this happens with the probe enabled?  Why does it only happen in the debug configuration?  My last quiz was answered pretty quickly, so please contribute any of your thoughts!

Comments (15)

  1. Phil Hochstetler says:

    This is simple. The call to "key.GetValue()" is the call to unmanaged code that returns an object as the return value. Note that this call is not the problem. The problem is that the return value has no object reference that the GC can find so it will be freed whenever the GC runs. The next call to unmanaged code happens when Console.Writeline is called (and thus the return object from the key.Getvalue() call is poof gone). Good bug and not something I would have thought about. I have written some code to do a mix of managed and unmanaged code (a service that talks to multiple serial ports) so I am going to use this technique to find any subtle bugs in my implemenation. Thanks!

  2. Phil Hochstetler says:

    Humm. on more thought, I don’t think I am right on this one. I am waiting for sharper minds than mine on solving this one. I think a stack reference must still be a reference. Plus the stack reference it self is the result of creating a new object as a result of the expression "string + obj.ToString()". Anyone got this one figured out?

  3. Fernando Tubio says:

    The answer to this question can be found in Chris Brumme’s (excelent!) blog (see Lifetime, GC.KeepAlive, handle recycling).

    RegistryKey encapsulates a handle, and presumably this handle is closed during Finalize. Forcing a garbage collection just before the transition to unmanaged code will close it, since the reference to key is no longer needed as soon as the handle is extracted from it.

    Why doesn’t it fail with debug builds? I vaguely remember (but I am not certain) that in debug builds the lifetime of references is prolongued so that you can examine the value of objects in the debugger, and not loose them as soon as they become unreachable.

  4. I disagree that the code fragment shown is buggy. The bug is actually in the implementation of RegistryKey. Chris Brumme discusses this, plus the fact that it will be solved in a future version of the .NET Framework.

  5. Mike says:

    I agree that the debug/release thing is a question of how aggressively the key variable reference is scavenged from this function (i.e. whether we keep it alive to the end of the scope to aid debugging or not).

    Because the probe forces a GC before we get into unmanaged code you’d imagine that at the transition between managed/unmanaged code our reference to the instance that key pointed to has gone and so when we GC there we end up disposing "key" which closes the handle.

    However, if the probe only causes a GC then you’d think that this would be unlikely – does the probe also make sure that we wait for any pending finalizers at this point? It would seem like a good idea to do that – if this is the right tack then maybe Adam would comment on whether the probe just does a GC or whether it does the equivalent of GC.WaitForPendingFinalizers to really flush everything through? From looking at it in a debugger it looks like it does do this.

    It would seem (from the CIL) that when Win32Native.ReqQueryValueEx is called we insert the probe and that forces the GC and the RegistryKey’s finalizer ends up called which closes the handle and, at a later point, the unmanaged code then actually queries the value of the key.

    If you watch this in a debugger you should be able to see the call to RegCloseKey go through with the handle of the key that the RegistryKey class is wrapping for us (obtained the value of this handle by using the sos.dll debugging extension from MSDN) before a subsequent call to RegQueryValueEx goes through using the same (now closed) registry key handle and (if you step through the disassembly) it looks like that call to ReqQueryValueEx returns a value of 6 which is ERROR_INVALID_HANDLE.

    Something like that, anyway – might have got the specifics a bit wrong here.

    These debug probes are really, really smart 🙂 but then the bug is really, really tricky to nail down 🙁

  6. Adam Nathan says:

    You got it! The RegistryKey instance can be collected while its GetValue method is still executing, and the Object Not Kept Alive probe forces this to happen at the exact time that causes trouble for this code. The handle encapsulated by RegistryKey is indeed closed during finalization. Here’s essentially how RegistryKey is implemented (a simplified form of the truth):

    public class RegistryKey
    static extern int RegQueryValueEx(IntPtr hKey, string lpValueName,
    IntPtr lpReserved, ref int lpType, byte [] lpData, ref int lpcbData);

    static extern int RegCloseKey(IntPtr hKey);
    IntPtr hkey;
    public object GetValue(string name)
      RegQueryValueEx(hkey, name, IntPtr.Zero, ref type, array, ref size); 
      // Convert the byte array to the desired object, then return it.
    // Finalizer
      if (hKey != IntPtr.Zero) RegCloseKey(hKey);


    When the current instance gets collected as the PInvoke call to RegQueryValueEx is initiated, RegCloseKey ends up being called before the unmanaged implementation of RegQueryValueEx executes. By the time the RegQueryValueEx API processes the handle, the handle is invalid!

    This problem is easy to introduce in any managed classes that wrap unmanaged handles. Yet the time window for failure can be so small that it would usually take a really long time for you to discover the bug.

    So yes, I consider it a bug in the RegistryKey class that it doesn’t prevent this situation from happening, and yes, this should be addressed in the next major release of the .NET Framework. There are two things that the RegistryKey class could have done to prevent the RegQueryValueEx failure inside the GetValue method. One way is to use the GC.KeepAlive idiom after the PInvoke call to guarantee that the instance will at least be alive until that call:

    RegQueryValueEx(hkey, name, IntPtr.Zero, ref type, array, ref size);
    GC.KeepAlive(this); // The current instance must be kept alive to make this call

    A nicer way would be to change the first parameter of the PInvoke signature from IntPtr to System.Runtime.InteropServices.HandleRef. You construct this type with an object and an IntPtr, and when passing it to unmanged code, the marshaler keeps the object alive while it marshals the IntPtr to unmanaged code. The call to this updated signature would look as follows (with no GC.KeepAlive needed):

    RegQueryValueEx(new HandleRef(this, hkey), name, IntPtr.Zero, ref type, array, ref size);

    When you use HandleRef, you don’t require the callers of the PInvoke signature to remember these lifetime details. They’re forced into doing the right thing.

    What makes the current problem with RegistryKey even worse is that the failure of RegQueryValueEx doesn’t cause an exception! The call to RegQueryValueEx ignores the return value (which would be the Win32 ERROR_INVALID_HANDLE error code in this situation), so the implementation of GetValue doesn’t notice the failure and returns an empty string. Had the code detected this and thrown an exception with a message like "Handle is invalid," then that would have made this problem more debuggable. (It’s reasons like this that the .NET Framework design guidelines tell you to avoid indicating failure in a return value!)

    Jeroen writes, "I disagree that the code fragment shown is buggy. The bug is actually in the implementation of RegistryKey." I guess I should have said that the program had a bug, in that the code, as written, clearly doesn’t work correctly in all situations. But I’d claim that the original code snippet still arguably has a bug in that we forgot to dispose the RegistryKey instance, which is part of its contract since it implements IDisposable. And just by virtue of calling its Dispose (or Close) method after calling GetValue, there’s no longer a possibility for the instance to get collected in the middle of the GetValue call! The extra call is guaranteed to keep the RegistryKey instance alive, at least until the beginning of the Close/Dispose method. So you can avoid the subtle premature collection problem by adding a single line of code:

    public static void Main()
    // Open the key for read-only access
    RegistryKey key = Registry.LocalMachine.OpenSubKey(
    Console.WriteLine("Install root: " + key.GetValue("InstallRoot"));

    Or, of course, you can use the C# using construct which conveniently wraps the code in a try/finally block with the call to Close/Dispose in the finally block:

    public static void Main()
    using (RegistryKey key = Registry.LocalMachine.OpenSubKey(
    Console.WriteLine("Install root: " + key.GetValue("InstallRoot"));

    Fernando is correct that the JIT compiler tracks local variables differently in debuggable code versus non-debuggable code. In the debuggable case, the generated code happens to keep the RegistryKey instance alive during the entire Main method, so when the garbage collection occurs, the object is not considered eligible for collection. So although the Object Not Kept Alive probe eliminates the non-determinism in when collections occur, it can be hard to predict what the JIT compiler is going to do with your code due to various optimizations (such as inlining methods). And these optimizations may change depending on the version of the CLR or other factors. Maybe in the future there will be more probes to play with these settings as well!

  7. Adam Nathan says:

    I just got Mike’s comment and I see I repeated a lot of what he said. 🙂

    Yes, the probe does also wait at most a second for pending finalizers.

  8. When I read this I could not believe it. So I typed the following code :

    using System;

    public class Toto {

    ~Toto() {
        Console.WriteLine( "Toto.Finalize" );
    public void Test() {
        Console.WriteLine( "Toto.Test begin" );
        Console.WriteLine( "Toto.Test end" );


    public class MainApp {

    public static void Main() {
        Console.WriteLine( "MainApp.Main begin" );
        Toto t = new Toto();
        Console.WriteLine( "MainApp.Main end" );


    and seeing the following display (with /o switch) on my console make me feel really disappointed :

    MainApp.Main begin
    Toto.Test begin
    Toto.Test end
    MainApp.Main end

    I can’t understand such behaviour from the CLR. I’m in a method call. The "this" reference should not be garbage collected. It is still behing used. That’s amazing !

    So virtually, any classes could crash when the finalizer thread is kicked start. Frightening.

  9. Keith Hill says:


    While it is true that in your example the class is collected whilst you are in the middle of executing an instance method. However, in your trivial example that is OK. Theren’t aren’t any roots to this object even while your Test method is executing. However, in a perhaps more realistic scenario you would reference an instance variable in a method. That stack reference on an instance variable is enough to keep the object alive.

  10. OK, I might feel guilty for providing such test case.

    This was stupid because Test() does not need the state of its instance in order to finish properly. That’s certainly why the "this" reference is GC’ed and Finalize’d. But, I still have concerns about this weird calling order in this case.

  11. James says:

    Indeed, this is incredibly brain-damaged behavior. Aside from making a programmer’s life hell, what purpose does this ‘feature’ of the garbage collector serve?

  12. José says:

    How do you enable these probes (Object Not Kept Alive probe)?