LCG + Debuggability, and your feedback


I mentioned earlier that you can debug Reflection.Emit code. Unfortunately, Ref.Emit code can’t be unloaded unless you unload the entire appdomain. I wanted to lay out the current landscape, and then get feedback about possible solutions.


In Whidbey, we added Light Weight CodeGen (LCG), which are dynamically generated methods that can be garbage collected. This fixes the unloading problem with Ref.Emit code. Unfortunately, LCG methods aren’t really debuggable. You can’t currently associate them with source, and they’re basically hidden when debugging. In Whidbey, they show up as an “InternalFrame” marker in the callstack, which has no information other than the annotation that there is some dynamic code on the stack. If a debug event fires (such as an exception / Debug.break statement) is in dynamic method, we will stop, and then it’s nice to have that internal Frame on the callstack. But nothing is really debuggable about it in Whidbey.

In other words, with ICorDebug (and thus debuggers like VS that use that):
– You can’t set breakpoints in dynamic methods (LCG)
– You can’t see the IL of dynamic methods
– You can’t see the locals / arguments of dynamic methods
– You can’t map dynamic methods back to source code
– You can’t step into or out of dynamic methods (it’s like native code when managed-only debugging)

Workarounds?
The current workarounds are:
1) Use windbg and SOS. See demo here
2) Use Ref.Emit for code that you want to be debuggable.
3) Use other tricks, like Haibo’s IL visualizer.


But ultimately in Whidbey (.NET 2.0), you’re forced to choose between debuggability (use Ref.Emit or normal code)  vs. fine-grained unloadability beyond just appdomain unloading (use LCG).


Your feedback?
There’s a few ways the CLR can solve this, including (but not limited to):
1. Make LCG debuggable, particularly adding the ability to associate source information with LCG methods, like you do with normal Ref.Emit when you call MarkSequencePoint.
2. Make Ref.Emit code collectable, ala LCG. 


Do folks out there have any preference in solution?


I realize there’s also a large dial about what is considered debuggable. Eg, source-level debuggability, vs. assembly level vs. IL level. (I could do a whole other blog entry about that). Any comments on where the dial should be set are welcome too.

Comments (13)

  1. Ooh, please please please make Ref.Emit code collectable. And (I assume if you do THAT much it’s not a big deal to…) make Assembly.Load and LoadFrom’d code collectable too.

    This is one of the few areas where the JVM still beats the pants off the CLR. AppDomains are nice but the ability for types to be unloaded out from inside the *same* domain when they’re provably no longer used is really useful.

    I’m sure the ASP.NET team would thank you too! (as would all of us developers who have to deal with the slow memory leak that is any large ASP.NET site… until appdomain unload comes around and hangs everything for 30 seconds in the middle of some poor sucker’s pageload)

  2. andrew@pmall.com says:

    Having ability to step debug LCG methods would be great.

    Even if it is at IL level only.

  3. Tough choice, however I’d vote for making Ref.Emit code collectable, mainly because as a side effect, you would potentially improve existing libraries using Ref.Emit.

    I have used LCG a lot, and found I’ve typically generated very small methods.  There are usually to replace reflection for performance.  That being the case, I’ve not had the need to debug any LCG code.  Using the IL Visualizer has been adequate.

  4. *Please* make Ref.Emit code collectable.

  5. I was glad to hear many positive feedbacks about the DebuggerVisualizer for DynamicMethod ; on the sad

  6. David Levine says:

    I vote for making LCG debuggable. I really want a general purpose IL debugger (i.e. make IL a first-class citizen in the debugger IDE), but I’ll settle for a LCG debugger. I’ve used Haibo’s visualizer; it rocks, and it makes me want more.

  7. David – can you elaborate? There’s a spectrum of "debuggability". Are you content to be able to debug the LCG IL? Or do you want source-level debugging for LCG (ie, associate generate LCG with source) like you have for ref.emit?

  8. David Levine says:

    Hi Mike,

    I want source level debugging with LCG….and more 🙂

    Ideally it would be as easy to use, powerful, intuitive and integrated as the C# IDE debugger; for that matter, I want IL support in DevStudio (I know, it aint likely to happen, but you asked 🙂 ). It would be cool to be able to use a wizard to generate the skelton of an IL project….

    Debugging LCG is important but I really want source level debugging because I want to write some code in IL. There are a number of .NET constructs that are only possible to do in IL because the feature is not surfaced in C# (my language of choice). For example, exception filters and fault handlers….a grab-bag of utilities I can wrap up in a DLL and expose as nice, easy-to-use .net classes.

    Thanks,

    Dave

  9. David – if you had to choose between:

    1) Making Ref.Emit collectable.

    2) Making LCG be able to associate with source line info (ala MarkSequencePoint etc) and debugging that ala Ref.Emit

    which would you prefer?

    Would #1 alleviate your need for #2?

  10. David Levine says:

    I honestly can’t say which is more important.

    My gut says that I need to be able to debug any code I write.

    But….for production code it may be more important to be able to unload ref.emit. It’s a tradeoff between ease of development versus features that the user sees.

  11. I’m looking for feedback about Ref.Emit usage patterns. When using Ref.Emit , how many types do you generally

  12. dimkaz says:

    Making Ref.Emit collectable

    It covers so much more. I don’t see a loss of anything compared to LCG

  13. The CLR team recently had a compiler dev Lab on campus in building 20, with a strong focus on dynamic