Some tools work by replacing instructions in the target app. The tool may introduce a jump to some stub that will still execute the original instructions. This may work fine in native code, but you can’t safely changed arbitrary instructions in JITed (e.g. managed) code. For example, tools can’t safely rewrite the prolog of a managed method. To make matters worse, it will probably work most of the time, giving the illusion of stability.
This is because the CLR makes intimate assumptions that the jitter’s output is what actually executes, and these assumptions may change from build to build of the CLR, making them difficult to compensate for in a 3rd-party Tool. Possible assumptions include:
1. The CLR may decode its own instruction stream to collect information. This could happen with the CLR’s stackwalker.
2. The CLR has variable home and lifetime information associated with instruction ranges. This is used by the garbage collector to update references. For example, the CLR may know that local variable “abc” of method Foo is stored in register ecx for the first 10 bytes of Foo. Then during a GC, if abc is moved, the GC will update ecx to point to the new reference. If a tool has rewritten the first 10 bytes of Foo, then the variable tracking information is wrong and the GC will update the wrong thing.
3. Stubs may decode their return addresses and expect them to be in a particular range. This then breaks if a the call to a stub is moved out of that address range.
4. Some things may expect a particular instruction layout in order to identify if target code belongs to some group.
5. The GC’s suspension logic may expect certain branching and looping contracts in the code to ensure that it is suspendable.
6. The debugging services will certainly make assumptions here too. Thus rewriting the opcodes without cooperating with the CLR is practically guaranteed to break debugging. For example, the debugger
Unfortunately, several of the above are very non-deterministic. For example, GCs may occur at random times. You may be getting “lucky” and no GCs occur while the IP is in the rewritten region; and thus the tool appears to work. But then as soon as you go to demo in front of a large audience, that fateful GC will come.
One very cool technology that does this are Detours ( http://research.microsoft.com/sn/detours ). Detours can do some great things for in unmanaged code. But for reasons like the ones above, they will likely break with managed code.
[Update: 12/18/05] To clarify, I mean don’t manually rewrite native opcodes by calling WriteProcessMemory yourself. The CLR offers several “safe” APIs to do this (like the Profiler’s SetFunctionILBody, and Debugger’s EnC). These APIs change the IL and cooperate with the CLR, and thus account for the issues above.