Correction: you can't reasonably instantiate your own ICorDebug

There’s a significant problem for 3rd-party hobbyists trying to implement their own ICorDebug: how do you get a debugger (like VS) to instantiate your private version? I briefly talked about that here but realize that my advice there was misleading.
 
ICorDebug is just a set of interfaces; and a debugger has various ways of creating an ICorDebug object that implements those interfaces:
The V1.0, V1.1 desktop CLR ICorDebug objects were just COM-classic objects, and so a debugger would just co-create them. One perk for hobbyists here is that you can trick a debugger into creating your private version by just re-registering with the proper class ids.

For reasons discussed here, in V2.0, we switched instantiations to use a new function, CreateDebuggingInterfaceFromVersion, which takes a version parameter. The down side for hobbyists is that there's no longer an easy way to trick that into instantiating objects from your private version of mscordbi.dll. That's because there's a private protocol between CDIFV and mscordbi. I don't think there's anything to stop somebody from reverse-engineering that protocol by looking at the rotor sources and debugging CDIFV and mscordbi.dll; but such technique is certainly unsupported and not guaranteed to be robust.

Another point to reiterate is that if you own the debugger sources, you can always change it to instantiate your new version of ICorDebug. This makes Cordbg / Mdbg useful for experimentation. For example, Visual Studio has special instantiate logic for non-desktop flavors of the CLR.