Now that version 4 of the .NET Framework supports in-process side-by-side runtimes, is it now okay to write shell extensions in managed code?

Many years ago, I wrote, "Do not write in-process shell extensions in managed code." Since I originally wrote that article, version 4 of the .NET Framework was released, and one of the features of that version is that it supports in-process side-by-side runtimes. Does that mean that it's now okay to write shell extensions in managed code?

The answer is still no.

The Guidance for implementing in-process extensions has been revised, and it continues the recommendation against writing shell extensions and Internet Explorer extensions (and other types of in-process extensions) in managed code, even if you're using version 4 or higher.

Although version 4 addresses the side-by-side issue, it is still the case that the .NET Framework is a high-impact runtime, and that there are various part of COM interop in the .NET Framework that are not suitable for use in an extension model designed around native code.

Note that managed code remains acceptable for out-of-process extensions.

Comments (36)
  1. [ it is still the case that the .NET Framework is a high-impact runtime ]

    That's the lesson everybody should have learned from the infamous <major 3-letter graphic cards manufacturer> Control Center shell extension (right click on the desktop -> five seconds of hard disk noise -> the menu finally opens; number of times the average user used that shortcut: maybe once).

  2. Anonymous says:

    So C#/.NET is good enough for X-Box games and Exchange Server and SQL Server stored procedures but now it is no good for… an I/O-driven file browser?

    C# is the cat's whiskers even for people who mastered C/C++, and this whole internal Microsoft religious war against managed code is sending your developers and my peers to iOS in veiled anger (WinRT, anyone? no takers? /sadtrombone). Nobody likes being told what to do against what the technology makes possible, and consequently they won't be. There's a reason my favorite software company now has HALF of Apple's revenues, and it needs tough love…quick.

    [The difference is that your XBox game runs in its own process. Exchange Server and SQL Server run in their own process. In all these cases, the host process is in on the trick. But shell extensions are potentially injected into foreign processes. That's the main difference. One is a controlled environment; the other is not. If you could convince every app developer to say it's okay to load the CLR into their process, then it would be okay to load the CLR into every process. The XBox, Exchange Server, and SQL Server teams say it's okay to do that to their process. Contoso Music Designer, haven't heard back. This is an engineering decision, not a marketing decision. -Raymond]
  3. 12BitSlab says:

    different tools for different tasks.  I understand that it would be convenient to use managed code for in-process shell extensions, but that does not make it the right choice.

  4. Joshua says:

    [If you could convince every app developer to say it's okay to load the CLR into their process, then it would be okay to load the CLR into every process.]

    And you're guaranteed that at least one says no. Loading just about anything into a Cygwin process that wasn't loaded by Cygwin -> your software makes the BLODA list. COM load on the file dialog works only if COM free works correctly and the DLL is unloaded.

  5. Simon says:

    "that there are various part of COM interop in the .NET Framework that are not suitable for use in an extension model designed around native code"

    too bad the Visual Studio SDK team is unaware of that :-) It's indeed a horrible spaghetti between native and managed code, and it crashes too.

  6. CL says:

    For what it's worth, Android doesn't use a Java runtime, it has its own bytecode and VM, even though the apps are mostly written in the Java language.

  7. [ So C#/.NET is good enough for X-Box games and Exchange Server and SQL Server stored procedures but now it is no good for… an I/O-driven file browser? ]

    The problema is that C#/.NET has great performance *after it starts*, but you have a big runtime to load the first time, and, if the "reasonable expectation" for the action is to have a quick response (like, as before, a right click on the desktop) the delay is not affordable from an UX standpoint.

    That's the same story as Java applications: Android apps run smoothly on my 650 MHz phone (where the JVM is among the first things to start after the kernel), but a Java applet takes ages to load on my 3,2 GHz x 4 PC.

  8. Extendo says:

    Well darn, I was just about to try making an IE extension in C# :(

  9. EduardoS says:

    Hum… It seens the war only will be over when the .Net runtime becomes part of the kernel and already loaded once every apps loads…

  10. Harry Johnston says:

    @Joshua: fascinating, I'd never heard of BLODA.  The first app on the list should, methinks, be Cygwin itself; in most cases, from the looks of it, the problem only occurs because of Cygwin's use of unsupported native calls and improper memory mapping requirements.  Nowadays a VM-based approach would seem far more sensible, but I don't suppose that was practical back when Cygwin first appeared.

  11. The sad thing is that there would be no need of all those hacks built over the Win32 API since NT supported the idea of a "POSIX personality" from the beginning; it's a pity that Interix/SUA/however it's called now is included only in higher-end SKUs. I doubt Microsoft is actually making much money from it (which is confirmed by the fact that on Windows 8 it's being deprecated), so giving it away for free on any Windows edition "without warranties of any kind" shouldn't be such a problem.

    Actually, given that it's not deemed "strategic" anymore (again, it's deprecated on Windows 8) they could just release the sources and let anyone interested take care of it, but honestly I don't really see that happening.

    [The phrase "without warranties of any kind" never stopped a customer from asking for support. -Raymond]
  12. [The phrase "without warranties of any kind" never stopped a customer from asking for support. -Raymond]

    On the other hand, people actually asking for support to Microsoft are those who are likely to have the higher-end SKUs that already provide SUA; we "lower end SKU, OEM edition" peones don't have any right to bother the official support (unless maybe keeping the credit card ready), so that's not really a serious concern.

  13. Joshua says:

    @Harry Johnston Here fundamentally is the problem:

    There is no way in practice (and perhaps in principle) for Cygwin to accomplish its fundamental goal without the very set of assumptions about memory model it has. These assumptions were in fact TRUE on all versions of Windows in existence in which it was created.

    VM is useless here as it is the bridging technology. All other attempts have failed due to lack of maintenance. It would appear as though Cygwin, having beaten all opponents, is now threatened by its own success as Microsoft sees the loss of their implementation without the reason and so thinks there is no market.

  14. Smit-Tay says:

    L O O K !

        You know you're in trouble as soon as you have to decipher the phrase "in-process side-by-side runtimes"

    The whole issue is complete idiocy, and manifests, side-by-side, and what-ever only make it worse.  The senior developers at Microsoft must hate this stuff, being from the good old days when one wrote a single executable image, and ran it.  

    Anyone who has fought with the inane rules Microsoft has contrived in their attempt to mitigate "DLL hell" without actually solving "DLL hell" will know that this stuff just makes life harder for devs, and support people, and usually also the end user.

    Heaven forbid MS adopts the *NIX method, but no, pedantry beats simplicity any day at Microsoft.

  15. Danny says:

    It's how many years that .NET exists as M$ response to Java? Quite a few. And is Still Under eXpectations. Maybe it's time to either ditch it or make it right. Because everytime I need to write something fast I have to revert to unmanaged code. The only thing in my C# that is managed is the GUI and petite implementations of "here, let's press this button to show a message". All else that requires speed is unmanaged.

  16. Ian Boyd says:

    @Danny "everytime I need to write something fast I have to revert to unmanaged code"

    What code are you writing that is slower in C#/Java compared to native? C# and Java can be just as fast or faster because the JIT compiler can make optimizations that a C++ compiled program cannot because it can query the machine. It can determine if the machine is Intel or AMD; Pentium 4, Core Solo, or Core Duo; or if supports SSE4, etc.

  17. David Ching says:

    What would it take to rewrite Explorer in C# and support real .NET add-ins?  If the Windows team would be forced to eat .NET dogfood, they would do what was necessary to mitigate the negatives. Similar to how the Visual Studio team finally ate WPF dogfood and fixed the fuzzy font problem for VS2010.

    [Oh, you mean Windows Longhorn? -Raymond]
  18. @Ian Boyd: I'm not saying that managed code is slow (it isn't), but that's just the theory… the reality is that the JIT compiler can exploit machine-specific tricks, but won't do "expensive" optimizations because the JITter must keep a balance between "clever" optimization techniques and speed of execution (you can find many examples of this fact on StackOverflow, it's not uncommon to find questions about suboptimal JIT-generated assembly).

    A "traditional" compiler has to target "generic" CPUs (unless you explicitly tell differently – e.g. -march=native on gcc), but can take all the time/memory it wants to analyze the AST and perform complex optimizations; also, the standards for languages like C and C++ often give more opportunities to optimize with wide openings to undefined behavior (not that it's always a good thing).

    By the way, I read somewhere (maybe SO again) that not even ngen (that isn't time/memory-constrained like the JIT) performs heavy optimization, to avoid different behavior/bugs between running an assembly "normally" and running the NGEN-ed version.

  19. @David Ching: I suppose that it would be quite a pain; you'd make .NET add-ins first-class citizens, but I strongly suspect that it would break a lot of native shell extensions (heck, they break even if the current code is changed a tiny bit, go figure what would happened in a total rewrite in a managed language!); I don't even know if all the COM shell interfaces could be reimplemented in .NET without jumping through hoops. But most importantly, rewriting the shell in .NET would mean injecting the CLR in every process that makes use of the relevant shell facilities (e.g. the open file dialog), which is exactly what Raymond described as a bad idea.

  20. @Ian Boyd:  You just quoted the theoretical reasons why managed code is faster.  In practice, very few if any of those JIT optimizations are realistically possible and, also in practice, many of the theoretical optimizations defeat superior optimizations that may be obtained from static analysis.  Even if all those optimizations were possible and did actually take place, the other problem that you have to take into account is that much of the supposed performance benefit of managed code comes not from less or more efficient code, but simply deferred inefficiency.  That is, the process of returning memory to the system is shelved until your process is not busy doing other things.

    That's great, right up to the point that your system (as a whole or just your process) becomes memory constrained, at which point performance in your managed code then tanks as all that housekeeping work HAS to happen RIGHT NOW, and the business of work that your process wants to get on with just has to wait.

    So yes, managed code can be faster, but it might also run into a performance wall.  If you need RELIABLY FAST then taking ownership of the housekeeping is the only way to ensure that ALL aspects of your code are performed optimally, not just the "fun stuff".

  21. Ian Boyd says:

    There's a popular archiving program that adds context menu items, and icons, to "All" files and folders. i know that every time i right-click a file, this shell extension is triggering a page-fault to go load the icons, and analyzing the file selection to decide which menu items should appear.

    i always reconfigure the *extraordinarily common* shell extension, on any computer or server i use, to "cascade" the menu items to a sub-item, and to disable adding icons to the context menu items.

    Explorer is a high-traffic area, that needs more careful thought before applications simply add things in.

  22. Harry Johnston says:

    @Joshua: I see no reason (either in principle or in practice!) why Cygwin couldn't solve its problems by running the Unixy processes inside a VM, other than the fact that to do so at this point would involve rewriting the entire thing (and would probably break binary compatibility into the bargain).

    If anyone cared enough to build a more reliable solution from scratch, I think that would be the way to go.

  23. David Ching says:

    @Raymond:  Longhorn did indeed have perf issues, but what of Win Phone 7 and Android, whose app model is based on code running in a VM?  Why do they not have perf issues?

    [You're assuming that perf was the only issue. -Raymond]
  24. LonghornIssues says:

    As David Ching, I thought that the main issue of Longhorn was the perf one. So, what were the other issues? Thanks.

  25. Joshua says:

    As you are probably aware, Raymond does not discuss products that aren't released in any way in which they can be identified.

  26. @Harry: if anyone cared enough to build a more reliable solution from scratch I don't see why there would be a need to add the overhead of a VM and a virtualized kernel – as said above, the NT kernel already supports the "personalities" concept, and a POSIX subsystem has already been written and is being used (so, it *can* be done), so *that* would be the "clean" solution. On the other hand, I don't know how many people actually have the knowledge to write an NT POSIX subsystem from scratch…

  27. ender says:

    Since Cygwin originally supported Win9x/ME, it couldn't have relied on the NT kernel subsystems. Any by the time 9x/ME support was dropped, it was way too late to change that.

  28. Joe says:

    Based on the fact that there was a Windows XP embedded and a Windows 7 embedded, but no Vista/longhorn embedded, and vague statements by some developers on the embedded team, I've surmised that the internals of Vista were such an intertwined mess that it couldn't be componentized. I also suspect there were WDDM issues to be ironed out. That said, Vista was much better than it's reputation.

  29. Rick C says:

    @LonghornIssues, it's right there in Raymond's first link.

  30. LonghornIssues says:

    @Rick C: Please be more specific. Raymond's first link is about side by side CLRs: that's now fixed with .NET 4. I was asking about non-perf issues in Longhorn (because David Ching's point seemed valid to me). Please clarify.

  31. Harry Johnston says:

    @Matteo: I thought only Microsoft could write NT subsystems?  At any rate, I can't find any documentation on MSDN.

    VM software isn't all that big, and Cygwin isn't exactly slender, so I don't think the overhead would be all that much of a problem.  Platform fragmentation might be, though, since you'd have to support systems that have Hyper-V running as well as systems that don't.  (Can you install something like VirtualBox side-by-side with Hyper-V, I wonder?)

  32. @Harry: I too thought only Microsoft could write Windows NT, but the ReactOS guys managed to reverse-engineer a good part of it. :)

  33. ender says:

    @Harry: if Hyper-V role is installed, no other virtualization program that relies on hardware virtualization will work.

  34. John Doe says:

    Cygwin is proposedly not a subsystem, it's based on Win32. That's why its fork() sucks so much, among other things. But hey, if you're using Cygwin for production, You're Asking For It™.

  35. Joshua says:

    @John Doe: No choice. It's the only to get a working sshd on Windows.

    Short comment is short.

  36. Mike Dimmick says:

    LonghornIssues: *Dependencies* were the problem. .NET Framework 2.0 could not stabilize because WinFS and SQLCLR were pulling it in two different directions and WPF was pulling it a third. Each time something changed it broke all the user scenarios built on top of WPF. It pushed out the release schedule for VS 2005, SQL Server 2005, Exchange 2007 (which was supposed to have a SQL Server back end – still doesn't in 2013), Windows 'Longhorn' (client and server). Eventually Jim Allchin cried 'enough' and WPF was pulled from .NET 2.0, WinFS was killed, and Windows Vista rebooted to not use any managed code.

    It's very clear that Windows Runtime is such a minimal API because they have tried to avoid this problem: the runtime's feature set was locked about three years ago, so that apps could be built on it without the shifting-sands problem. Lack of updates to the in-box Windows Store apps are likely to be because the runtime is now in flux again for the next release, and the app teams are now working toward their next releases on top of that, not on Windows Runtime 1.0. You'll still notice many areas of Windows 8 where you get booted out of the 'modern' environment into the desktop, even for some apparently straightforward configuration tasks. Why? Because there wasn't enough time to do that on top of Windows Runtime as well as everything else.

Comments are closed.