Tell us about your experience with COM interop


COM interoperability is a very powerful feature in the CLR that bridges the gap between native and managed code. It lets you reuse your existing COM and non-COM native DLL in the managed environment easily. It also lets native code call back to managed code in a seamless way.


 


It provides a wealth of features which I wanted to talk about here in some detail.


 



  1. tlbimp

  2. tlbexp

  3. tlbref

  4. regasm

  5. RCW and Life Time Management

  6. CCW

  7. Marshalling and Custom Marshalling

  8. pInvoke

  9. Eventing

  10. System.Runtime.Interopservices namespace

  11. MDA and debugging

  12. Threading/Apartment/Context

  13. Availability of documentation with code samples

 


There are inherent differences in the way native code, COM code and managed code work. This adds to lots of confusion in the ways they interact and work together. This in my opinion has been a significant pain point for users of this technology. I also feel that this area is highly complex when it comes to identifying and solving a scenario, task or problem that you have to solve. As an example, I have been asked the question of when should users use custom marshalling many times now. We want to make the user/developer experience with this feature better and are looking for some input on the same. Here are some questions that I have that will help me understand the problems that our customers face with this technology


 


Questions:



  1. Knowledge/Skill of user


    1. Is your usage of COM primarily as a component consumer (e.g. via RAD-style usage in an IDE) or component developer (e.g. via ATL)?

    2. Have you developed a COM/Unmanaged component in a commercial software package?

    3. Have you developed a commercial software package on the Visual Basic platform (specifically VB6/VB.NET)?

    4. Do you view your usage of COM Interop as transitional/migratory, or is it a permanent element of your architecture going forward? How skilled are your developers with unmanaged, COM and managed code?

    5. Has the COM Interop functionality from the CLR played a role in your application compatibility / backwards compatibility story? Do you find your adoption of .NET is biased by legacy needs?

  2. Customer Scenario and choice of technology


    1. What is your end user customer scenario? (Ex: Five components A, B, C, D, E. A is a legacy COM component. B is our UI/Business application in C# which needs functionality from A. etc)

    2. What is the scenario inside that which drives you to use COM interop as a solution? (Ex: Reuse of existing functionality/library that is available in Com from managed code. Exposing managed API’s for COM to customers etc)

    3. How did you decide that COM interop was the best solution for the problem?

    4. How difficult was the decision and what were the alternatives that you considered? (Ex: complete native implementation, re-implementing in managed, managed C++ bridge, COM interop etc)

    5. What features of COM interop do you use and for what scenarios and reasons? If disparate state them separately with explanation. (Ex: tlbimp, RCW, CCW, Eventing, pInvoke, Marshalling etc)

    6. What is the diversity of your environment in terms of architecture (32 and 64 bit) and languages (C++, COM, VB, C# etc)?

  3. Developer experience during development


    1. Do you use the COM interop technologies from Visual Studio or from command prompt? (Ex: run tlbimp from command or import through Visual Studio)

    2. What parts of the COM interop technology from the list above does development use to solve the above scenario?

    3. What is the heaviest of the above that makes the meat of the product? (Ex: Marshalling, Eventing etc)

    4. What are the most complicated problems that you have encountered during development and in which areas?

    5. How do you investigate problems in this area? (Ex: Use MDA, Debug using windbg etc)

  4. Servicing experience


    1. Have you encountered significant end user problems because of failures in your COM interop later? If yes, which part of COM interop did they come from? (List top 4 areas)

    2. What were the problems in most of the cases? Was it with the platform or your implementation?

    3. How did you identify the problem?

    4. How did you solve the problem?

  5. Debugging experience


    1. How do you debug problems in COM interop?

    2. Do you use windbg, SOS, public symbols etc?

    3. Do you use MDA’s for your debugging? (http://msdn2.microsoft.com/en-us/library/d21c150d.aspx)

    4. Do you look at you native and managed code in oleview and ildasm, reflector to see what is happening under the hood?

    5. What is your average time spent in a problem in this area? How many problem have you seen in this area in a given product cycle (months)?

  6. Help and documentation available


    1. What specific material do you refer for this area and how useful is it?

    2. Do you know about pInvoke.net? Do you use it?

    3. How would you rate MSDN documentation for this area? What is lacking if any?

    4. Do you find enough samples available to get you on the ramp with the various technologies here?

    5. How do you rate our help and documentation on MDA’s?

  7. Other – Any other things that you want us to know


I want to be clear that I am not looking for adding new features to this area, but looking for making the experience with the current features easier and better. Your input here will be valuable in driving this technology in the right direction.


 


Comments (27)

  1. Friends Do Not Let Friends Use COM says:

    ·  Knowledge/Skill of user

    a. Is your usage of COM primarily as a component consumer (e.g. via RAD-style usage in an IDE) or component developer (e.g. via ATL)?

    Presently I refuse to create any new COM components. I consume them as a last resort. In the past, I used to develop them, both manually and with ATL.

    b. Have you developed a COM/Unmanaged component in a commercial software package?

    Yes.

    c. Have you developed a commercial software package on the Visual Basic platform (specifically VB6/VB.NET)?

    No. I work strictly in C++ and C#.

    d. Do you view your usage of COM Interop as transitional/migratory, or is it a permanent element of your architecture going forward? How skilled are your developers with unmanaged, COM and managed code?

    COM interop is a last resort. When possible, I choose to rewrite the component in C#.

    e. Has the COM Interop functionality from the CLR played a role in your application compatibility / backwards compatibility story? Do you find your adoption of .NET is biased by legacy needs?

    As COM interop is a last resort, it plays a very minimal role. I will rewrite when possible. COM interop plays a lesser and lesser role with each year.

    ·  Customer Scenario and choice of technology

    a. What is your end user customer scenario? (Ex: Five components A, B, C, D, E. A is a legacy COM component. B is our UI/Business application in C# which needs functionality from A. etc)

    At present, none anymore. There were a few legacy COM components, but they have been rewritten in C#.

    b. What is the scenario inside that which drives you to use COM interop as a solution? (Ex: Reuse of existing functionality/library that is available in Com from managed code. Exposing managed API’s for COM to customers etc)

    The lack of time or resources to rewrite the COM component in managed code.

    c. How did you decide that COM interop was the best solution for the problem?

    In my opinion, it is never the best solution.

    Use it as a final, last resort.

    And know that you will be fixing that mistake later in the future.

    d. How difficult was the decision and what were the alternatives that you considered? (Ex: complete native implementation, re-implementing in managed, managed C++ bridge, COM interop etc)

    The main factors where whether I had access to the original code and how long it would take to rewrite the COM component in managed code. (Native/managed mixed) Managed C++ is jus another fudge like COM interop; it may provide a quick solution, but it is hardly a good one.

    e. What features of COM interop do you use and for what scenarios and reasons? If disparate state them separately with explanation. (Ex: tlbimp, RCW, CCW, Eventing, pInvoke, Marshalling etc)

    Tlbimp, pInvoke, marshalling.

    f. What is the diversity of your environment in terms of architecture (32 and 64 bit) and languages (C++, COM, VB, C# etc)?

    Generally 32-bit and C#. The amount of C++ and COM diminishes more each year.

    ·  Developer experience during development

    a. Do you use the COM interop technologies from Visual Studio or from command prompt? (Ex: run tlbimp from command or import through Visual Studio)

    When used, generally though Visual Studio.

    b. What parts of the COM interop technology from the list above does development use to solve the above scenario?

    Generally N/A because I end up rewriting it in managed code.

    c. What is the heaviest of the above that makes the meat of the product? (Ex: Marshalling, Eventing etc)

    Again, generally N/A anymore.

    d. What are the most complicated problems that you have encountered during development and in which areas?

    Debugging problems.

    e. How do you investigate problems in this area? (Ex: Use MDA, Debug using windbg etc)

    Usually Visual Studio.

    ·  Servicing experience

    a. Have you encountered significant end user problems because of failures in your COM interop later? If yes, which part of COM interop did they come from? (List top 4 areas)

    Most of my problems were from COM itself, not interop.

    For example, incomplete COM registration errors.

    b. What were the problems in most of the cases? Was it with the platform or your implementation?

    Platform.

    c. How did you identify the problem?

    Debugging.

    d. How did you solve the problem?

    Debugging.

    ·  Debugging experience

    a. How do you debug problems in COM interop?

    Visual Studio.

    b. Do you use windbg, SOS, public symbols etc?

    Public symbols.

    c. Do you use MDA’s for your debugging? (http://msdn2.microsoft.com/en-us/library/d21c150d.aspx)

    No.

    d. Do you look at you native and managed code in oleview and ildasm, reflector to see what is happening under the hood?

    All of the time.

    e. What is your average time spent in a problem in this area? How many problem have you seen in this area in a given product cycle (months)?

    Some issues a few hours. Others a few weeks.

    ·  Help and documentation available

    a. What specific material do you refer for this area and how useful is it?

    I search for reports of similar problems.

    b. Do you know about pInvoke.net? Do you use it?

    Yes and no.

    c. How would you rate MSDN documentation for this area? What is lacking if any?

    Typically poor. You need more examples.

    d. Do you find enough samples available to get you on the ramp with the various technologies here?

    Absolutely not.

    e. How do you rate our help and documentation on MDA’s?

    Typically poor. But there are some good blog posts about it.

    ·  Other – Any other things that you want us to know

    COM was truly the Dark Ages for computer programming. I initially viewed .Net as a clean break from the horror and nightmares that it has caused. Unfortunately, COM still exists, and it still causes problems. The most disappointing thing is that COM is to blame for unfixable bugs between versions. A pure .Net component has excellent versioning controls and will execute on the version that it was designed for. COM, on the other hand, doesn’t (I think it loads the latest version). Thus, recognized bugs cannot be fixed because it could potentially break COM components that rely on them. I would so love if you could fix your bugs without having to worry about backward compatibility. COM is ruining the .Net experience.

    I try to minimize the amount of COM that I use. If I have the code, I will generally rewrite the component in pure C#. Interop is a last resort. The less COM, the better.

  2. borisj says:

    Why no mention of direct C++ interop using C++/CLI?

  3. Robert says:

    I think spending resources making COM interop easier is not the way to go.

    Make it unnecessary by means of managed wrappers.  There are still too many pieces

    of Win32 with no wrappers.

    I was hoping for more of this with Vista, but it looks like most of that got chopped.

    A set of questions for another topic would be:

    a) Windows Platform interrop,

    b) 3rd party dll’s, ocx’s etc,  

    c) legacy in house code.

    If A, then find out the best subsystems to wrap next..  

  4. Ryan Cromwell says:

    Sorry, don’t have time to go through the full list, but I’ll give you our current story.

    We have a third party platform in which Socket communication is the only mechanism for interop.  They also have a limited number of connections available and we have to leave the sockets open rather than open/close for performance reasons.  In our original release (this is a Loyalty/CRM system for one of the largest Conv Store chains in the  country) we used in process connection pooling via our own BlockingQueue in .Net 1.1.   As we started adding new features and functionality we found it hard to manage the number of connections being used and limit them.  We also started to see idle apps taking up vital connections.  

    To solve this we went to a COM+ Server Application which we use Object Pooling for.  We reduced the amount of code and now have the ability to manage the number of open/available connections over all applications in one place.  It’s worked beautifully and hasn’t been shutdown in almost a year.  We actually use Proxy’s on our development machines against development servers to avoid having to set everything up every time.  

    We process upwards of 20 socket messages/second over this COM+ Server and I’ve seen it spike above 100 after client/network outages that force queued transactions up in batches.

  5. rsclient says:

    Questions:

      1. Knowledge/Skill of user

            1. Is your usage of COM primarily as a component consumer (e.g. via RAD-style usage in an IDE) or component developer (e.g. via ATL)?

    ANSWER: component developer

            2. Have you developed a COM/Unmanaged component in a commercial software package?

    ANSWER: Yes

            3. Have you developed a commercial software package on the Visual Basic platform (specifically VB6/VB.NET)?

    ANSWER: No

            4. Do you view your usage of COM Interop as transitional/migratory, or is it a permanent element of your architecture going forward? How skilled are your developers with unmanaged, COM and managed code?

    ANSWER: cannot answer

            5. Has the COM Interop functionality from the CLR played a role in your application compatibility / backwards compatibility story? Do you find your adoption of .NET is biased by legacy needs?

    ANSWER: COM Interop was used only to deal with a (explitive deleted) other bit of software from a "major vendor".

    ANSWER: Adoption of .NET is hindered only by the awful run time size and interopability issues.

      2. Customer Scenario and choice of technology

            1. What is your end user customer scenario? (Ex: Five components A, B, C, D, E. A is a legacy COM component. B is our UI/Business application in C# which needs functionality from A. etc)

    ANSWER: One .NET component from a "major vendor" and our COM component.

            2. What is the scenario inside that which drives you to use COM interop as a solution? (Ex: Reuse of existing functionality/library that is available in Com from managed code. Exposing managed API’s for COM to customers etc)

    ANSWER: Business requirement for our COM object to interact with a .NET thing made by a "major vender"

            3. How did you decide that COM interop was the best solution for the problem?

    ANSWER: no other solution possible.  Could not rewrite component from "major vender"; could not possibly rewrite out (large) COM object.

            4. How difficult was the decision and what were the alternatives that you considered? (Ex: complete native implementation, re-implementing in managed, managed C++ bridge, COM interop etc)

    ANSWER: only one possible solution was found.

            5. What features of COM interop do you use and for what scenarios and reasons? If disparate state them separately with explanation. (Ex: tlbimp, RCW, CCW, Eventing, pInvoke, Marshalling etc)

    ANSWER: cannot answer.

            6. What is the diversity of your environment in terms of architecture (32 and 64 bit) and languages (C++, COM, VB, C# etc)?

    ANSWER: 32 bit, c++, support for COM languages

      3. Developer experience during development

            1. Do you use the COM interop technologies from Visual Studio or from command prompt? (Ex: run tlbimp from command or import through Visual Studio)

    ANSWER: Visual Studio.  No other tools were documented.

            2. What parts of the COM interop technology from the list above does development use to solve the above scenario?

    ANSWER: no answer

            3. What is the heaviest of the above that makes the meat of the product? (Ex: Marshalling, Eventing etc)

    ANSWER: interop is the glue between the "major vender"s product and ours.  It should ideally be so lightweight and invisible that we should not even realize that it is there.

            4. What are the most complicated problems that you have encountered during development and in which areas?

    ANSWER: the fact that COM interop works 99.9% of the time — which means that for every million tests, one thousand fail.  The failures are hard to reproduce, and when they can be reproduced, are not fixable in any useful way.

    For example, every now and then the COM/Interop will decide — for no good reason — that it has to SEND a message to my component.  Except that my component called into the interop with a SEND message, and it waiting for the answer.  Result: a hang that cannot be gotten rid of with major changes to our component.

            5. How do you investigate problems in this area? (Ex: Use MDA, Debug using windbg etc)

    ANSWER: Visual Studio.  No other tools were documented.

      4. Servicing experience

            1. Have you encountered significant end user problems because of failures in your COM interop later? If yes, which part of COM interop did they come from? (List top 4 areas)

    ANSWER: COM/Interop works 99.9% of the time; the rest of the time it fails, generally by causing the entire application to hang.

            2. What were the problems in most of the cases? Was it with the platform or your implementation?

    ANSWER: the platform.

            3. How did you identify the problem?

    ANSWER: we cursed Microsoft a lot.  Also the salespeople that brought this particular contract to us.  But mostly Microsoft for doing unsafe, undocumented actions in the middle of "working"

            4. How did you solve the problem?

    ANSWER: lots of swearing and hoping.

      5. Debugging experience

            1. How do you debug problems in COM interop?

    ANSWER: with a lot of swearing.

            2. Do you use windbg, SOS, public symbols etc?

    ANSWER: no, because the problems are that the (deleted) thing hangs every now and then.

            3. Do you use MDA’s for your debugging? (http://msdn2.microsoft.com/en-us/library/d21c150d.aspx)

    ANWER: never heard of them.  Maybe if it had been documented when the project was underway we would have.

            4. Do you look at you native and managed code in oleview and ildasm, reflector to see what is happening under the hood?

    ANSWER: no

            5. What is your average time spent in a problem in this area? How many problem have you seen in this area in a given product cycle (months)?

    ANSWER: mean or median?  median is a few hours.  mean is closer to a week.

      6. Help and documentation available

            1. What specific material do you refer for this area and how useful is it?

    ANSWER: documented is largely useless.

            2. Do you know about pInvoke.net? Do you use it?

            3. How would you rate MSDN documentation for this area? What is lacking if any?

    ANSWER: Documentation is pitful.  It claims that everything just works, the work is tra-la-la, and there are never issues.

            4. Do you find enough samples available to get you on the ramp with the various technologies here?

    ANSWER: samples are useless.

            5. How do you rate our help and documentation on MDA’s?

    ANSWER: no answer.

      7. Other – Any other things that you want us to know

    ANSWER: It’s com/interops inability to work 100% that is the primary issue.  When it works, it just works; when it fails, it’s a pain to even look at.   Too much "magic" happens behind the scenes; the majic fails much to often.

  6. Matt Davis says:

    Knowledge/Skill of user

    Is your usage of COM primarily as a component consumer (e.g. via RAD-style usage in an IDE) or component developer (e.g. via ATL)?

    – Both, but mostly component development

    Have you developed a COM/Unmanaged component in a commercial software package?

    – Yes, we ship a number of them in our financial application suite.

    Have you developed a commercial software package on the Visual Basic platform (specifically VB6/VB.NET)?

    – Yes, much of our legacy code is VB6/VBA.

    Do you view your usage of COM Interop as transitional/migratory, or is it a permanent element of your architecture going forward? How skilled are your developers with unmanaged, COM and managed code?

    – Permanent element. We use COM interop to bridge legacy code to managed infrastructure- those parts will go away as more legacy code moves to managed. But, we also use COM interop to expose our managed APIs to VBA for customer extensibility.

    Has the COM Interop functionality from the CLR played a role in your application compatibility / backwards compatibility story? Do you find your adoption of .NET is biased by legacy needs?

    – Absolutely. The ability to reimplement existing COM interfaces in managed code without affecting existing end-user code has sped our adoption significantly.

    Customer Scenario and choice of technology

    What is your end user customer scenario? (Ex: Five components A, B, C, D, E. A is a legacy COM component. B is our UI/Business application in C# which needs functionality from A. etc)

    – Client application is an Excel "dictator app"- consists of a managed launcher .exe that starts an Excel instance via COM interop, loads a custom unmanaged COM addin shim into Excel that spins up a CLR and loads our managed code into a new AppDomain there. Managed code takes over, driving Excel in-process, loading and driving legacy VBA and VB, all via RCWs. Legacy VBA/VB6 also calls back into managed objects via CCWs.

    What is the scenario inside that which drives you to use COM interop as a solution? (Ex: Reuse of existing functionality/library that is available in Com from managed code. Exposing managed API’s for COM to customers etc)

    – Our product relies heavily on Excel, legacy VB6 and VBA, for both internal use and external customer extensibility. We wanted to leverage managed code for new infrastructure without rewriting the entire product at once or requiring customers to reimplement their custom extensions. COM interop lets us do exactly that.

    How did you decide that COM interop was the best solution for the problem?

    – It was by far the simplest solution that still let us use managed code.

    How difficult was the decision and what were the alternatives that you considered? (Ex: complete native implementation, re-implementing in managed, managed C++ bridge, COM interop etc)

    – Pretty easy decision

    What features of COM interop do you use and for what scenarios and reasons? If disparate state them separately with explanation. (Ex: tlbimp, RCW, CCW, Eventing, pInvoke, Marshalling etc)

    – tlbimp (get managed redeclarations of IDL-defined interfaces that can’t be fully described in C#, creating RCWs for Excel 2000, other in-house and 3rd-party legacy components)

    – tlbexp (publish typelibs for internal and customer use against extensibility assemblies)

    – RCW/CCW (driving our VB6/VBA objects from managed code and vice-versa)

    – Eventing (consuming Excel and internal events, raising internal events, allowing customer VBA extensions to consume managed events)

    – P/Invoke (drive internal legacy code wrapped by managed code, bridge to Excel’s XLL C SDK)

    – Marshaling (no custom marshaling, mostly OA marshaler to ship interfaces between processes/AppDomains)

    – Type attributing (where possible for creation of TLBs- often must resort to MIDL for "full-fidelity" typelibs)

    What is the diversity of your environment in terms of architecture (32 and 64 bit) and languages (C++, COM, VB, C# etc)?

    – Currently only support 32-bit, w/ C++ (managed and unmanaged w/ ATL), C#, VB6, VBA

    Developer experience during development

    Do you use the COM interop technologies from Visual Studio or from command prompt? (Ex: run tlbimp from command or import through Visual Studio)

    – Mostly command-line except for quick-n-dirty prototyping. We’d rather have the ability to sign/strong-name interop assemblies, control names/namespacing, etc.

    What parts of the COM interop technology from the list above does development use to solve the above scenario?

    – tlbimp, tlbexp, type attributing

    What is the heaviest of the above that makes the meat of the product? (Ex: Marshalling, Eventing etc)

    – RCW/CCW, marshaling

    What are the most complicated problems that you have encountered during development and in which areas?

    – typelibs/interface definitions out of sync (usually crashes)

    – "low-fidelity" typelib creation (missing type attributes/interface layout options from IDL)

    – problematic managed-to-VB6 ActiveX DLL marshaling (classes marshaled must be ComVisible or only IUnknown is available, even if other ComVisible interfaces are present)

    How do you investigate problems in this area? (Ex: Use MDA, Debug using windbg etc)

    – Use MDAs if available

    – Try to build simpler test cases

    – WinDbg + SOS w/ symbol server (thanks for publishing CLR/BCL symbols!)

    Servicing experience

    Have you encountered significant end user problems because of failures in your COM interop later? If yes, which part of COM interop did they come from? (List top 4 areas) What were the problems in most of the cases? Was it with the platform or your implementation? How did you identify the problem? How did you solve the problem?

    – CCWs not released under some exception/failure HRESULT conditions (platform bug fixed in 1.1 SP). Noticed that we were leaking native objects when a failure HRESULT had been returned. Waited for hotfix.

    – Shipped out-of-sync interface definition between managed/native (our bug, but wouldn’t have occurred if platform supported more IDL attributes in managed interface definitions). Used tracing to narrow down to a rarely called method in customer code that was occasionally crashing.

    Debugging experience

    How do you debug problems in COM interop? Do you use windbg, SOS, public symbols etc? Do you use MDA’s for your debugging? (http://msdn2.microsoft.com/en-us/library/d21c150d.aspx) Do you look at you native and managed code in oleview and ildasm, reflector to see what is happening under the hood?

    – Yes to all. We try to use WinDbg as a last resort, but it has been very useful at times, especially since most CLR/BCL symbols are on the symbolserver. If only Office would do it too. 🙂 Reflector is also very useful.

    What is your average time spent in a problem in this area? How many problem have you seen in this area in a given product cycle (months)?

    – Usually figured out within a few hours, though the platform bug with exceptions took us several weeks to narrow down. We’re not seeing issues too often as our developers learn "what not to do".

    Help and documentation available

    What specific material do you refer for this area and how useful is it?

    – Adam Nathan’s book, MSDN, PSS

    Do you know about pInvoke.net? Do you use it?

    – Yes, and yes, but "with a grain of salt"- we’ve been bitten by subtle errors, especially in struct layouts.

    How would you rate MSDN documentation for this area? What is lacking if any?

    – Pretty decent. I’d like to see a little more of the "deep down" stuff around marshaling, especially documentation of the "special cases" that are deep in the CLR during marshaling/casting where TP/RP/MBRO/__ComObject are near each other, and what’s happening to preserve CLR type-identity in a managed->unmanaged->managed marshaling scenario (marker interfaces?) Rotor’s no help here, since it doesn’t include the full interop stack, and the CLR implements most of this magic in unmanaged code, where Reflector can’t help either. I’ve had weird behaviors with intersections of these technologies where I’ve had to guess what’s going on with WinDbg+symbol names or by crafting experiments.

    Do you find enough samples available to get you on the ramp with the various technologies here? How do you rate our help and documentation on MDA’s?

    – Adam Nathan’s book was the best resource for COM interop in general. The in-box docs are getting better. It was maddening during 2.0 beta because we didn’t understand why some MDAs were firing, but between the MSDN articles and the platform docs, it’s fine now.

    Other – Any other things that you want us to know

    – Not really. It’s a great stack and has enabled us to do lots of really cool stuff. There are a few rough spots that I wish would get ironed out (IDL attribute fidelity, especially), but overall we’re happy.

  7. Ben says:

    I have experience with COM Interop from my work in the defense field.

    COM represents everything that is bad about programming and software technology. Poorly/un -documented. Fiddly. Cryptic error messages. Difficult to debug. Innumerable ways to achieve the same effect, but each with subtle differences. Poor developer tools support.

    Put simply, completely non human-friendly.

    IMO the COM interop mechanism for .NET does not hide enough of the underlying COM from the programmer.

    Seeing as much of Microsoft Windows NTx (and even Vista) is based-on or uses COM, Microsoft should provide an improved and simpler mechanism for programmers wishing to integrate COM with .NET.

  8. Thottam Sriram says:

    We do support wrappers in the form of pInvoke today. It is too complicated to provide a library of wrappers to all existing API’s and hence we have an open invocation model for unmanaged functions.

    The mechanism that we have provided to interact with unmanaged and COM components is by using COM interop. The managed implementation tries to hide a good amount of COM implementation that you don’t have to worry about like GUIDS, CoCreateInstances etc. I agree that it also has its complications.

    I will post a summary of my learning from this post soon.

  9. Bob S says:

    Here is a major issue I have encountered with COM Interop – its a MSDN newsgroup posting I made yesterday :

    A class in assembly A implements a COM interface as follows :

    [ComImport, Guid("CEF04FDF-FE72-11d2-87A5-00C04F6837CF"),

    InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]

    internal interface IMyInterface

    {

    [PreserveSig]

    int MyMethod();

    }

    class MyClass : IMyInterface

    {

    int MyMethod()

    {

    }

    }

    This interface is exposed to the unmanaged world via COM Interop.

    Now, assembly B also defines IMyInterface and gets this interface from the

    unmanaged world as follows :

    IntPtr unk = GetMyInterface(); // Gets IMyInterface from unmanaged world

    object obj = Marshal.GetObjectFromIUnkown(unk); // its really the class

    MyClass from assembly A above

    IMyInterface myIntf = obj as IMyInterface; // this cast does not work and

    returns null becuase of separate declarations of IMyInterface in the two

    assemblies !

    How to get this scenario to work? One would think that .Net would recognize

    that both the interfaces are the same ( based on the ComImport and Guid

    attributes) and allow the cast, but it doesnt allow the cast.

    It is not possible for me to use a shared assembly and put the declaration

    of IMyInterface in that assembly.

    So how can I solve this problem?

    Is not always possible to have the interface in the shared assembly, because

    one .Net assembly may expose an interface (eg COM IDataObject) to the

    unmanaged world while another .Net assembly may consume the interface.

    Though the underlying interface is the same, .Net will not allow the

    casting. (In the case of IDataObject, this situation will not arise if you

    use the Microsoft defined type IOleDataObject – however this will not be

    possible for other interfaces).

    Thus .Net does not support the basic tenet of COM which says that COM is a

    "binary standard". It should not matter what the underlying ".Net type" of

    the interface is as long as the binary contract of the interface is the

    same.

    I think this is a serious flaw in COM support of .Net. I would love to hear

    some more views on this.

    Regards

    Bob

  10. Jude says:

    I’ve been investigating COM interop but haven’t started it yet.  

    1 I’ve both developed and consumed COM components, including shipping COM as part of a commercial product.  

     We have an older product in C++ and a newer one in C#.

     If I have to use COM interop I hope it will be transitional.  

    2 In this instance we have .Net(c#) component which needs to import data from MS Word.

     Direct usage of the Word interop wrapper is proving to be too slow because the interface ends up being chatty.

     At the minute I am looking into writing a COM wrapper for the Word COM object which does the preprocessing necessary for my application and interacts with my application in bigger chunks.

     Newer versions of the Word object model may help but for now we still have to support older versions of Word.

    This will be a last resort though, after all other performance improvements have been tried.

  11. B.Wa says:

    I have worked daily for the last couple of years with COM interop and the underlying framework and toolset. One short thing I WISH were added to the interop (I know you weren’t looking to add, but thought I would mention):

    Disposable RCW’s that did a hard-release on the underlying COM object. This would of made our .NET experience much more natural and intuitive and our COM experience predictable (not to mention saving weeks of work). =)

    Thanks for gathering feedback!

  12. pm says:

    COM interop works great (providing you have adam nathan’s book!)

    The hard one is p/invoke with non-trivial function signatures. Try wrapping (for example) AcceptSecurityContext. The new MDAs help but its still a lot of trial and error. I still end up write clr/c++ code to wrap the win32 calls in many cases.

    Solutions

    a) all win32 wrapped

    b) a wizard in vs

    c) a file of declarations (VB had this ages ago )

  13. Vivek says:

    1. Knowledge/Skill of user

    a. Is your usage of COM primarily as a component consumer (e.g. via RAD-style usage in an IDE) or component developer (e.g. via ATL)?

    ANSWER: CONSUMER AND DEVELOPER

    b. Have you developed a COM/Unmanaged component in a commercial software package?

    ANSWER: YES

    c. Have you developed a commercial software package on the Visual Basic platform (specifically VB6/VB.NET)?

    ANSWER: NO

    d. Do you view your usage of COM Interop as transitional/migratory, or is it a permanent element of your architecture going forward? How skilled are your developers with unmanaged, COM and managed code?

    ANSWER: PERMANENT. GOOD

    e. Has the COM Interop functionality from the CLR played a role in your application compatibility / backwards compatibility story? Do you find your adoption of .NET is biased by legacy needs?

    ANSWER: YES. YES

    2. Customer Scenario and choice of technology

    a. What is your end user customer scenario? (Ex: Five components A, B, C, D, E. A is a legacy COM component. B is our UI/Business application in C# which needs functionality from A. etc)

    ANSWER: A IS COM SERVER AND B IS C# .NET CLIENT

    b. What is the scenario inside that which drives you to use COM interop as a solution? (Ex: Reuse of existing functionality/library that is available in Com from managed code. Exposing managed API’s for COM to customers etc)

    ANSWER: COM API IN UNMANAGED CODE AND THE CLIENT IS IN MANAGED CODE. BASICALLY USING COM API’S

    c. How did you decide that COM interop was the best solution for the problem?

    ANSWER: SIMPLER AND MANAGED SOLUTION.

    d. How difficult was the decision and what were the alternatives that you considered? (Ex: complete native implementation, re-implementing in managed, managed C++ bridge, COM interop etc)

    ANSWER: NOT AT ALL.

    e. What features of COM interop do you use and for what scenarios and reasons? If disparate state them separately with explanation. (Ex: tlbimp, RCW, CCW, Eventing, pInvoke, Marshalling etc)

    ANSWER: TLBIMP,RCW, MARSHALLING.

    f. What is the diversity of your environment in terms of architecture (32 and 64 bit) and languages (C++, COM, VB, C# etc)?

    ANSWER: C++, COM,C#

    3. Developer experience during development

    a. Do you use the COM interop technologies from Visual Studio or from command prompt? (Ex: run tlbimp from command or import through Visual Studio)

    ANSWER: VS

    b. What parts of the COM interop technology from the list above does development use to solve the above scenario?

    ANSWER: TLBIMP,RCW

    c. What is the heaviest of the above that makes the meat of the product? (Ex: Marshalling, Eventing etc)

    ANSWER: MARSHALLING

    d. What are the most complicated problems that you have encountered during development and in which areas?

    ANSWER: WORKING WITH RUNTIME CALLABLE WRAPPERS.

    e. How do you investigate problems in this area? (Ex: Use MDA, Debug using windbg etc)

    ANSWER: WINDBG

    4. Servicing experience

    a. Have you encountered significant end user problems because of failures in your COM interop later? If yes, which part of COM interop did they come from? (List top 4 areas)

    ANSWER: NOT MUCH

    b. What were the problems in most of the cases? Was it with the platform or your implementation?

    ANSWER: RCW. PLATFORM.

    c. How did you identify the problem?

    ANSWER: MS SUPPORT

    d. How did you solve the problem?

    ANSWER: MS SUPPORT

    5. Debugging experience

    a. How do you debug problems in COM interop?

    ANSWER: A

    b. Do you use windbg, SOS, public symbols etc?

    ANSWER: WINDBG

    c. Do you use MDA’s for your debugging? (http://msdn2.microsoft.com/en-us/library/d21c150d.aspx)

    ANSWER: NO

    d. Do you look at you native and managed code in oleview and ildasm, reflector to see what is happening under the hood?

    ANSWER: YES

    e. What is your average time spent in a problem in this area? How many problem have you seen in this area in a given product cycle (months)?

    ANSWER: NOT MUCH. NOT MUCH LATELY.

    6. Help and documentation available

    a. What specific material do you refer for this area and how useful is it?

    ANSWER: MSDN

    b. Do you know about pInvoke.net? Do you use it?

    ANSWER: YES. NO.

    c. How would you rate MSDN documentation for this area? What is lacking if any?

    ANSWER: -1. CONCISE DOCUMENTATION AT ONE PLACE.

    d. Do you find enough samples available to get you on the ramp with the various technologies here?

    ANSWER: BIG NO

    e. How do you rate our help and documentation on MDA’s?

    ANSWER: OK

  14. Alois Kraus says:

    Hi,

    my biggest wish is that we get with tlbimp assemblies where every function exists in two forms:

    COM Function:

    HRESULT Action([retval] BSTR result);

    Generated Interop Assembly Functions:

    string Action();

    bool TryAction(out result);

    This way we could use API’s in a much easier way (e.g. Active Directory) when I want to test for the existence of an object I can simply use the TryMethod. It does greatly affect debugability when you can choose if you want to use the throwing or non throwing version of funciton. One Interop call which fails often (but it is expected to do so) during application startup can give you hard time to find the real error.

    1. Both

    2. Yes

    3. No (I am not going that far)

    4. We have very skilled developers and COM is a vital part of the application architecture will will be around for a long time.

    5. Integration with existing systems is coming up and it will be quite challenging.

    # Customer Scenario and choice of technology

      1. 3D Imaging

      2. DirecShow Pipeline was not available as managed component.

      5. tlbimp, RCW, CCW, Eventing, pInvoke, Marshalling

      6. 32 and 64 bit and languages C++, COM, C#

    # Developer experience during development

      1. Both we have a configuration management team where we need to create makefiles for each VS project anyway.

      2. What parts of the COM interop technology from the list above does development use to solve the above scenario?

      3. –

      4. Nesting level of COM interfaces exposed by managed code which are used by unmanged code which itself expose parts as COM component.

      5. MDA, WindDbg if nothing helps but normally debugger and Tracing is enough.

    # Servicing experience

      1. No 64 Bit Type Libraries until today.

    Yours,

    Alois Kraus

  15. Thottam Sriram says:

    Bob:

      I understand your scenario as follows. You defined a COM interface in unmanaged code and want to use that in managed code. In that process you redeclared that interface and tried casting the object that you got from unmanaged code to the managed interface that you had declared locally and this failed. Is this correct?

      Wouldn’t tlbimp solve the probem for you that it will expose the COm interface in the managed code as well?

  16. Thottam Sriram says:

    Jude:

       Microsoft Office has interop assemblies that it ships. Microsoft.Office.Interop.Word is available already for you to use. Please let me know if this helps.

  17. Thottam Sriram says:

    B. Wa:

       Thanks for your feedback. We cache our RCW as part of our design and implementation for various reasons. You could have a single RCW for multiple objects and those could have references from others as well. It will be hard for us to ensure that the COM component is completely released, but we can ensure that we release our reference. We are thinking into making the RCW experience better and so this feedback from yours actually is in line with what we are thinking. Thanks for the feedback.

  18. Thottam Sriram says:

    pm:

       I agree to your point above. I ran into the same problems when I attempted to solve the CreateProcess in my other blog. I don’t think wrapping all the Win32 API is a feasible, scalable and maintainable solution. A wizard will be a good idea which we have in our mind as well.

    Thanks for your feedback.

  19. Thottam Sriram says:

    Vivek:

       Was your problems with RCW around anything in specific, say life time management for example.

       Can you provide information on the marshalling problems you faced?

       Thanks for your valuable feedback.

  20. Thottam Sriram says:

    Alois:

       I have seen similar requests when it comes to arrays and size which are implied in COM and that they would like to have that exposed in the managed signature. Your ask is different but shares the similar concept.

       Can you eloborate on your problem with nested interfaces? I am not able to understand it clearly.

       Thanks for your feedback

  21. Alois Kraus says:

    Hi Thottam,

    I was thinking of way to import COM interfaces in a way where I can decide if I want to use the imported functions with default error handling (Marshaller throws an ComException if it does encounter an unkown HResult) or and here comes the extension an extended interface which does allow me to the the original HRESULT without any exception at all. This can be beneficial if you have to call some CoM functions quite often in a try-manner. The rationale behind my idea is very much the same that did lead to the int.TryParse method. A possible way to achieve this would be to create from the base interface a second interface where every function returns the original HRESULT code and passes the return value as out parameter.

    namespace ComImportLibrary

    {

       // [Flags(TYPEFLAGS.SpecialNonThrowing]

       // instruct the Marshaller to treat type casts to this interface in a special way

       public interface ITryBaseCalc

       {

           HResult TryAdd(int a, int b, out int result);

       }

       [ComImport, Guid("845FB958-4279-11D2-BF23-00805FBE84A6")]

       public interface IBaseCalc : ITryBaseCalc

       {

           // interface functions

           int Add(int a, int b);

       }

       // [Flags(TYPEFLAGS.SpecialNonThrowing]

       public interface ITryExtendedCalc

       {

           HResult TryDiv(int a, int b, out int result);

       }

       [ComImport, Guid("845FB958-4279-11D2-BF23-00805FBE84A6")]

       public interface IExtendedCalc : IBaseCalc, ITryExtendedCalc

       {

           int Div(int a, int b);

       }

    }

    Yours,

    Alois Kraus

  22. Ruslan says:

    1.

    a. Component developer in C#, Delphi

    b. Yes

    c. No

    d. Transitional (as any computer technology ultimately is), but quite for a long time. The other developers in my team do not write COM-related code – I do that; but there’re a few such developers in other teams in our organization.

    e. The CLR COM features are very useful; and we use them because we make applications for Excel, so we also integrate with VBA. But I can’t say the CLR support of COM is complete, – for example, there’re marshalling problems between VBA and .Net.

    2.

    a. Excel front-end to Java applications; .Net is used for business logic and as auxiliary components for VBA.

    b. The only way to expose our components to VBA is COM. Add-in extensibility interface is COM. Also, EnterpriseServices use COM/DCOM, which is useful to access remote components.

    c. COM is natural choice for extensibility add-ins and for implementation of components for VBA.

    d. The decision wasn’t difficult.

    e. CCW, IDL (have to edit it manually and compile with midl, because the native .Net tools do not export my interfaces quite right for VBA), tlbexp. The other scenario is OLE Automation and RCW/PIA.

    f. 32 bit, C# on client and server side, Java / web services at server side, Managed C++ to integrate with C++ libs.

    3.

    a. Command prompt mainly

    b. I use midl only to compile IDL; also regasm to create initial tlb.

    c. Don’t quite understand the question; the most useful and heavily used is the automatic marshalling done by CLR, RCW and CCW. Also, Managed C++ wrapper was both not trivial to make and very helpful after it was done.

    d. I’d say our usage scenario is not too complicated; but the worst problem is incopatibility of CCW and VBA. To workaround, I use custom IDL and declaring method parameters in my COM classes / interfaces in a special way (which [the way] is not always optimal; I mean, it’s driven by CCW / marshalling specifics rather than design).

    e. The main method is trying. Don’t use MDA. VS debugger works in most cases.

    4.

    a. No, all works fine

    b. n/a

    c. n/a. I use careful automated testing, so I don’t have components misworking in production.

    d. n/a

    5.

    a. VS debugger, cordbg and some other tools. They don’t help in some cases, but I used to manage with them.

    b. No

    c. no

    d. I don’t; only out of curiosity a couple of times.

    e. At the beginning of the project I spent in total about 3 months fixing problems related to interop. Now I reuse the solutions I don’t have any problems.

    6.

    a. MSDN library and Google; they are not always contain solutions, but often direct you to the correct way.

    b. Sometimes. Pinvoke just works, and I know I can use it when I need to, that’s it.

    c. Yes, marshalling is not fully described. E.g. how to do optional byref parameters? indexers? variant arrays?

    d. No, but I’ve written enough my own samples 🙂

    e. Don’t use MDA

    7. Thanks for your work; I can give more specific / extended feedback if needed. My email is "interop" "at" "svoi.biz"

  23. Thottam Sriram says:

    Alois Kraus:

       Thanks for the suggestion. It sounds like a good idea.

  24. Alois Kraus says:

    Hi Thottam,

    what does happen next? Is this a feature that will seriously considered to be implemented in the next version or is it on the long list of minor improvements which will very likely be skipped for the next release?

    Yours,

     Alois Kraus

  25. Thottam Sriram says:

    We are currently planning on the features for the next release. There are a few essential work that we have to do. Apart from that I personally feel that it will be very useful to make the existing experience with this feature better. From the comments here and the thougts that our team has had from working in this area, I will be presenting a plan to our management and take it from there.

    I am optimistic at this point. I am sorry that I am unable to give details on releases now.