Successfully debugging VSTO projects

It seems that everyone I know who is writing a blog has a
long list of things they want to talk about, and the list grows faster than
they can write entries. This is fundamentally different from the way newsgroup
postings work, where it's very much a reactionary thing (someone posts a
question and then you post the answer). I've often wanted a "low
priority" flag for news posts (the same way e-mail has low priority flags)
so that I could just write random stuff for people to read at their leisure.
But of course then you'd have to have a corresponding "high priority"
flag, and everyone and their dog would abuse it.

But now I have this blog. Hmmm.

My wish list contains some things that will take a while to
write about, but nothing like what Chris
Brumme
is able to do on a regular basis. Even his e-mail replies to casual
questions are incredibly long and detailed, and we're very lucky to have him at
Microsoft!

Anyway, a small topic for tonight. One thing I'd like to do
is talk a lot about some of the finer points of VSTO (in particular its
security model), but unfortunately (!!!) the
documentation
is actually pretty detailed and so I'm not sure how much
value I can add. Oh well, we'll see how often things come up on the newsgroups
and in internal support e-mail aliases.

Oh well, on to my 3 main tips for the evening:

Swallowed Exceptions

If you have some code in your solution that generates an
exception, and that exception propagates back to the caller (Word or Excel)
then you will not be notified about it. This is in contrast to VBA which will
tell you when your code has messed up, and it often confuses people -- their
code just silently dies.

As Chris points out in his latest treatise
on exceptions
, the CLR tries to interop with the unmanaged world when it
comes to exception propagation, and the way that managed exceptions are
returned to the COM world is via failed HRESULTs and the IErrorInfo
interface
.

For better or worse, when Word or Excel receives a failed
HRESULT from a method call (usually an event handler in this case) they just
ignore it and move on. In the VBA case they have a much tighter integration
with the runtime portion and VBA will do its thing when it realises it's about
to lose control of execution, and so you get the error message (which of course
is very annoying if you're an end-user because you have no idea what a Type
Mismatch is, and can't do anything about it).

So in VSTO, if you have any code that is called by Word or
Excel, you should wrap it in try-catch blocks so that you can display an error
or do other processing as appropriate before the error is forever lost in the
transition to unmanaged code. Another trick is to turn
on the "Break on 1st chance exceptions" setting
, although this
will get you more exceptions than you need. Fine-tuning the exceptions that
will break into the debugger by expanding the TreeView and unselecting some of
the exception types you are uninterested in can help.

Failing to execute

A common problem people have is that their assembly does not
execute at all. They got the original solution working in Visual Studio (where
we handle the basic security policy changes for you), but when they move it to
another machine or a different directory it fails to load. The most likely
reason for this is that policy was not updated correctly, and the thing I
always ask people to do is run the following commands and send me the results:

caspol -all -lg

caspol -rsg path_to_assembly

What this will tell me is what their security policy looks
like (lg == list groups == list all policy rules), and then how the CLR thinks
the evidence of the assembly maps to those rules (rsg == resolve groups ==
list groups the assembly matches). This tells me whether or not they have set
up policy correctly, and whether or not their assembly is matching the code
groups they think it should match. Most problems are caught here for one of
three reasons:

  1. A
    network rule (eg, https://server/ -->
    FullTrust) was added to the MyComputer zone, but it's in the LocalIntranet
  2. There's
    a typo in the filename or URL
  3. The
    asterisk (*) was not added after a directory to indicate "and all
    folders under here"

I'll show an example of a fourth problem I've only seen once,
but the effect is the same as for the three cases above:

A simplified output of caspol
-all -lg

Enterprise

  All_Code: FullTrust

Machine

  All_Code: Nothing

    MyComputer: FullTrust

    LocalIntranet: LocalIntranet

      https://localhost/*: FullTrust

    TrustedSites: LocalIntranet

    RestrictedSites: Nothing

User

  All_Code: FullTrust

A simplified output of caspol
-rsg https://localhost/myassembly.dll

Enterprise

  All_Code

Machine

  All_Code

    TrustedSites

User

  All_Code

Immediately it is obvious to the trained eye that the user
thought https://localhost/ was in the
LocalIntranet zone (which it is by default), but for one reason or another they
have added it to the TrustedSites zone in IE. The answer is simply to move the
localhost rule from LocalIntranet to TrustedSites, and you are golden.

Other random security
failures

If your main assembly loads, but you get random errors at
some other time (especially if you are using 3rd party components or some code
that may be automatically generating assemblies via ICodeCompiler or VSA) then
you may be able to use the technique I describe in this
newsgroup post
. Essentially you can temporarily grant unrestricted
permissions to all code on your local machine and then at some opportune time
in your program you dump out the evidence used to load all the
"interesting" assemblies. For example, if you use a component that
dynamically compiles and loads assemblies via ICodeCompiler,
you will probably find some randomly-named assembly loaded into your AppDomain
with a certain set of evidence.

Depending on what control you have over the generated
assembly, you may be able to force it to be generated in a particular location
which you can then trust. It is likely to be tricky to get these kinds of
solutions working though, and your best bet in this instance may be to create a
new AppDomain with its own (probably default) security policy that can load the
dynamically generated code. But then of course you'll have problems marshalling
the stuff between AppDomains.... sigh.

----------

Something else I may talk about in more detail at some stage
is the Excel
/ .NET mismatched locale issue
. This is a problem that came up during beta,
and spawned many person months of discussion and brainstorming to come up with
a workable solution. At the end of the day, the best we could do was document
the problem (ie, the link above) and try and work something out longer-term.
The problem is really quite hard to solve, and maybe I'll delve into it in a
bit more detail later on. (Actually there is a fairly simple solution that will
work in most cases, but it "feels" like a hack and it just wasn't
feasible at this point in time).

Oh and some trivia for the evening:

1) IUnrestrictedPermission
is the thing in the CLR that lets a permission become part of the FullTrust pseudo permission set.

2) There
are no Microsoft-provided implementations of ICodeParser

Notes for #1: Pretty much all permissions except StrongnameIdentityPermission
implement the IUnrestrictedPermission interface. The difference between the Everything permission set and the FullTrust pseudo permission set is that
Everything contains a static list of
all permissions that Microsoft ships out of the box but no 3rd party
permissions. FullTrust on the other
hand is a dynamically computed list that includes all permissions that
implement IUnrestrictedPermission,
including any 3rd party permissions. You should probably never use the Everything permission set unless you
have a really good reason to, and if you ever find yourself in the position of
having to implement your own permission (not something you'd do for an ordinary
application) then you should strongly consider implementing IUnrestrictedPermission if the
semantics of your permission mean that FullTrust
code should get it (and since FullTrust
code can do anything it wants, including lie about evidence, call unmanaged
code, read and write random memory locations, etc. it can do whatever it is
you're trying to prevent anyways, so you shouldn't really think you're buying
any additional security by not implementing IUnrestrictedPermission).

Notes for #2: Every now and then someone
asks a question about ICodeParser
, and how they go about getting their
eager little mitts on an instance for C# or VB (strangely no-one ever seems to
ask about JScript...<sniff>). Anyway, you can't. Obviously someone
designed that interface because they had planned to ship something along the
lines of a source-code-to-CodeDOM translator, but at some point in the long and
arduous journey of shipping a product it got left behind. I even remember
hearing about a feature of VS where you could copy VB .NET source code and
paste it as C# (and vice versa), but this was a year or more before VS 7.0 shipped
and it's still not in the product today. I may have even seen a demo at one
point in time, but I don't know if I'm imagining that or not ;-).

That's it for now!