Vexing exceptions


Writing good error handling code is hard in any language, whether you have exception handling or not. When I’m thinking about what exception handling I need to implement in a given program, I first classify every exception I might catch into one of four buckets which I label fatal, boneheaded, vexing and exogenous.

Fatal exceptions are not your fault, you cannot prevent them, and you cannot sensibly clean up from them. They almost always happen because the process is deeply diseased and is about to be put out of its misery. Out of memory, thread aborted, and so on. There is absolutely no point in catching these because nothing your puny user code can do will fix the problem. Just let your “finally” blocks run and hope for the best. (Or, if you’re really worried, fail fast and do not let the finally blocks run; at this point, they might just make things worse. But that’s a topic for another day.)

Boneheaded exceptions are your own darn fault, you could have prevented them and therefore they are bugs in your code. You should not catch them; doing so is hiding a bug in your code. Rather, you should write your code so that the exception cannot possibly happen in the first place, and therefore does not need to be caught. That argument is null, that typecast is bad, that index is out of range, you’re trying to divide by zero – these are all problems that you could have prevented very easily in the first place, so prevent the mess in the first place rather than trying to clean it up.

Vexing exceptions are the result of unfortunate design decisions. Vexing exceptions are thrown in a completely non-exceptional circumstance, and therefore must be caught and handled all the time.

The classic example of a vexing exception is Int32.Parse, which throws if you give it a string that cannot be parsed as an integer. But the 99% use case for this method is transforming strings input by the user, which could be any old thing, and therefore it is in no way exceptional for the parse to fail. Worse, there is no way for the caller to determine ahead of time whether their argument is bad without implementing the entire method themselves, in which case they wouldn’t need to be calling it in the first place.

This unfortunate design decision was so vexing that of course the frameworks team implemented TryParse shortly thereafter which does the right thing.

You have to catch vexing exceptions, but doing so is vexing.

Try to never write a library yourself that throws a vexing exception.

And finally, exogenous exceptions appear to be somewhat like vexing exceptions except that they are not the result of unfortunate design choices. Rather, they are the result of untidy external realities impinging upon your beautiful, crisp program logic. Consider this pseudo-C# code, for example:

try
{
  using ( File f = OpenFile(filename, ForReading) )
  {
    // Blah blah blah
  }
}
catch (FileNotFoundException)
{
  // Handle filename not found
}

Can you eliminate the try-catch? 

if (!FileExists(filename))
  // Handle filename not found
else
  using ( File f = …

This isn’t the same program. There is now a “race condition”. Some other process could have deleted, locked, moved or changed the permissions of the file between the FileExists and the OpenFile.

Can we be more sophisticated? What if we lock the file? That doesn’t help. The media might have been removed from the drive, the network might have gone down…

You’ve got to catch an exogenous exception because it always could happen no matter how hard you try to avoid it; it’s an exogenous condition outside of your control.

So, to sum up:

• Don’t catch fatal exceptions; nothing you can do about them anyway, and trying to generally makes it worse.
• Fix your code so that it never triggers a boneheaded exception – an “index out of range” exception should never happen in production code.
• Avoid vexing exceptions whenever possible by calling the “Try” versions of those vexing methods that throw in non-exceptional circumstances. If you cannot avoid calling a vexing method, catch its vexing exceptions.
• Always handle exceptions that indicate unexpected exogenous conditions; generally it is not worthwhile or practical to anticipate every possible failure. Just try the operation and be prepared to handle the exception.

Comments (40)

  1. Dean Harding says:

    About that TryParse thing, something that’s bothered me is that while there’s Int32.TryParse (etc), there’s no Enum.TryParse. Is there any reason for that? Enum.Parse is something I do on occasion, and it’s, ehem, vexing that I have to do it in a try/catch…

  2. c says:

    The problem with the exogenous exceptions is that it’s very hard to figure out the full list of ones you need to catch.  For example, try coming up with the full list of exceptions that could possibly be thrown when you open a file.  In addition to all the usual "file not found, directory not found, access denied, etc" messages you have to handle weirder cases, like the file being on a UNC share, the file being on an overlong junction, the file being pulled from a shadow copy, the file being served from a local directory that is actually a redirected folder, the file existing on a removable drive, the file living on a fake drive that is really a redirect to a URL, etc.

    Coming up with that full list of exceptions is really hard.  But if you miss one, your app crashes.  So, of course, you catch (Exception) and handle it with a generic "that didn’t work" message.  But now you’re committing the "sin" of "catch (Exception)".

  3. KooKiz says:

    The "do not catch fatal exceptions" need to be clarified a bit. I agree that you probably cannot do anything to recover from the error, but you can at least do some logging stuff. I guess you consider in this article that "catching an exception" and "Logging some stuff in a catch block" are two differents concepts.

  4. Inigo says:

    Interesting classification. From a Java perspective, I’d say:

    Fatal exceptions are Errors (i.e. unchecked, and typically not caught)

    Boneheaded exceptions are typically RuntimeExceptions (i.e. unchecked)

    Exogenous exceptions are the reason for checked exceptions

    Vexing exceptions are the reason people complain about checked exceptions

    The relative frequencies of Exogenous vs Vexing exceptions in the language’s libraries could determine whether checked exceptions are a good idea for that language or not.

  5. Doug says:

    This is an excellent top-level taxonomy for exceptions. My biggest general complaint about the .NET framework is that the framework’s exception hierarchy is completely useless. The hierarchy really should have been broken into something like what you describe (though Vexing and Exogenous can’t truly be split via the type system). Instead of ApplicationException and SystemException, we should have had something like System.FatalException or System.DangerousException (parent class of StackOverflow, ExecutionEngine, AccessViolation, etc., the ones that should basically never be caught), System.BoneheadException or System.CodeErrorException (parent class of Argument, NullReference, or IndexOutOfBounds, etc., the ones that should immediately trigger a bug report), and System.RuntimeException (parent class of most other exceptions, the ones that might indicate a runtime failure but might also indicate a bug or a design flaw).

  6. Matt says:

    XmlSerialization => FormatExceptions left right and center.

    Defininitely in the Vexing section if you’re trying to debug a specific formatting problem in it since lots of perfectly valid stuff is dealt with by catching the FormatExceptions.

  7. [ICR] says:

    @C

    While it doesn’t catch all of them, you can catch IOException to get a lot of them.

    I’ve always thought that while having the explicitness of Java’s checked and unchecked exceptions (having to explicitly say what it throws) is bad, it would have been nice to have a distinction between fatal and non-fatal exceptions (a different base class). That way you can catch non-fatal exceptions for things like logging while letting fatal exceptions through.

  8. Matt says:

    Sorry Doug I disagree. they type hierachy is a very blunt tool which shouldn’t be used for this purpose.

    If you are catching an exception  you either want ***really*** specific. So your response can be well defined (FileNotFound, Autorization, specific IO exceptions mainly[1]).

    If you’re catching anything else it is solely note the occurrence (optionally increasing the amount of information available) and rethrow or handle a domain transition (between app domains or to create your own specific exit code for a process) as such Catching Exception is fine

    Anything else is almost certainly a BadIdea

    [1] something the could definitely be improved, If I open a stream I’d like to know the difference between a file being locked verses a file being read only verses not being there etc…

  9. Ollie Riches says:

    ‘Writing good error handling code is hard in any language’

    Absolutely! thats why I think more effort should have been put into the exception strategy for .Net.

    Yes the idea of ‘checked exceptions’ in java seems a good idea in theory but in practice it can get out of hand and I can understand why this wasn’t implemented in .Net. I can also see there were ideas that all user (dev = me) exceptions should have been derived from system.application and therefore easy to handle but that idea broke down very quickly. I don’t understand why the exceptions aren’t groupedsegregated along the lines of namespaces – it would be nice to be able to catch just one exception type for all user related file exceptions – invalid path, invalid filename, permissions etc.

    I would love the ability to do this in a ‘contract first approach’ – be able to define on an interface the exception types that can be generated by the implementation, then at compile time this constraint could be checked and verified.

    Anyway great post cos I’m in the middle of trying to sort kinda thing out :)

  10. MiB says:

    2 Dean Harding

    You can use Enum.IsDefined for checking and then safely call Parse.

  11. Randolpho says:

    Eric wrote this:

    "Just let your "finally" blocks run and hope for the best."

    Keep in mind that there are times when a finally block will *not* run, and they’re almost always during Fatal exceptions.

    What can we do about it? Don’t put "hast to be run" code in your finally block — only use finally for resource de-allocation.

  12. murman says:

    In the try/catch case, the catch covers all the processing that occurs within the using block. To avoid this you have to manually code a try/catch Dispose, as using cannot be attached to an already assigned variable. Both of these options smell bad. Is there a trick that can get you the best of both – limited scope on the FileNotFoundException, and the syntactic support of the using block?

  13. EricLippert says:

    > as using cannot be attached to an already assigned variable

    Why do you say that?

  14. murman says:

    Perhaps I’m mistaken? I just tested this in VS2008 and VS2005 and it works, so my memory is foggy. More likely I hit the ‘Use of unassigned variable’ error case if you don’t have the throw or return in the sequence below.

    The sequence of Declare mydisposable / try { assign mydisposable } catch { throw or return } / using (mydisposable) { } is a little clunky, but not half as bad as without being able to use using that way. Thanks!

  15. Tony Cox says:

    Is there another category of exceptions, which perhaps I would term "internal exceptions"? (Perhaps this is really just a subcategory of exogenous exceptions?)

    What I mean by "internal exceptions" is exceptions you throw and catch internally to a single component because doing so makes your program structurally more readable and maintainable, but that you do not (necessarily) intend to let fly out of your component to an external consumer.

    For example, suppose you have a component which parses some sort of fairly complex data file. Structurally, let’s say it’s convenient to write your parser in a recursive fashion, burrowing top-down through the file as it encounters elements to parse. However, errors of various kinds could be detected at any one of many levels of this parsing process, including all the way down at a leaf node. Let’s say it’s okay to stop trying to parse the rest of the file when you encounter the first error. How do you report these errors back up to the caller?

    You could use the traditional error code reporting method, bubbling back up a return code at each level of parsing, but that adds a lot of boilerplate error passing code to every level. It’s also not terribly easy to elegantly attach additional information to the error. So instead, I think a perfectly reasonable pattern is to throw exceptions where errors occur, and then catch them all at the top-level method of the parser. You can then decide how to report this to the caller (maybe letting them handle the exception instead is okay, or maybe you want to use another method).

    It’s an exogenous exception in that the root cause is something not under your control (the input file, which came from elsewhere), but you threw the exception yourself because it was a convenient way to structure your code.

  16. Bryan Watts says:

    "…but you threw the exception yourself because it was a convenient way to structure your code."

    Expection handling!

  17. Ramblings says:

    It’s pretty rare that I do straight-up trackbacks into other people’s blogs, but the latest post by Eric Lippert really deserves it. He discusses the four different classes of exceptions very eloquently. While he’s talking about .NET, the same truths..

  18. You’ve been kicked (a good thing) – Trackback from DotNetKicks.com

  19. swythan says:

    I attended a session by Jason Clark, who claimed to have worked on defining the exception strategy for the BCL (in v1.0?). He said the rule was:

    * The method name MUST be (contain?) a verb describing the effect of the function.

    * If that effect does not completely occur, the method MUST throw an exception.

    Thus double.Parse(string) throws an exception if it does not parse the string.

    Also:

    * If there are common causes of failure, ways SHOULD be provided to check if it will fail before calling the function (as far as possible).

    E.g. : Enum.IsDefined, File.Exists

    This isn’t always provided, and can’t always be bullet-proof (e.g. File.Exists, as described earlier) but helps avoid many common exceptions.

    IF there is a very common failure scenario where using exception handling causes a (measurable) performance degradation, then there’s another set of rules:

    * Add another function that handles that error condition and JUST that error condition by returning e.g. false.

    * The new function MUST have a name that reflects the fact that it might not succeed.

    Thus: bool double.TryParse(string) which will, of course, throw exceptions if it hits any failure other than a badly formatted string (e.g. OutOfMemoryException).

    He also said that the reason that CLR exceptions are slow is that they are built on top of Win32 SEH. The CLR team apparently told him there was no reason they HAD to be built on top of SEH, it was just easier. They apparently said that if the speed of exceptions ever got into (e.g) the top 10 performance issues seen in real applications, they would probably re-implement them to be faster. The implication, of course, is that they’re along way outside the top 10 right now! :-)

  20. hmm, interesting idea – how would we implement this?

    public class LippertException: Exception, ISerializable;

    public class FatalLippertException: LippertException;

    public class BoneheadedLippertException: LippertException;

    public class VexingLippertException: LippertException; /* or fom bonehead? */

    public class ExogenousLippertException: VexingLippertException;

    maybe? ;-)

  21. Michael Hohlios says:

    So this article sparked a bit of a controversy where I work.

    We all agree with your logic of Vexing Exceptions but we do have one area that none of us can agree on.

    When a user enters in the username and password does the login Manager throw and exception or does it offer a TryLogin?

    Some of us believe it should be an Exception because you expect the user to be logged in and it is exceptional if the users credentials are incorrect.

    While the rest of us think that you should always expect that the users input could be wrong making it not exceptional if they put in the wrong credentials.

    So really the argument comes down to which persons point of view for an exceptional case is programmed for. Is it the user or is it the developer? Users expect to be logged in thus the Exception handling on login, however, the Developer expects the user to be incorrect thus no exception.

  22. Weeble says:

    I don’t see what’s vexing with double.Parse other than that exception handling is so slow. Part of the value of exceptions is that you can’t accidentally forget to check the return code and muddle on thinking that – in this case – the string was correctly parsed. So *is* this just about performance, or are there deeper reasons to dislike exceptions? What exactly is "vexing" about a method throwing an exception when it is unable to do what you asked of it? Many people seem to agree that "exceptions should only be thrown in exceptional circumstances", but everyone seems to understand something slightly different by "exceptional circumstances".

  23. Nick Meyer says:

    @Michael

    Interesting question: to add my $.02, I don’t think the user’s should care how this is implemented.  All they need to know is that their credentials were incorrect, not how the login code treated the failure.  Moreover, it’s not necessarily exceptional to have the wrong credentials, particularly if you’ve changed your password recently, like I have ;)

    I wouldn’t say that the developer "expects" the user to be wrong, either.  I think the developer doesn’t know what to expect.  Either case is as likely, so I would probably indicate the failure by a return value.

  24. Mormegil says:

    I don’t like the concept of exogenous exceptions too much. There is nothing magical in exceptions in this situation – the OpenFile method (note that you did not use a constructor!) might as well return null (etc.) in case of an error instead of throwing an exception. The fact that you need to have atomic check/open has nothing to do with exceptions. So you could have categorized this as a vexing exception, too. The difference is just a bit of taste and syntactic sugar.

  25. chris says:

    These are taught at university Java courses, in the format Inigo pointed out.

  26. Jarrod Ramsey says:

    Good thoughts.  I liked your definition of exceptions and your categories.  Your pointing out the race condition was superb.

    One thing I disagree with is not trying to catch them.  I try to catch everything so I may log the errors.   Otherwise, the error may thrown and the program die, but the admin won’t know what caused it.  Could bad logic have allowed the program to terminate willingly, even though it wasn’t ready to end or did someone send a kill signal to it?

    And even though they can be a pain, .NET stack traces have the ability to provide some critcal information, so if you do nothing more than catch and log a general exception, you have set your self up to proactively understand what caused your program to cease.

    Secondly, you stated to not catch bonehead exceptions.  In my opinion, you never want the exception to be thrown up the stack until it displays the trace to the screen or spit out an unhandled exception error for the user to see.  Log it, email it, anything, but do something with it, then rethrow it or return a "polished" error message.

  27. David Nelson says:

    @Jarrod

    I think you took Eric too literally. When he says "don’t catch" these exceptions, he means don’t catch them at the point where they occur in an attempt to soldier on despite the problem. Of course it is good practice to use a global error handler to log any "unhandled" exceptions that reach the start of the call stack, and display an appropriate error message to the user.

  28. EricLippert says:

    Sure. Use mechanisms (such as Watson) to backstop exceptions and report them to the user, or, even better, to the development team.

  29. madduck says:

    The problem is trickier than it seems. If the blah blah blah code itself throws a FileNotFoundException, do you still want to catch it? Probably not, since it broadens the catch beyond its intent. You’re quite prepared for the file at hand to be missing or unreadable, but do you really want the same catch block to trigger when a configuration file is missing deep inside some library code? More likely than not, you want the latter exception to go straight to the debugger. Delegates or lambdas provide an interesting generalised solution for this:

    public static bool Exogenous<T, E>(Func<T> get, Action<T> use) where E : Exception

    {

       T value;

       try {

           value = get();

       } catch (E) {

           return false;

       }

       use(value);

       return true;

    }

    Usage is quite simple, though some may need time to acclimatise:

    if (!Exogenous<File, FileNotFoundException>(

            () => OpenFile(filename, ForReading),

            file => {

                // blah blah blah

            }))

    {

       // Handle filename not found

    }

  30. Nick says:

    Interesting article.  One of the first things I do when I’m given unfamiliar code is search for "catch" to see what kind of developer(s) I’m dealing with.

    Nothing bothers me more than two patterns (which I see far too often):

    catch(Exception x)

    {

        throw Exception("Failed to achieve happieness");

    }

    and

    catch(Excetpion)

    {

       // no code at all here

    }

    return false;

    So while you’re on the subject of catching things.   Don’t obliterate important debugging information in your catch block.  Don’t leave *any* catch block completely empty.  If there’s nothing to do then use

    catch(Exception x)

    {

       // let the sap maintaining your code see what you’re hiding.

       if(Debugger.IsAttached)

           Debug.WriteLine(x);

    }

  31. Marc says:

    I wish there were a way of checking a string is a valud guid, without calling new Guid(s) and catching the exception. I check it’s length is 36, not exactly water proof.

  32. sam says:

    Use a regex to match the string, no exception catching required.

  33. In this carnival there&#39;re a lot of software design/patterns and frameworks a bit of SOA, UML, DSL

  34. Michael says:

    I really enjoyed this post and the attendant comments.  I you ever felt inclined to address the related topic of when to throw exceptions, I would be a quite happy reader as you address the topic so well.

    ~Cheers~

  35. temp.stuff says:

    I don’t see the big difference between vexing and exogenous exceptions in the examples you made. Just like I could say that TryParse() can be implemented like this:

    TryParse() {

       try {

           this.Parse();

       } catch (…) {

           return false;

      }

    }

    I can say that a "TryOpen()" method can be implemented as:

    TryOpen() {

       try { this.Open(); }

       catch(…) (return false;}

    }

    and with this new method there’s no race when doing:

    if (!FileExists(filename))

     // Handle filename not found

    else

     using ( File f = TryOpenFile(filename, ForReading) )

     {

       // Blah blah blah

     }

  36. Tergiver says:

    @Nick: That's good advice, but I would remove the Debugger.IsAttached check because both Debug.WriteX and Trace.WriteX use the Win32 API method OutputDebugString which can be observed by an external trace viewer such as DebugView (not just a debugger).

  37. Zbigniew Kosior says:

    In case of boneheaded exception, would you still advise to check for null argument in your private method and throw exception accordingly, or only ensure that null will never be passed and let whatever underlining exception there is to point  you the design flow during development?

  38. David says:

    Just want to point out that for exogenous exceptions like your file IO example, it makes sense to do both:  check if file exists AND then use the try-catch block for the race condition.  This is because there are times when a code maintainer needs to turn on debugger breaking upon first-chance exceptions; maybe they're even after the exact type of exception that would now cause a bazillion first-chance-exception breaks instead of just breaking at the one they're after.

  39. Nick says:

    I think the biggest issue I have with exceptions is the impracticality of catching them all. Pokémon might think otherwise, but often it's a huge time waster to catch every exception and try to handle them differently.

    I write small programs and tools within Office / Outlook based company and often I've reused this extension method.

    public static void GenerateEmail(this Exception exc)

    {

       // format and display OutlookItem email with Message, StackTrace, etc

    }

    Then I might use it as follows…

    try {

       // some code which may fail

    }

    catch (ExceptionICanHandle exc)

    {

    }

    catch(Exception exc)

    {

       // Pokemon: catch them all!

       exc.GenerateEmail();

    }

  40. Jill says:

    Where can I get a single list of ALL the extra "Try" (TryParse) functions available in vb.net?