C# Exception Handling - A case study

The Problem Statement

Recently, I came across an interesting situation related to exception handling in C#. Here's a short program that illustrates the core issue we had in our production code.

What do you think is the output of the code below?

  1 2 3 4 5 6 7 8 910111213141516171819202122232425262728293031323334353637383940414243

 using System;namespace ExceptionDemo{    class Program    {        static void f()        {            try            {                g();            }            catch(Exception e)            {                throw e;            }        }        static void g()        {            try            {                throw new Exception("A");            }            catch(Exception e)            {                throw e;            }        }        static void Main(string[] args)        {            try            {                f();            }            catch(Exception e)            {                Console.WriteLine(e.StackTrace);            }        }    }}

As you can see, Main calls f in a try block, which in turn calls g in a try block. Each try block has an associated catch block. Once the control reaches g, g throws an Exception initialized with the string "A". This exception is caught inside the catch block on Line 27. The catch block in g throws the incoming exception object, which is then caught in the catch block in f, which in turn throws the incoming object once again to be finally caught and dumped in the catch block of the Main function.

Here is the output of the code in Visual Studio 2015 with target .NET version as 4.5.2

   at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 15
   at ExceptionDemo.Program.Main(String[] args) in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 35
Press any key to continue . . .

What do you think about this output? Does this output help you to resolve this issue when reported from the field? Notice, that all one gets is that Line 15 threw an exception when f was called from Main function. We have lost lots of valuable information regarding the exception that is required to effectively troubleshoot the issue. So, what went wrong?

The case of Rethrow

As I looked at our production code, I concluded that the author of the production did not clearly understand the right way to do exception handling. The StackTrace information is lost because of the incorrect way to re-throw the exception. Similar to C++, the right way to re-throw an exception in C# is used to an empty throw statement 'throw;'. Refer to any good book on C# and they clearly call out that perils of using the incoming Exception object as an argument of throw in the catch block. One such quote is like so from the book "C# 6.0 Cookbook"

Rethrowing an exception
You can capture and rethrow an exception as follows:
try { ... }
catch (Exception ex)
{
// Log error
...
throw; // Rethrow same exception
}
If we replaced throw with throw ex, the example would still work, but the StackTrace property of the newly propagated exception would no longer reflect the original error.

So, what is this empty throw statement, you say?

Here's what MSDN says
Usually the throw statement is used with try-catch or try-finally statements. A throw statement can be used in a catch block to re-throw the exception that the catch block caught. In this case, the throw statement does not take an exception operand. For more information and examples, see try-catch and How to: Explicitly Throw Exceptions.

Confidently, I modified the production code on similar lines like so:

  1 2 3 4 5 6 7 8 910111213141516171819202122232425262728293031323334353637383940414243

 using System;namespace ExceptionDemo{    class Program    {        static void f()        {            try            {                g();            }            catch(Exception e)            {                throw;            }        }        static void g()        {            try            {                throw new Exception("A");            }            catch(Exception e)            {                throw;            }        }        static void Main(string[] args)        {            try            {                f();            }            catch(Exception e)            {                Console.WriteLine(e.StackTrace);            }        }    }}

Notice, how the throw operator at Line 27 and Line 15 no longer takes the operand 'e'. So, now, the empty throw statement at Line 27, rethrows the original exception which is e. This exception is now caught in the catch block in f. Once again, f rethrows the exception which is now caught by the catch block in Main. This catch block consumes the exception and prints the value of the StackTrace member of the Exception Object e. What do you think happens now?

I was completely surprised by the output of the program shown below

at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 27
at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 15
at ExceptionDemo.Program.Main(String[] args) in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 35

This call stack clearly indicates that line 27 throws an exception and as part of the stack unwinding process, line 15 rethrows the same exception and finally it is caught in the catch block in Main function which then prints the associated StackTrace of the exception object. This is clearly way more information than our original program. However, I am still missing information about the original line which threw the exception in the first place. The StackTrace still has no trace of line 23 knowing which may be very critical for troubleshooting the issue.

A Scuba Dive into the world of CIL

I decided to look at the IL generated for the two programs.

Here is the IL for function f in the first program which throws the incoming exception e using throw e;

.method private hidebysig static void f() cil managed
{
// Code size 16 (0x10)
.maxstack 1
.locals init ([0] class [mscorlib]System.Exception e)
IL_0000: nop
.try
{
IL_0001: nop
IL_0002: call void ExceptionDemo.Program::g()
IL_0007: nop
IL_0008: nop
IL_0009: leave.s IL_000f
} // end .try
catch [mscorlib]System.Exception
{
IL_000b: stloc.0
IL_000c: nop
IL_000d: ldloc.0
IL_000e: throw
} // end handler
IL_000f: ret
} // end of method Program::f

Here is the IL generated for function f in the second program which leverages rethrow using an empty throw statement throw;
.method private hidebysig static void f() cil managed
{
// Code size 16 (0x10)
.maxstack 1
.locals init ([0] class [mscorlib]System.Exception e)
IL_0000: nop
.try
{
IL_0001: nop
IL_0002: call void ExceptionDemo.Program::g()
IL_0007: nop
IL_0008: nop
IL_0009: leave.s IL_000f
} // end .try
catch [mscorlib]System.Exception
{
IL_000b: stloc.0
IL_000c: nop
IL_000d: rethrow
} // end handler
IL_000f: ret
} // end of method Program::f

Note, how the empty throw generates the CIL instruction 'rethrow' instead of 'throw'.

A Scuba Dive into the world of C# Specification Documents

I then decided to refer to the C# language specifications and I could hold of the specs for .NET 3.5 here. This is what I found about the rethrow specifications

rethrow – rethrow the current except ion
Format Assembly Format Description
FE 1A rethrow Rethrow the current exception.
Stack Transition:
…,

…,
Description:
The rethrow instruction is only permitted within the body of a catch handler (see Partition I). It
throws the same exception that was caught by this handler. A rethrow does not change the stack
trace in the object.
Exceptions:
The original exception is thrown.
Correctness:
Correct CIL uses this instruction only within the body of a catch handler (not of any exception
handlers embedded within that catch handler). If a rethrow occurs elsewhere, an exception will
be thrown, but precisely which exception, is undefined
Verifiability:
There are no additional verification requirements.

Lost and Suddenly Found

This was once again inline with the impression I had formed while referring to the topic of Exception in several books on C#.  So, what is happening here? Is the C# compiler violating the C# specifications? Is this a bug? Is it in the latest version of the compiler? Is it something to do with .NET version 4.5.2?

This quest led me to some posts suggesting an approach of wrapping up the exception object and throwing a new exception object from within the catch block. Here's how my program looks after implementing this suggestion

  1 2 3 4 5 6 7 8 910111213141516171819202122232425262728293031323334353637383940414243

 using System;namespace ExceptionDemo{    class Program    {        static void f()        {            try            {                g();            }            catch(Exception e)            {                throw new Exception("f", e);            }        }        static void g()        {            try            {                throw new Exception("A");            }            catch(Exception e)            {                throw new Exception("g", e);            }        }        static void Main(string[] args)        {            try            {                f();            }            catch(Exception e)            {                Console.WriteLine(e.StackTrace);            }        }    }}

This gave me the output like so, once again missing out Line 23, which raises the exception in the first place.

   at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 15
   at ExceptionDemo.Program.Main(String[] args) in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 35
Press any key to continue . . .

My Unwinding Starts

But at this point I discovered that the Exception class has a function ToString() which displays more information than StackTrace.

ToString returns a representation of the current exception that is intended to be understood by humans. Where the exception contains culture-sensitive data, the string representation returned by ToString is required to take into account the current system culture. Although there are no exact requirements for the format of the returned string, it should attempt to reflect the value of the object as perceived by the user.
The default implementation of ToString obtains the name of the class that threw the current exception, the message, the result of calling ToString on the inner exception, and the result of calling Environment.StackTrace. If any of these members is null, its value is not included in the returned string.
If there is no error message or if it is an empty string (""), then no error message is returned. The name of the inner exception and the stack trace are returned only if they are not null.

After changing Line 39 to Console.WriteLine(e.ToString()), here's the output I got:

System.Exception: f ---> System.Exception: g ---> System.Exception: A
   at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 23
   --- End of inner exception stack trace ---
   at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 27
   at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 11
   --- End of inner exception stack trace ---
   at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 15
   at ExceptionDemo.Program.Main(String[] args) in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 35
Press any key to continue . . .

Aha, now I am able to get the information about Line 23 as part of the Inner Exception Stack Trace. But, why is throw; not working, continued to bother me.

Nailing the issue

At this stage, I reached out to the compiler and platform team at Microsoft. The folks in the compiler and language team redirected me to the excellent book "CLR via C#" by Jeffrey Richer. Here's the specific text that talks about the issue

The following code throws the same exception object that it caught and causes the CLR to reset its
starting point for the exception:
private void SomeMethod() {
try { ... }
catch (Exception e) {
...
throw e; // CLR thinks this is where exception originated.
// FxCop reports this as an error
}
}
In contrast, if you re-throw an exception object by using the throw keyword by itself, the CLR
doesn’t reset the stack’s starting point. The following code re-throws the same exception object that it
caught, causing the CLR to not reset its starting point for the exception:
private void SomeMethod() {
try { ... }
catch (Exception e) {
...
throw; // This has no effect on where the CLR thinks the exception
// originated. FxCop does NOT report this as an error
}
}
In fact, the only difference between these two code fragments is what the CLR thinks is the original
location where the exception was thrown. Unfortunately, when you throw or re-throw an exception,
Windows does reset the stack’s starting point. So if the exception becomes unhandled, the stack
location that gets reported to Windows Error Reporting is the location of the last throw or re-throw,
even though the CLR knows the stack location where the original exception was thrown. This is
unfortunate because it makes debugging applications that have failed in the field much more difficult.

The last paragraph is very important. It almost talks about the issue I have been facing.

Final Nail in the Coffin

As I continued with my search, I finally could rest my quest to peace, as I hit the exact MSDN documentation that talks exactly about the issue I have been facing.

The common language runtime (CLR) updates the stack trace whenever an exception is thrown in application code (by using thethrowkeyword). If the exception was rethrown in a method that is different than the method where it was originally thrown, the stack trace contains both the location in the method where the exception was originally thrown, and the location in the method where the exception was rethrown. If the exception is thrown, and later rethrown, in the same method, the stack trace only contains the location where the exception was rethrown and does not include the location where the exception was originally thrown.

So, now it was abundantly clear to me, that rethrow; does not always preserve the call stack, which is the impression I had formed while referring to various books and articles on C# exceptions. The special case arises "If the exception is thrown, and later rethrown, in the same method, the stack trace only contains the location where the exception was rethrown and does not include the location where the exception was originally thrown."

So, now that I understood why the program behaves the way it does, what are the options that I have to enable better call stack during an exception. The folks in the compiler and language team pointed me to the new facility available in .NET 4.5 onwards which is called the ExceptionDispatchInfo. Here's the beauty of this new mechanism

The ExceptionDispatchInfo object stores the stack trace information and Watson information that the exception contains at the point where it is captured. The exception can be thrown at another time and possibly on another thread by calling the ExceptionDispatchInfo.Throw method. The exception is thrown as if it had flowed from the point where it was captured to the point where the Throw method is called.

  1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435363738394041424344454647484950

 using System;using System.Runtime.ExceptionServices;namespace ExceptionDemo{    class Program    {        static void f()        {            ExceptionDispatchInfo exceptionDispatchInfo = null;            try            {                g();            }            catch(Exception e)            {                exceptionDispatchInfo = ExceptionDispatchInfo.Capture(e);            }            if (exceptionDispatchInfo != null)                exceptionDispatchInfo.Throw();        }        static void g()        {            ExceptionDispatchInfo exceptionDispatchInfo = null;            try            {                throw new Exception("A");            }            catch(Exception e)            {                exceptionDispatchInfo = ExceptionDispatchInfo.Capture(e);            }            if (exceptionDispatchInfo != null)                exceptionDispatchInfo.Throw();        }        static void Main(string[] args)        {            try            {                f();            }            catch(Exception e)            {                Console.WriteLine(e.ToString());            }        }    }}

Here is the output of the above code, which looks pretty similar to the case when a new Exception was thrown with the incoming exception wrapped up inside.

System.Exception: A
   at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 28
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 35
   at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 13
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 20
   at ExceptionDemo.Program.Main(String[] args) in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 42
Press any key to continue . . .

The above code arrangement highlights the power of ExceptionDispatchInfo, which captures the exception info at the point of capture to later on Throw (not throw) it again even outside of the associated catch block. Note that it is perfectly possible to throw the exception in the catch blocks like so (thanks to Landen for pointing this out).

  1 2 3 4 5 6 7 8 91011121314151617181920212223242526272829303132333435363738394041424344

 using System;using System.Runtime.ExceptionServices;namespace ExceptionDemo{    class Program    {        static void f()        {            try            {                g();            }            catch (Exception e)            {                ExceptionDispatchInfo.Capture(e).Throw();            }        }        static void g()        {            try            {                throw new Exception("A");            }            catch (Exception e)            {                ExceptionDispatchInfo.Capture(e).Throw();            }        }        static void Main(string[] args)        {            try            {                f();            }            catch (Exception e)            {                Console.WriteLine(e.ToString());            }        }    }}

This program too generates the desired output capturing the original line triggering the exception

System.Exception: A
at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 24
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at ExceptionDemo.Program.g() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 28
at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 12
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at ExceptionDemo.Program.f() in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 16
at ExceptionDemo.Program.Main(String[] args) in D:\DemoProjects\ExceptionDemo\Exceptions\Program.cs:line 36
Press any key to continue . . .

What about C# Code Analysis

Resharper and C# code analysis rules clearly flag a warning when an exception is being rethrown using throw ex; instead of a throw;. The warning id is CA2200. Here is how it looks for our sample program

 So, in my opinion there must be a real strong reason for code with CA2200 warning to be allowed for checkin. Ignoring this warning can cause some serious difficulty in troubleshooting and diagnosing field issues due to lost stack trace during exception unwinding.

Conclusion

Here's the conclusion that I arrived at finally. Since our product supports .NET 4.5+, I am of the opinion to resort to the technique of ExceptionDispatchInfo to enable better troubleshooting and diagnostics. I am also of the opinion to flag CA2200 warning as an error instead of a dismissible warning in our project settings.

References
1. CLR Via C#
2. StackOverFlow
3. C# code base
4. Core CLR Git Issue
5. Thorain.NET
6. C# Specs
7. CA2200
8. MSDN