Merlin: Better Specifications for CAT.NET

Guest post by Ben Livshits of Microsoft Research here....

In the last several years we have seen a proliferation of static (and sometimes runtime) analysis tools for finding web application vulnerabilities. Companies such as Fortify, Ouncelabs, Klockwork, and others have been selling tools for finding security flaws for a while now. Most focus of the OWASP top 10, a list being dominated by XSS, SQL injection, CSRF, etc.

CAT.NET is a state-of-the-art static analysis tool for .NET that we have built at Microsoft and released for free to all of our customers.

The effectiveness of these tools is generally seen as a function of false positives and false negatives. Ideally, of course, you want to find all real vulnerabilities and eliminate  all false alarms. In practice, most tools have to cut corners to avoid erring too much in either direction. While much has been said about the approaches to static analysis and improving result quality, an Achilles heel of any analysis tool is the quality of the specification that comes with. Simply, the analysis engine needs to be told what to look for. If the tool doesn’t know what methods are used for sending queries to a database and are therefore crucial to SQL injection finding or if some taint sources are omitted for XSS, some potential vulnerabilities will go missing. The following is but a short list of taint sources in typical .NET APIs:

However, as a recent blog post discusses, this list is woefully incomplete. Even in the case of taint-style vulnerabilities, coming up with a complete specification is really hard for several reasons. First, it often requires pouring over tons of potentially relevant APIs. There are about 30 methods in .NET base class libraries alone that CAT.NET considers to be taint sources. Second, in many cases, the specification is application- or library-specific. For a large-scale app, it’s not uncommon to have a custom encoder or a sanitizer. Unless the tool knows about it, false positives will likely result.

Enter Merlin, an add-on for CAT.NET that aims to produce a better, more complete and accurate specification for finding security bugs. Merlin uses the intuition embedded within the application itself to infer a better specification.

Code Example

Consider a simple example below. Suppose we are trying to classify functions ReadData1, ReadData2, WriteData, and Cleanse as either sources, sanitizers, or sinks. Unless we know something else about these example, this is hard to do.

1: void ProcessRequest() 2: { 3: string s1 = ReadData1("name"); 4: string s2 = ReadData2("encoding"); 5:  6: string s3 = Cleanse(s1); 7:  8: WriteData("Parameter " + s1); 9: WriteData("Header " + s2); 10: }

However, if you are told that ReadData1 and ReadData2 are sources and Cleanse is a sanitizer, then WriteData is likely to be is a sink. Why? Because why else would the developer sanitize tainted input on not one, but two paths. This is most obvious if you look at the propagation graph on the right.

In other words, the program itself contains valuable clues that tell us what the developer thought these functions were with respect to tainting or untaining values. Of course, the developer could still be wrong, there would be no vulnerabilities otherwise, so Merlin generally looks for preponderance of evidence before drawing any conclusions. In this case, Merlin will be able to classify WriteData as a sink with a pretty high probability. The technique that Merlin uses called statistical inference is a generalization of the intuition above.

What does Merlin give us: some results

Before we talk about the gory technical details, which are also described in great detail in our technical report, I’ll give you a glimpse of what our approach can do by citing some experimental results. What is utterly surprising is that, without requiring any initial specification at all, Merlin is able to derive the following specification shown on the right from the little program below.

    1: protected void TextChanged(object sender, EventArgs e) {

    2:     string str = Request.QueryString["name"];

    3:     string str2 = HttpUtility.HtmlEncode(str);

    4:     Response.Write(str2);

    5: }

    6: protected void ButtonClicked(object sender, EventArgs e) {

    7:     string str = Request.UrlReferrer.AbsolutePath;

    8:     string str2 = HttpUtility.UrlEncode(str);

    9:     Response.Redirect(str2);

   10: }

Here is what Merlin infers for this example program:

Sources (1):

string System.Web.HttpUtility+UrlDecoder.Getstring()

Sanitizers (8):
string System.Web.HttpUtility.HtmlEncode(string)

string System.Web.HttpUtility.UrlEncodeSpaces(string)

string System.Web.HttpServerUtility.UrlDecode(string)

string System.Web.HttpUtility.UrlEncode(string, Encoding)

string System.Web.HttpUtility.UrlEncode(string)

string System.Web.HttpServerUtility.UrlEncode(string)

string System.Web.HttpUtility.UrlDecodestringFromstringInternal...

string System.Web.HttpUtility.UrlDecode(string, Encoding)

Sinks (4):

void System.Web.HttpResponse.WriteFile(string)

void System.Web.HttpRequest.set_QuerystringText(string)

void System.IO.TextWriter.Write(string)

void System.Web.HttpResponse.Redirect(string)

This is actually a pretty good start! I highlighted a few of the methods so that you can look up their definition at MSDN. The sanitizers are in fact standard URL and HTML encoders provided by .NET. Note that even some of these are missing from the default .NET specification. You will recognize some other methods like HttpResponse.Redirect as sinks for the reflective XSS vulnerability. Similarly, HttpResponse.WriteFile exposes a command injection vulnerability.

Of course, the better your initial specification, the more Merlin can help. The second part of this blog will talk about the technical aspects of Merlin inference and also show some more results.

Technorati Tags: CAT.NET,static analysis

More reading: Merlin technical report