Clearing up some data on comparisons

I was reading Trend Micro’s blog the other day where they take aim at Microsoft’s claims for its new Internet Explorer 9 browser and how the Smartscreen built-in protection feature was doing at blocking malware.  Trend decided to look at the effectiveness of Smartcreen in Internet Explorer and compare it some other products:

Figure 1 – Rate of malicious URL blocking across WRS

Going by this, Internet Explorer 9 does very poorly at blocking malicious URLs.  However, when Microsoft was touting its effectiveness in its blog, it was not claiming to block malicious URLs, it was blocking malicious files in downloads.  It does this by including reputation in its analysis.  Is the executable file digitally signed?  If not, that makes it more suspicious.

Of course, nobody believes that a signed file grants it trustworthy status anymore than passing an SPF check means that the sender of an email is good.  However, the failure of an SPF check makes it more likely that an email is bad, just like not signing a file makes it more likely that a file is malicious.  That’s what Microsoft is claiming.  It does not make sense to compare a product designed for URL scanning with a browser designed for file inspection.

Next up, Trend compared the threats blocked by itself vs a suite of its competitors, pulling the data from AVtest.org.  Below are the results:

Trend should be congratulated for doing such a good job in this test, but this is not comparing apples to apples.  The Microsoft Forefront product does not use the Smartscreen technology.  It is an on-premise antispam filter that uses five different A/V products.  Trend is not comparing itself to the same thing; it remains to be seen how Smartscreen would perform as a piece of antivirus software.