Principles behind IE7’s Phishing Filter


My last post was intended to introduce our overall security strategy and the specific features in IE7 Beta1 for XP SP2 and Windows Vista. A lot of responses to my post were questions about why and how the Microsoft Phishing Filter in IE7 will check websites. We have also have heard from a number of site owners who want to know how they can correct an evaluation of “suspicious” or “confirmed phishing”. Before we continue posting on the rest of the IE7 security features, I want to let you know that we’re listening to your feedback about the Phishing Filter and take this opportunity to clarify the process.

The prime directive of the Phishing Filter feature is to help protect users from phishing websites, while maintaining user privacy and being transparent and flexible about how we do it. Protecting your privacy means we will not collect personally identifiable information, we will explain clearly how the feature works, we will give you the choice to use it only when you want to, we will provide a clear indication of how we will use any data, and we’ll use SSL encryption to help protect any queries you send to the anti-phishing server. These are the principles we used to design the Phishing Filter.

  1. Readers asked why we decided to use real-time look ups against the anti-phishing server as opposed to an intermittent download list of sites in the way that an Anti-spyware product might.  We included real-time checking for phishing sites because it offers better protection than only using static lists and avoids overloading networks. Phishing Filter does have an intermittently downloaded list of “known-safe” sites but we know phishing attacks can strike quickly and move to new addresses, often within a 24-48 hour time period which is faster than we could practically push out updates to a list of “known-phishing” sites. Even if the Phishing Filter downloaded a list of phishing sites 24 times a day, you might not be protected against a confirmed, known phishing site for an hour at a time, at any time of day. Because Phishing Filter checks unknown sites in real-time you always have the latest intelligence. There would also be network scale problems with requiring users to constantly download a local list. We think the number of computers that could be used to launch phishing attacks is much higher than the number of spyware signatures that users deal with today. In a scenario where phishing threats move rapidly, downloading a list of new reported phishing sites every hour could significantly clog internet traffic.
  2. Readers asked about how the data from the Phishing Filter will be used. We want to be very clear about this so we actually updated the privacy statement last week to spell it out in more detail: We use the data to make the Phishing Filter service better and constantly improve the level of accuracy in our results, not to personally identify you.

    The updated privacy statement also explains how and when the Phishing Filter will check sites.
    • No site will be checked on the server unless you choose to enable the feature.
    • Phishing Filter only checks sites that aren’t in IE’s downloaded “known-safe” list
    • Potentially sensitive data, like the URL query string, is stripped out of the URL before it’s sent to the server for checking. Other types of navigation-related information, like http cookies, are not sent to Microsoft.
    • The URL is sent securely over an encrypted SSL connection to help protect your privacy

    You may not find the privacy statement to be a page turner, but it does represent our promise to you. I hope this clarification helps dispel a conspiracy theory or two.

  3. Folks have also raised concerns about how Microsoft will judge sites for the confirmed-phishing-site list. We want you to know that the process to evaluate reported phishing sites will be fair, simple and clear. To be sure it’s fair, the process will allow sites to ask for a reevaluation if the site owner does not agree with the Phishing filter rating. You won’t have to find a support number to call, instead the link to report an incorrect evaluation is built into the UI of IE7. If you dispute an evaluation by the phishing filter, the situation will be addressed as quickly as possible.  If the review process determines that there was a mistake on part of the phishing filter, your site will instantly be restored to good standing once it’s been reevaluated as not-phishing. The phishing filter whitepaper includes more information about best practices to prevent your site from being marked suspicious.

I hope this has helped folks understand the benefits of getting dynamic protection with a real-time service. I encourage you to try it out. Even if you turn real-time protection off, it’s nice to know that you can always manually check on a site if you have reason to suspect foul play.

Tariq’s post is teed up and will go into way more detail about the UX and how Phishing Filter actually works. If you have questions like that you should hold for him. If you have questions or feedback on the privacy concerns, fire away!

Thanks,
Rob Franco

Update: Changed the link of the phishing filter whitepaper to reflect the correct URL (it got changed).

Comments (51)

  1. Anonymous says:

    [If the review process determines that there was a mistake on part of the phishing filter, your site will instantly be restored to good standing once it’s been reevaluated as not-phishing.]

    Also if a filter has made a "mistake" – please keep some database and analysis of WHY it came to those conclusions – and use the info to further "tweak" the filtering ALGOs

    All in All – this technology is long overdue

    🙂

    Also those "phishing" pages – if they appear on MSN Search – should be automatically "banned" from the SERPs ….

    and if the domains are owned by one person – the entire domain should be banned permanently!!

  2. Anonymous says:

    So, it’s gonna be a "Manual Verification" to see if a "reported site" is really a Phishing ?

    So the process has a delay ? There is any kind of SLA ? Because a phishing site has a short TTL and if this manual verification don’t be done faster, it could be later.

  3. Anonymous says:

    Just for the record, you guys are doing a really good job of keeping information flowing.

  4. Anonymous says:

    Just for the record, it’s remarkable what you are not talking about. Time to unsubscribe from this feed.

  5. Anonymous says:

    Rob, you still haven’t cleared our doubts that the thing actually works… Give us URLs or testcases that WE CAN USE to see how the filter works first-hand.

  6. Anonymous says:

    Thanks for keeping us current on what’s going on with IE. You can never convince the hardcore conspiracy theorists but you’re doing a lot to reach out to the fair-minded with these posts.

    I’d like to know more about the "bureau" that rules on phishing sites. Will it be headquartered in Redmond? Staffed by how many? Is it a 24/7 operation? It seems like this is something that should work in conjunction with MSN and the Hotmail spam filter group.

    Ken

    MVP [ASP.NET]

  7. Anonymous says:

    This is all well and good but can you include Firefox in your automatic updates? The thing is barely hanging by it’s fingernails as far as SSL goes. Heck, if you want to spoof it all you have to do is go to it’s own Bugzilla, type in SSL Security and bingo, you can rip off anyone using it.

    I know they are the big underdog and should be able to build marketshare on that alone. So I would like to congradulate you guys on actually gaining marketshare last month. Looks like Firefox was a blip on the radar.

    Now on a more serious point. We do still need to deal with Mozilla and it’s children. I’m not big on Open Source but for the good of the web developer community could you open up a bit of your source code? Specifically dealing with the onmouse* events. Mozilla can’t pick them up if you extend over an IFrame. It would be great if you could give the Open Source developers a point in the right direction.

    Another big issue is that if you dynamically put a table (or just about any other element) into a Div’s innerHTML Mozilla once again craps out and can’t even capture the mouseover or mousemove events. I guess there are no web standards on common sense usability.

    Once again, if you could open up that source code to the Open Source community the consumer would greatly benefit. If you want to have fun check out the Bugzilla on Firefox for SSL. Using that information you can find close to 3 dozen ways to go phishing with Firefox.

    Keep up the good work! Here’s to hoping you gain marketshare once again in August. Ken.

  8. Anonymous says:

    My worries are that the phishing filter is gonna turn out like something at HotOrNot. The reason I say this is because people can rate anything however they want. You’ll have people that will want to report sites whether they think they’re safe or not. I think Microsoft should have some type of thing to protect against this kinda thing, otherwise it’ll slow down the whole review process for addresses.

  9. Xepol says:

    I predict that this is going to totally backfire on MS. Based on MS’s recent classification of MsgPlus’s main EXE as spyware because the INSTALLER might install adware, MS’s behaviour is already suspect. There will ALWAYS be suspicions that MS is going to use this to track how popular competitor websites are, and may even abuse the filter much in the same way that MSantispyrware appears to be being abused in relation to MsgPlus.

    In fact, I would suggest that it is laughable to sugges that MS will not use the information made available by the phishing filter to check up on competitor popularity.

    The sad part is, I’m pro-MS, and I think this way.

    Perhaps the BEST way for MS to avoid this would be to make the whole process more transparent and either place the database in a non-profit third party hand, or let us pick between different vendors for our phishing database and validation.

    Until then, the phishing filter will remain off on ALL the machines overwhich I have control.

  10. Anonymous says:

    How many people have to report a site before someone at microsoft decides to investigate it?

    And, could microsoft effectively block every website if they were feeling extra evil?

  11. Anonymous says:

    Please stop it flashing up for reserved (i.e. internal / local) ip addresses as for end users seeing the phishing warning on their intranet applications will frustrate/ worry them and i cannot see a way around it at present

  12. Anonymous says:

    If I receive an obvious phishing mail, can I report the URL in it without visiting the site? I don’t visit phishing sites because they could use a exploit that hasn’t been fixed therefore it is mandatory that phishing sites from emails can be tagged as such without visiting them.

  13. Anonymous says:

    Does the story about internet congestion bother anyone besides me? Granted information will be more up to date when it is queried in real time, but as far as traffic, how can querying for every distinct visited domain result in LESS traffic than an incremental hourly (or even more frequent) update? An hourly update needn’t take much more bandwidth than just one of the domain queries. Imaging a simple request containing the last update’s "snapshot number" or timestamp or something, and the response would contain a new snapshot number or timestamp and a signed list of changes since the previous stamp. You wouldn’t need to download the whole list every time. An occasional crc or hash check of the whole list could make sure no errors have crept in, and unless a HUGE number of sites get added or removed within a single hour, this wouldn’t result in much traffic.

    That said, the Google Toolbar already does this with PageRank, and timeliness is worth something. And I suppose one typically only visits a small (5 or 10 or 20?) unique domains in a day, so with some amount of caching it could keep the number of queries down. On the other hand caching would reduce the responsiveness of corrections to erroneous phishing status.

  14. Anonymous says:

    Hey, what zzz just wrote is true. There should be a possibility to tag a phishing mail as such without first having to browse to that site.

  15. Anonymous says:

    Ben, it would only query URLs for domains a person hadn’t previously visited. This would only be a small number of requests for the average person, for many people none at all, considering they visit few ‘new’ sites a day. Which equates to sending and recieving only 1kb or so each day in lookups. Compared to the size of the actual page they are visiting, this is fairly negligble.

    Compare this to everyone having to download an entire list of phishing urls everyday, regardless of whether they ever visit those sites or not, they would have to get the whole list – this would equate to a lot more traffic. I guess MS don’t want to give out their database of bad urls either, though it could easily be a one way hash lookup db if theyre worried about that.

  16. Anonymous says:

    Hey Rob, it’s Paul. I’m really glad to see that ieblog is keeping customers up to date on internet explorer features. I think the phishing filter is a great feature, and I’m glad to see it implemented in the internet explorer base installation so there’s no need to download bloatware from a 3rd party site that has spyware bundled :).

    Also, I think it’s really cool to see the reader comments have gone from slashdotesque (anti-ms) to productive and encouraging. It shows that Microsoft is definately going in the right direction with their browser and giving Mozilla a run for their money.

    Kind regards,

    Paul

  17. Anonymous says:

    Inside Microsoft tells of news that MSNs Phishing Filter add-in is available for download for US…

  18. Anonymous says:

    Is there an official procedure to post bugs to Microsoft regarding IE7 Beta 1?

  19. Anonymous says:

    Excellent article. Informative and nicely targetted at real concerns – it’s exactly what this blog should be about.

    Unfortunately, these efforts are struggling against a wider mis-trust of Microsoft which is regularly reinforced in much more public places. An earlier comment raises the example of Windows Update only working with Internet Explorer. OK, this is off-topic but it is a valid point. Why are there no answers coming from Microsoft about these other areas of customer concern? Who could and should be answering them?

    Until some of these issues are addressed, I don’t see Microsoft being able to regain the trust it has lost in a lot of the IT community.

    As for the phishing filter, is the reporting of dodgy URLs partly in the hands of users? If so, that could cause a world of pain. Never underestimate the power of stupid people in large numbers!

    Keep up the good work,

    Chris

  20. Anonymous says:

    From your description of this lookup feature I would assume the following so it works without mutch hassle:

    * every user running IE from everywhere can use it

    * thus the request has to be done over port 80 (or 443 as in the privacy statement SSL is meantioned) or it won’t really work inside companies due firewalls

    * there’s no restriction on to who can you this service (i.e. very IP is allowed)

    It isn’t mentioned in detail in the privacy statements how the SSL encryption is exactly done, but let us assume for a moment that’s not much different than from a standard https nowadays used everywhere.

    This drives me to the question: is there any limitation which client can ask the (microsoft?) server about a url whether it’s used in phishing fraud or not?

    Basically, is microsoft providing a free of charger public SSL encrypted interface to query any client whether a given site is maybe a phishing site?

    The privacy statement says the following "standard" information is sent:

    * url of site

    * ip of client

    * browser type

    * phishing version number

    So what if browser type is lynx/opera/firefox? Are you allowing these?

    On a related note:

    the privacy statment says:

    For example, if you visited the MSN search web site at http://search.msn.com and entered "MySecret" as the search term, instead of sending the full address "http://search.msn.com/results.aspx?q=MySecret&FORM=QBHP", Phishing Filter would remove the search term and only send "http://search.msn.com/results.aspx".

    Nowadays it’s not uncommon to use the usual paths of an uri to actually pass information around, think about:

    http://server/url/with/sessioid/and/other/maybe/sensitive/info

    Would this also send the complete path to the server?

    Thanks to the IE Team for providing this in-depth information.

    – Markus

  21. Anonymous says:

    *LOL*

    the next step to get userinformation and a try to keep the browser monopol.

    i hope this will NEVER be reality.

    daniel

  22. Anonymous says:

    I’ve been thinking about this for a while, and I came up with a list of example URLs that should trigger the filter:

    1. http://#.#.#.#/ (addresses from ip addresses are always more likely)

    2. http://address.com:##/ (same as above except port number)

    I think those two are the most likely ones for phishing attacks.

  23. Anonymous says:

    This is a repost, but I’d still like to see this occur.

    The phishing filter can be smarter…

    "It checks web pages that don’t even have fields. The filter could scan for key words by input forms. Phishers must identify fields like credit card number, password, id, etc. for a victim to input. An additional security measure would be to check for encryption."

    Phishers MUST identify the fields with personal information. How else is a user going to input information. Look how we post comments. Posting comments requires a tinput for a title, name, and comments. Unless you plan to go further with this filter and include sites that exploit IE holes, I think this implmentation would cut down on bandwidth and ease some privacy fears.

  24. Anonymous says:

    I’ve not followed the news about IE7 recently. But will IE7 distributed as a mandatory security update over Windows update? Will it be part of an XP SP3? Why is the feature not supported for IE6? Most Phishing victims will run the OS delivered with the PC they have bought and will not care for the version of their browser.

  25. Anonymous says:

    http://blogs.msdn.com/ie/archive/2005/08/31/458663.aspx

    Rob Franco discusses the anti-phishing technology…

  26. Anonymous says:

    Hi. I can’t seem to find the company you bought this from. I saw it once on a paper, but no more.

  27. Anonymous says:

    "the URL query string, is stripped out of the URL"

    This is rapidly going to act against you, I am already seeing phishing sites which have a unique query string emailed to you. If you enter the site with a valid string you are simply redirected to the real bank’s website.

  28. Anonymous says:

    Excuse me, Rob, but I would feel more confident if you used some kind of hash code to do the lookup, instead of the real URL address. Please consider that: if you do the checking based on a non-invertible hash code your users would not feel it’s privacy is being broken in any way.

  29. Anonymous says:

    1.) The only guarantee we have is MS’s word that it will not give out private info. I don’t trust privacy statements, they are not legally binding. What can you do to *prove* to me that you aren’t gathering private info?

    2.) I consider my IP (which is static) to be private info. With that, and the URLs I browse, you can keep track of my browsing history.

    3.) SSL might mean that "bad guys" can’t see what IE sends, but it also means that *I* can’t see what is being sent. Again, all I have is your word that you’re "playing nice." What proof can you give me?

    4.) No reason was given as to why a 1-way hash cannot be used. This would help protect privacy and has been suggested numerous times.

    Another, larger issue. Someone mentioned about it not querying for domains you already visited. Is this true? If it is, this is terrible. There is a new threat, pharming. Pharming attacks the Internet at the DNS level. Meaning I actually type in http://www.paypal.com, but the DNS server is compromised redirecting me to a malicious IP rather than the real one. If you implement caching, this threat cannot be stopped.

    Anyway, I’ll just say that at this point I’m still very disappointed and have no intention of using the anti-phishing feature. I know I will also be encouraging others not to use it. MS really needs to be more forthcoming here. You provided some great information, unfortunately, you provided "corporate line" information – you neglected many of the questions/suggestions that were posted regarding privacy. Don’t just give us information, give us the information we’d like. I still have not heard anyone comment on the one-way hash idea. The only thing I see a hash doing over the real URL is that MS can’t invade our privacy. So why not do it?

  30. Anonymous says:

    Is it just me or do some people really not know what they are talking about?

    MsgPlus SHOULD be classified as a possible spyware. Do you know how many complaints anti-virus companies get becasue of problems resulting from the install of MsgPlus?

    ANY application that ships with a third(4th) party addware SHOULD be flaged as spyware.

    eeeek

  31. Anonymous says:

    MSN punch out a Philter for IE6.

  32. Anonymous says:

    Hi, my name is Tariq Sharif and I am a Program Manager on the IE Security team. One of the threats users…

  33. Anonymous says:

    Inside Microsoft tells of news that MSNs Phishing Filter add-in is available for download for US…

  34. Anonymous says:

    Inside Microsoft tells of news that MSNs Phishing Filter add-in is available for download for US…

  35. IEBlog says:

    When we shipped the Microsoft Phishing Filter in Internet Explorer 7 Beta 1, many readers on the blog…

  36. IE7 – フィッシング詐欺検出機能

  37. IEBlog says:

    As we’ve worked on the new Phishing Filter in IE7, we knew the key measure would be how effective it…

  38. Moderatoren: Matthias Niess und Timon Royer

    Themen: Die FSF Kampagne Bad Vista, was steckt dahinter? Opera für Nintendo Wii und Samsung Handys Phishing Filter für Browser, wie funktionieren sie? Erste Eindrücke vom Azureus Nachfolger Z

  39. In the keynote today at the RSA Conference 2007, the technology-security industry’s annual conference,

  40. More Gumbo says:

    For the SECOND week in a row, I’m heading into town for a lunch meeting at Bayou City Seafood and Pasta. This time, I’m working with two guys who developed the best stock trading course I’ve ever seen.

  41. When you’re looking for get web site traffic news and websites, be certain to tap into all of the sources available.