How to reclaim your sender reputation - Introduction

The following document is part of a paper that I presented at Virus Bulletin in Sept, 2009, in Geneva.  It outlines the process that my team has iterated over to clamp down on the problem of outbound spam.

How To Reclaim Your Sender Reputation

Background

Sender reputation is one of the keys to email delivery. But before we get into the problem of sender reputation, we need to take a trip down memory lane and see how spammers have shifted tactics over time.

Back in the early days of spam, spammers would buy their own IP space from internet service providers and use it to send spam. This led to many of their static IP ranges getting listed on blocklists, thus preventing them from delivering their payload as spam filters would know not to accept mail from them.

Spammers evolved. Rather than use static IP ranges, they would deliver spam with viruses attached in it and trick users into infecting their machines and flipping them into botnets. These infected systems would then relay spam without their users’ knowing about it. Spammers could deliver spam from a very wide array of IPs and rotate them frequently, thus avoiding the problem of static listings on blocklists. Unfortunately for spammers, spam filters improved and got very good at listing dynamic IPs from infected machines. ISPs also started listing IP ranges for IPs that were not allowed to send email. Spammers could still deliver spam, but their job became as difficult as before.

Spammers evolved again. By moving to a technique called reputation hijacking, it became a huge problem in the world of email delivery. Rather than using a known bad IP range to send spam, or an IP without any reputation at all (such as the case of a botnet), spammers began to compromise known good senders and hide within their good reputation in order to get spam through. By sending a small amount of spam from a good sender, a spammer could avoid reputation filtering which is used to block around 90% of all spam.

This is the problem that many mail providers, like Hotmail and Yahoo, have faced over the past couple of years. Spammers abuse their free service to send out abusive content.

Shared Environments

Many email senders rely on their good reputation to ensure that mail sent from them lands in their target destinations’ mailboxes. When spammers compromise that sender’s reputation, they can seriously impede the flow of legitimate mail. Worse still, they can ruin that sender’s good reputation and it takes a long time to rebuild that credibility.

The problem of outbound spam is one of the most serious problems that Microsoft Forefront Online Security (FOSE) [1] faced in 2007 and 2008. It generated the single greatest number of urgent complaints among customers and posed a very serious threat to the business.

FOSE’s problem is different than that of Hotmail or Yahoo. FOSE is a hosted solution and is a shared environment. There are a common set of mail servers for inbound mail as well as for outbound mail. The inbound mail flows into FOSE’s servers, it is filtered, and then delivered to the customer’s on-premise mail server. For outbound mail, customers connect to FOSE’s servers and the mail is relayed to the rest of the world from FOSE’s outbound IPs. Not every customer uses FOSE for outbound mail, but the ones that do find it to be a very important part of their service.

To be continued.

[1] The full name for the division I work for is Microsoft Forefront Online Security for Exchange. However, the service we provide is not exclusively for mail servers running Exchange, we are platform-agnostic. For the rest of the document, I will use the FOSE acronym (pronounced “fozzie”).