Sender authentication part 6: The basics of SPF

  • Article

In our previous posts on sender authentication, we were introduced to the concepts of SMTP, internet headers and how spammers will try to spoof headers.

One of the weaknesses of SMTP is that the sender can assign any email address as the Envelope sender and specify another email address as the sender in the message headers. Thus, if a message gets past your spam filter and hits your inbox, you might be tempted to think that the message is coming from someone who, in fact, did not really send the message. Most spam today carries fake email addresses so you can't really complain to the one who is purportedly sending you the message.

Fortunately, spam fighters have a tool to combat this.  The Sender Policy Framework (SPF) is an open standard specifying a technical method to prevent sender address forgery. More precisely, the current version of SPF — called SPFv1 or SPF Classic — protects the envelope sender address, which is used for the delivery of messages.

We will get into more details in a later post, but here's how it works in a nutshell:

1. A domain registers the IPs that they permit to send email. Let's say that our example domain is example.com. They decide to say "Okay, the only IPs that are allowed to send email from this domain are 268.192.168.0 to 268.192.168.14 inclusive."

2. An email server, tzink.com, receives an email and the envelope sender says the message came from user1@example.com. The mail came from 268.192.168.11.

3. Tzink.com looks up example.com's SPF record and sees that they permit 268.192.168.0 through 268.192.168.14. The transmitting IP is 268.192.168.11 and is within this range, therefore this email did indeed come from example.com.

In this case, this would be an SPF pass because the IP that sent the mail is permitted to send mail from example.com. However, suppose that the next message arriving at tzink.com is a little bit different.

1. The message was sent by user2@example.com, and the transmitting IP is 269.55.183.19.

2. Tzink.com looks up example.com's SPF records and sees that the transmitting IP is not within the range declared by example.com that is permitted to send email.

3. In this case, the message fails an SPF check (this is known as an SPF Hard Fail); the interpretation is that user2@example.com is being spoofed and the message is routed to the user's spam quarantine.

What the email server does with the message from that point forward is up to them, but the point is we have a case where an email claims to be from example.com but is transmitting from an IP that is not permitted to send email from that domain.

Those are the basics of SPF, but by no means is that all there is to it. Besides looking up the SPF records for the sending domain, comparing it to the transmitting IP and returning SPF Pass or SPF Hard Fail, there are other possibilities:

1. SPF Hard Fail - the transmitting IP is not permitted to send mail for that domain.

2. SPF Soft Fail - the transmitting IP does not meet a domain's strict definition of legitimacy, but the domain cannot classify the message as a forgery for certain.

3. SPF Pass - the transmitting IP is permitted to send mail for that domain.

4. SPF None - the domain does not publish SPF records.

5. SPF Neutral - the SPF check doesn't know and is returning no opinion.

6. SPF Error - the mail server received an error when looking up a domain's SPF record.

7. SPF Unknown - the mail server could not look up the domain's SPF record, possibly the domain's SPF record is improperly configured.

We will examine some of the syntax of SPF records in a future post (which will explain the difference between Hard and Soft fails). For now, look at number 4, SPF None.

Suppose that tzink.com gets a message from 287.64.43.33 from user1@yahoo.com (envelope sender). The From address in the message header says accounts@citibank.com and the subject line says "Please confirm your identity". This, of course, is a classic phishing spam. With SPF in our arsenal, we can grab sending IP and compare it to the SPF records from Yahoo. If the IP fails the SPF check, we can filter the message as spam. If it passes the check, we can contact Yahoo and let them know this user is using their services to send spam.

But herein lies the rub: yahoo.com does not publish SPF records. Remember that we perform an SPF check on the envelope sender, which is yahoo.com. The receiving email server looks up the SPF records for yahoo.com, sees nothing there, and returns an SPF None. The message continues to sail through the reputation check and if the content filters don't grab it, this message arrives at the user's inbox with accounts@citibank.com as the sender in their email client. Conceivably, this can fool some users (and it does all the time, judging by the number of phishing false positives I see).

And that's the conumdrum, SPF only works with domains that publish SPF records, and most domains don't publish them. I don't know the numbers but last I heard, less than 25% of all domains do. The whole point of domains publishing their SPF records is to prevent themselves from being spoofed in the envelope sender, so if domains don't get on board with SPF, this check won't work.

Intelligent spammers who phish (and even generic spammers) will spoof sending domains without SPF records. Because those checks will return an SPF none, a spam filter cannot drop the message right there (assuming that the client wants auto-reject SPF Hard Fails). By getting past that, they only need to get past the content filters. That, in itself, is a challenge but it's one less hoop they have to jump through.