Some early stats on TMA

  • Article

We finally got around to deploying all of our new features from our latest release.  As I explained a couple of months ago, I created a hybrid of SPF and SenderID in response to customer demand.  I called it TMA, or Terry's Message Authentication.  It was an SPF check on the From or Sender header in the case that an SPF check returned a Neutral, Error or No result.

I have four days worth of statistics, though remember that this is an optional rule and customers must enable it, it is not on by default.  Here are some interesting figures (well, interesting to me):

  • A From: soft fail occurs twice as often as a From: hard fail.
  • A Sender: soft fail occurs 13 times as often as a Sender: hard fail.
  • A From: soft fail is subsequently marked as spam 93% of the time.
  • A Sender: soft fail is subsequently marked as spam 95% of the time.
  • By contrast, a regular SPF soft fail occurs 1.2 times as often as a hard fail over the same time period.
  • Similarly, a regular SPF soft fail is subsequently marked as spam 95% of the time.
  • A regular SPF hard fail is marked as spam 93% of the time (TMA hard fails are automatically spam).
  • TMA is only triggered on less than 0.004% of the messages that hit SPF.

Going by these figures, it would appear that examining the From: or Sender: headers looking for spam seems to work as often as a regular SPF check.  However, the distribution changes significantly.  In regular SPF checks, Soft Fails occur a little more often than Hard Fails, but in TMA, they occur much more frequently.

I haven't got any FP figures for this feature.  I was looking through some of the false positives today to get a feel for some potential problems for this.  I'll keep the blog posted.