Today is my six year mark
Today, July 12, 2010, marks the six year anniversary of the day I started to fight spam professionally. It’s also close enough to the four year anniversary of this blog. It’s well over 500 posts later and I am still going strong.
Looking back over the past six years, if I had to summarize the spam trends over time, I would put it thusly:
The biggest change is the proliferation and resiliency of botnets. They have gotten more important for spammers for getting their payload out there, but they have also developed more complicated and robust infrastructure. Both the good guys and bad guys have gotten better at infiltrating botnets.
Reputation hijacking is the other big shift. Sending spam out of Gmail, Yahoo, AOL, Hotmail, etc, has been around for a long time. However, over the past six years we’ve definitely seen a shift in botnets to not only sending out spam but also to create faux free web mail accounts… in order to use those to spam. They all also used to host fast flux, landing pages, and other sundry subterfuge.
Yet the more things change, they more they stay the same. Spam is still spam. There have been some tactical changes like the image spam outbreak in summer 2006, and pdf spam in summer 2007… but spam is still spam. They are still trying to push pharmaceuticals as much as ever, still pushing fake rolexes, still pumping out fake degrees; the point is that the content of spam is still the same. And they are still doing new tactics all the time. About the only thing that has changed here is that porn spam isn’t as prolific as it once was. I guess when it’s available for free online it’s not nearly as lucrative to the spammers.
I see spam as kind of a flattening business. Spam is not going away anytime soon, but it is being done by a more elite group of spammers. This is because spam filters are getting better and therefore it’s tougher to make money. You have to be a really good spammer to do it. But the flip side of this is that it has driven spammers to new business models – rogue anti-virus and black search engine optimization. These are two growth industries and have risen up in parallel to spam and grown at rates faster than spam has especially over the past 2-3 years.
Customer demands are moving from the realm of spam filtering to unwanted mail filtering. This relates to my particular organization more than antispam in general, I think, but we have started getting a lot of requests to blocked unwanted mail that is not spam. Gray mail is not spam; just because you don’t want a newsletter doesn’t mean that there should be a general rule to block it for everyone. But the flip side is that people want to outsource not only their spam filtering but also their unwanted mail filtering. This is a trend that has accelerated.
Measurement of false positives is still elusive. The industry cheats quite a bit with their SLAs, the language is deliberately ambiguous. If a company claims a 1 in 25,000 false positive SLA, what that means is that they permit 1 false positive per 25,000 messages. This means that if the spam/ham ratio is 10:1, then in 25,000 messages there will be 2272 hams and 22,728 spam messages. If one of the good messages is flagged as spam, then the good mail FP rate is 1/2272 = 0.04%, which is actually quite high. Yet by saying that you permit 1 in 25,000 messages, and messages is not defined but assumed to be both spam + non-spam, vendors have permitted themselves a lot of leeway when calculating how accurate their product is against good mail… by a factor of 10.
So, there are six things I have observed over time. Spam is still spam, I am still here writing about it (albeit less and less as I have expanded into general cyber-security). Who knows what the next six years will hold?