Our latest outbound spam mitigation technique
One of the things that has kept me busy the past few weeks (read: months) is outbound spam – again! No matter how many mitigations we put in place, it’s never enough.
The current challenge that we are dealing with is compromised accounts. Most of the time, but not always, this happens with educational institutions. Someone gets their username and password credentials leaked to a phisher. This can happen multiple ways – phishing attack, password reuse, weakened security elsewhere or possibly even brute force. The net result is that someone other than the owner of that email account is in possession of the login credentials.
Since we are a mail filtering service that acts as a relay, we are not in command of validating login attempts. Thus, all mail going through us we have to assume is already validated upstream. However, we know that this is not the case because piles and piles of spam passes through us all the time every single day from multiple different compromised accounts. Therefore, we know we have an outbound spam problem and that we are relaying it, the problem is that we do not control the login process. I should say that more accurately that piles and piles of spam attempt to pass through us all the time. For you see, in the pass few weeks we have been experimenting with a new brand of outbound spam mitigation. It works like the following:
We keep track of all of the outbound mail that flows through us, and we track how mail is marked as spam (we route all spam mail through another set of IPs). We also measure volume of mail coming from each account because not all mail is marked as spam (unfortunately, these are false negatives and it is a byproduct of spam; it’s zero-day spam).
Every xx minutes (it's not a long time), the logs are pulled to a central location, and then we have a script that parses these logs and rolls them up into a database.
Another script then fires and looks for accounts that are sending high percentages of mail marked as spam, or high volumes of mail that have never before sent mail in high volumes.
Email accounts that are deemed to be potentially abusive are automatically added to a list of banned senders. However, this is not a blanket addition. Only email addresses that belong to organizations that are high risk are automatically added. What is an organization that is high risk? We have done some profiling of orgs that are more prone to this. Those that fall outside that definition but are caught twice for spamming are then added manually. So, we give a company or school or government the benefit of the doubt the first time but not the second time. The time it takes to add to the banned sender list and replicate across the network is between xx minutes (not a long period of time).
Once we automatically ban, we fire off an email to our support team who contacts the customer about the action we have taken. One of the big problems we have had in the past is taking silent action on customer mail that impedes their delivery. Customers are not so much concerned that we take action, it’s that we take action and don’t notify them.
However, given this process, we acknowledge that from time to time, we might hit a false positive. If that happens, a customer must escalate to us. We can add a particular account to a list of exceptions to not auto ban (for example, a newsletter email account), and we have steps that our client services can take to remove the account from the banned list so that the feature team doesn’t have to get involved.
We have found that this process works very well for detecting and blocking the most egregious offenders. It is not perfect and it still misses some cases of accounts that are not compromised, but it is has managed to cut down a lot of our outbound spam automatically with minimal human interaction. We are still refining the process to catch other scenarios (small leaks of spam, compromised machines with malware) but we’ve made yet another good step to being a responsible email citizen.