One of the challenges of going from a small anti-spam company (like Frontbridge) to a big one (like Microsoft) is finding ways to automate what we do. We can write spam rules manually, and that works for a while but as soon as we start getting more and more customers it doesn’t work so well. So, the solution to that is to create processes that scale better, and to do that we incorporate automated processes.
The problem is that that the more automated an anti-spam process is, in my experience, the more time you have to put into manually processing false positives. It’d be great if users always submitted clean false positives, but again in my experience, the good false positive submission rate is quite low. Because we spend a great deal of time trying to get our FP rate down as low as possible and we care about the quality of what makes us adjust our filters due to false positives, we process most of them manually. This means we quickly go through them “by hand” verifying that the submission is a stock newsletter and not a stock spam (and stock spam submissions happen all the time). We spend less of our time manually fighting spam and more of it manually “fighting” false positives.
It’s not so bad a tradeoff, though. The good news is that if a spam process is automated, then once the false positives have been verified then the spam processes can be adjusted automatically as well. Indeed, the bottleneck is how quickly somebody can determine whether or not a submission is legitimate.
Note: I’m aware that there are methods for semi-automating the process of false positive submissions. The problem relates back to my post on Two-Point Filtering.