Twitter’s real time abuse detection

Last week I mentioned that Twitter was going to introduce their own URL shortner, https://t.co.  At about the same time that I wrote that post, Twitter announced via a tweet that they were getting better about Twitter spam:

Yep, we’re working on it. We are trying to migrate from “reactive” to “realtime” to “predictive” on that front.

I read about this on TheNextWeb, where writer Matthew Panzarino first described the story.  His thoughts:

Now, a reactive role is pretty easy to define. Currently Twitter users can report spam accounts right on Twitter.com by choosing the “Report spam” option on an account’s profile.

Once the account hits a certain threshold of complaints, it is suspended or removed, thereby killing off one spammer. Twitter is known to do this in batches, with a ‘spam purge’ of thousands of reported accounts resulting in a loss of somewhere between a few and several hundred spam accounts disappearing from users followers.

…

But the really interesting bit is the “predictive” part. Perhaps a database of spam phrases and accounts that Twitter uses to identify spammers and delete them automatically? A better way to vet users as actual humans before letting them sign up for accounts in the first place? A method for tying trends in with an internal spam search routine that monitors hot topics for spam and uses spikes in Twitter traffic to weed out the fake accounts?

Real time spam prevention is the Holy Grail of spam filters.  Everyone always talks about how they want to be more proactive when it comes to spam instead of reactive.  In plain English, this means that a filter wants to be able to predict that a piece of mail is spam without ever having seen it before, rather than having it sail through the filters which leads to users submitting it which results in spam signature updates.

Panzarino’s guesses are similar to what spam filters do.  In the early days of spam filters, they used to use key words and if spam contained them, the message would be marked as spam.  Soon, they moved on to regular expressions which allow spam analysts to create more complex rules and phrases. 

Vetting users as humans is similar to the CAPTCHA approach.  This isn’t novel because free web services accounts have been using them for years with limited results.

Tying trends to internal search results is interesting although more complex than other simple approaches.  I would think that Twitter would scan all URLs in messages against URL blocklists and also keep track of malicious DNS space (Where does the A-record point to? Has that IP been seen in spam before?  What about the name server?  How about the registrar?).  By keeping track of those types of things, Twitter can be predictive because if a spam URL resolves to bad IP space, they can block the tweet (or at least enforce another CAPTCHA).

Unfortunately, there are some spammy tweets (spweets?) that do not contain URLs.  Like anything else, Twitter probably has some sort of user reputation.  When did this person sign up?  How many followers do they have? What IP space are they signing up from?  How many malicious users have signed up from that space in the past?  Because Twitter deals with leading edge spam campaigns (spam that hasn’t been seen anywhere else), and because they are a new platform (as opposed to email which has been dealing with spam for over a decade), they have to get creative in their proactive techniques.  On the other hand, they all boil down to reputation (for pre-emptive guessing) and content filtering – the same as email.

One of these days, all of these services are going to have to get together and share data about what type of abuse they are seeing and from where.  It’s the same people doing it.