Over at Word to the Wise, Steve Atkins has a good post on URL redirectors. URL shorteners take long URLs and compress them into smaller ones. However, because bit.ly is the most popular URL shortener (because they originally were the exclusive shortener at Twitter), they are also the most abused by spammers.
The reason that spammers abuse bit.ly is because a bit.ly link is resistant to URL scanning by a content filter. They are betting that since a spam filter won’t mark a message as spam because of a bit.ly link (due to the volume of false positives), they can send spam messages with URLs to this particular shortener and not have to worry about getting caught by URL filter.
Atkins refutes this line of reasoning:
bit.ly have been on SpamHaus’s radar for quite a while. They’re listed on the SBL multiple times. They’re listed in the DBL – SpamHaus’s newish domain based blacklist, intended for content-based filtering of email. All this means that emails that contain bit.ly URLs are increasingly likely to have serious delivery problems.
This isn’t unique to bit.ly: many other URL shorteners have similar problems – j.mp, su.pr, and others. Nor is it unique to SpamHaus: many other spam filters, public and private, are starting to treat common URL shorteners with suspicion.
Naive use of URL shorteners in your email will send it to the spam folder.
One reason that Spamhaus lists bit.ly on their DBL is because they are seen in so much spam. However, they are not listed in DBL’s “block” zone but in their “URL shortener” zone. Their own documentation says that you shouldn’t use that zone to block outright, you should use as a weight in the spam filter.
But even then, using bit.ly as a weight in a content filter will be prone to false positives. The vast majority of links in bit.ly are legitimate. It is true that bit.ly is abused and that there are URL shorteners that either are set up for spamming, or don’t do a good job of abuse mitigation, but bit.ly is not among them. They fight abuse; this is straight off their blog:
The first [line of defense in bit.ly abuse prevention] is VeriSign’s iDefense IP reputation service. The iDefense system is focused on detecting and defeating malware. The iDefense blacklist includes URLs, domains, and IP addresses which host exploits, malicious code, command and control servers, drop sites and other nefarious activity.
The second is the Websense Threatseeker Cloud service, which we’ll be adding to our arsenal of anti-spam tools. Websense will analyze the web content behind bit.ly links in real time, using heuristic tools and reputation data to flag spammy URLs, malicious content and phishing sites.
The third is Sophos, an innovative security service whose behavioral-analysis technology goes beyond blacklists, to proactively detect spam and malware.
Obviously, bit.ly cares about making sure that spammers don’t abuse their service. They are not the lazy, fly-by-night single-coder type operation that sets up a redirector and doesn’t notice when someone takes advantage of them.
Because of this, a spam filter that decides to block messages with links to bit.ly will be prone to false positives – lots of them. Bit.ly is the most popular URL shortener. That’s reality and if you block it, users will complain (especially if you have a global antispam business) and it is not worth the support costs. Getting users to change their behavior is asking too much because they are accustomed to seeing and using bit.ly in Twitter.
Blocking mail because it contains a bit.ly link is like the current TSA screening procedures – it’s more trouble than what it is worth.