July 12 - My third year anniversary!

Today is a special day at Microsoft, it is the three-year anniversary of the day I joined Frontbridge (now Microsoft Exchange Hosted Services) as a spam analyst.  Ah, what a memorable three years it has been.

On our first day on the job, me and three others (the Fantastic Four) went down to Los Angeles for four weeks of training.  We met the other lone spam analyst and we spent the next two weeks learning about spam and how to fight it and then the subsequent two weeks doing that over and over again before returning north to Canada.

I have processed a lot of spam in my time but for the first two years my main focus was false positives.  I used to process about 90% of the FPs we saw and I became incredibly good at predicting which spam rules were going to perform well in the field and which ones were not.  In those days, our spam team's primary tricks of the trade were writing regular expression spam rules on the contents of the email message.  I would process all of the false positives and then go on to spam.  Whenever I came across a legitimate false positive (which wasn't often) I could often look at the message and predict what part of the message was tagged as spam by our spam rules.

Some time passed and we added on another spam filtering service (component) which was automated.  I was responsible for setting up the false positive process, and I became good at predicting what FPs were caused by this new component and which ones were caused by our spam rules.  Time passed but the spam stayed the same.  In those days, pornographic spam was one of the most common types of spam and obfuscation of words was the preferred filter-evasion technique.  We saw image spam back then, but it always was embedded in a link.

In 2005, we continued to process spam but we started seeing some more foreign stuff (due to our customer base).  Still, not much changed.  We saw stock spam, pharmacy spam, 419s, and so forth.  All the while I was still handling false positives.

In summer 2006, we saw a sudden shift in spam tactics.  Image spam hit our networks.  I had seen image spam before, spammers sometimes used it in their CAN-SPAM boilerplates in the footers of their messages.  But, this was a new tactic for which we were ill-prepared.  Spammers were inserting gif and jpg images into their spam messages and delivering mail that way.  At the time, there was a new outbreak every week and I was working six days a week trying to handle all of this stuff.  However, time passed, we got some new features implemented and the image spam problem started to drastically reduce.  My own personal image spam rules have blocked over a billion messages since they were implemented back in September.

Time passed and 2007 has rolled around.  There's a new breed of spam floating around, pdf spam and "gift-card" spam (which isn't new, but the payload to a virus is).  I don't process much spam anymore these days, but I still troll through our various inboxes to get a feel for what's going on.  Now, I am a Program Manager of (anti) spam effectiveness, which means I am in charge of collecting various measurements on our networks.  Furthermore, the scope of my duties has greatly expanded in the past three months so now I have a great deal of influence into driving and defining new antispam features.  In my opinion, this is a very natural progression because I felt that as a rule writer / spam analyst, I was getting close to the end of how far I could go and the logical next step was to move beyond spam rules.  I had to become familiar with a whole variety of anti-spam techniques.  This is not to say that we did not have techniques other than spam rules (far from it), but now I have a great deal of influence of reshaping the process of how we do it.

So, it's been an interesting three years.  Hopefully the next three are just as interesting.