How to create more aggressive Bulk email settings in Exchange Online

  • Article

 Update 2014-04-04: Updated the Text Patterns for ETR#1 - modified #2, added #12 and #13

Update 2014-08-25: This article is now updated by this one: Different levels of bulk mail filtering in Office 365

One of the more common requests in the Forefront Online Protection for Exchange (FOPE) and Exchange Online Protection (EOP) services is “How do we get more aggressive bulk email filtering?”

In a previous blog post, New Features in Office 365, I discussed how we released a feature that allows administrators to mark bulk email as spam.

The way this feature works is the following:

  1. We have a list of bulk senders that we identify by sending IP address (we are constantly updating this list). If an incoming IP is on this list, we stamp the message with SRV:BULK in the X-Forefront-Antispam-Report header.

  2. There are also spam rules on the back end that look for patterns that commonly occur in bulk email messages. When one of these spam rules matches content within the message, we the message with SRV:BULK in the X-Forefront-Antispam-Report header.

The drawback of these two methods is many customers find these settings are not aggressive enough (#1) or they are too aggressive (#2). Because bulk email is so subjective (some people want it, others do not), it is difficult to strike a balance such that everyone is satisfied. In order to control false positives, we generally are less aggressive than we could be on bulk email.

 

Making Bulk Email Scanning More Aggressive in EOP

For customers who want more aggressive bulk email settings, there are ways to accomplish this with Exchange Transport Rules (ETR) in the EOP service.

There are several patterns that frequently occur in bulk email. Create an ETR (Admin –> mail flow –> rules –> Create new rule by clicking the + sign) that says “If the message includes these text patterns in the message subject or body” and then start adding the following regular expressions: 

  1. If you are unable to view the content of this email\, please
  2. \>(safe )?unsubscribe( here)?\</a\>
  3. If you do not wish to receive further communications like this\, please
  4. \<img height\="?1"? width\="?1"? src\=.?http\://
  5. If you would prefer not to receive e\-?mails from
  6. To stop receiving these\s+emails\:http\://
  7. To unsubscribe from \w+ (e\-?letter|e?-?mail|newsletter)
  8. no longer (wish )?(to )?(be sent|receive) \w+ email
  9. If you are unable to view the content of this email\, please click here
  10. To ensure you receive (your daily deals|our e-?mails)\, add
  11. If you no longer wish to receive these emails
  12. to change your (subscription preferences|preferences or unsubscribe)
  13. click (here to|the) unsubscribe

In the above description, ensure that you choose text pattern and not “subject or body includes any of these words”. The patterns above are regular expressions that match on patterns of text, not exact matches of text.

This is not an exhaustive set of regular expressions; more can be added as appropriate and any can be removed as needed. However, it is a good starting point.

The action should be to modify the message properties and set the SCL to 5 or higher which will mark the message as spam.

When complete, the rule will look similar to the below.

image

A second ETR can be created that looks for exact matches and sets the SCL to 5. But whereas the above looks for text patterns, this one should look for “subject or body includes any of these words”. Below is a list of commonly used bulk email phrases:

  1. to change your preferences or unsubscribe
  2. Modify email preferences or unsubscribe
  3. This is a promotional email
  4. You are receiving this email because you requested a subscription
  5. click here to unsubscribe
  6. You have received this email because you are subscribed
  7. If you no longer wish to receive our email newsletter
  8. to unsubscribe from this newsletter
  9. If you have trouble viewing this email
  10. This is an advertisement
  11. you would like to unsubscribe or change your
  12. view this email as a webpage
  13. You are receiving this email because you are subscribed

This second ETR goes beyond the scope of what is in the first one. Once again, this list is not exhaustive and more can be added as necessary, or removed as required. However, it is a good starting point.

Using both of these ETRs can help customers cut down on the amount of unwanted bulk email in EOP.

What about legitimate bulk email that some users want?

The above ETRs are aggressive and will most likely flag messages as spam that some users want. To work around this, customers have two options:

1.
Use safe senders to override a spam verdict
Customers who use safe senders (in Outlook) and sync them to the EOP service will be able to get email from a sender even if the ETR marks the message as spam. In EOP, a user’s safe senders list overrides a spam verdict set by an ETR.

There is more information about how to use safe senders in this blog post:  
[How to use safe senders in EOP and FOPE<br>](../tzink/how-to-use-safe-senders-in-eop-and-fope)

  1. Narrow the scope of the ETR down to a more specific set of users
    The above ETRs mark a message as spam for the entire organization. However, ETRs are very powerful and you can specify multiple conditions. By selecting the “Add Condition” option, customers can specify which recipients the rules should apply to.

    This way, the aggressive bulk email filtering settings can apply to a few users who are highly targeted, while the rest of the user base (who mostly get the bulk email they signed up for) do not have their mail flow interrupted.

Both of these options will help ensure that people can still get the bulk email they want to receive. Option 1 is disruptive for the general user base but allows individuals to self-service (they can add safe senders as required), whereas option 2 is less disruptive to the user base but forces the customer’s administrator to do a bit more work up front by collecting a list of highly targeted users, and creating a more complicated ETR.

What about FOPE?

The above ETRs can be created as Policy Rules in FOPE. However:

  1. FOPE does not support the same level of regular expressions as EOP

    EOP has a much richer set of regular expression support compared to FOPE. Some of the regular expressions can be rewritten, but some of them may not work.

  2. Safe senders in FOPE do not override Policy Rules
    In FOPE, a Policy rule to block spam can either Quarantine or Reject. It does not mark a message as spam. Therefore, if a user wants to receive email from a particular sender, a safe sender will not work if the Policy rule is scoped to the entire domain. The only workaround is to scope the original policy rule down to a smaller set of users, or create a second policy Allow rule from Sender A to Recipient B.

As you can see, FOPE is more limited than EOP. It will not be upgraded to support the same variations as EOP.

Summary

The above descriptions are the current workarounds for more aggressive bulk email filtering in the service – how to flag more of it as spam, how to allow users to receive the mail they want, and what the limitations in FOPE are (the older version of service).

 

Related Articles