When doing IP reputation, generally speaking when you do an IP check, you usually do it on the connecting IP. The assumption is that the IP sending the mail directly is the one responsible for the IP reputation. There are exceptions, of course, but that’s how you do it.
One scenario where this fails is in the case of a relay. If an organization is forwarding mail through a relay from the outside world, and then relaying it into another organization, this creates a situation where the spam filter needs to know that the IP in question is actually a relay and it is not the source IP address. It’s kind of like the following:
IP1 –> spam filter –> destination
IP1 –> relay (IP2) –> spam filter –> destination
You might be wondering why this is even a problem. A typical spam filter would assume that IP2 is the source IP and has no need to look back a level. Indeed, what guarantees are there that IP1 is not spoofed? It’s easy to spoof headers in SMTP, and therefore, perform the IP reputation check on the IP that handed off the mail into your network because it’s the only one you can trust.
But here’s the problem; if the relay is relaying mail from the outside world, then you don’t want to perform the IP reputation check on IP2, you want to perform it on IP1. IP2 will look like a spam source since it will be forwarding all sorts of mail from all sorts of IPs from over the internet. These “all sorts of IPs” will include good mail and spam, but most of it will be spam. But since it’s all going out through IP2, the spam filter assumes that IP2 is spewing out tons of spam and builds a reputation on it. Its reputation degrades fairly quickly and it may even end up being blocked as a spam source. The spam filter simply cannot know that it has to look back past the hand off IP in order to extract the actual source of the email.
In the absence of more information, this really is a reasonable approach to take for the filter. You need to do IP reputation checks in order to cut down on expensive content filtering, and you need to do it on the IP that you can trust. Without that, you end up using a lot of bandwidth and resources. Unfortunately, the reputation check is being performed on an IP that you shouldn’t be checking.
So, how do you get around this?
You don’t, really. Not elegantly. For you see, somebody has to tell your filter IP2 is a relay IP. When the spam filter sees IP2, it says “Oh, this IP is a relay. Therefore, I have to look back a level and grab the IP before it.” The assumption here is that IP2 is a trusted IP, and because it stamped the IP connecting to it, the spam filter can trust the IP one level back as the originating IP. It is that IP upon which the reputation check must be performed.
Why do I say that this is inelegant?
Because somebody has to maintain all of those trusted sets of IPs. And they change. IPs get added, removed, change ownership, and go stale. And when they do, somebody has to update them on the filter side. People forget to do this when they do it infrequently. And when people forget, there are unintended consequences. Spam gets through, or good mail gets dropped. People complain, new processes are cooked up, but it only starts all over again because small companies don’t have budgets to automate this process.
Human management of these tasks is prone to error and my feeling on this is that it’s the only way to do it, and it needs to be done this way, but there really ought to be a better way.