I am Annoyed with Spam
…and I don’t get it, why is it still a problem.
If you follow my blogs, then you may remember an article, where I mentioned that today approach to computer security is extensive and not leading anywhere in the end. Recently, I was asked, which approach would lead somewhere? At the time I had to evasively answer that it would take one genius to invent such approach, and another to “sell” it, even for free.
I suspect that the second statement still holds, but I am not so sure about the first one. Genius? Really? Let me show it on the example of spam – the problem that rages on Internet for years. I dare to show a solution, which does not take a genius – just the desire and a little respect to the users are enough, but significantly undermines both user impact and spam economics, if not eliminating spam altogether in a few years.
A little off-topic – how do we fight spam today?
Frankly, when I think of spam fighting today, it reminds me the fight for communism in the former Soviet Union. It takes a lot of time, it creates jobs, and it does not bring tangible results. Interestingly enough, the fight with span is almost as vast – it includes legislation (CAN-SPAM Act, which essentially permits spam, even if a single-timed one), intricate schemes for SMTP and other mail servers, and even such Frankensteins like Artificial Intelligence-based spam filters. By the way, if you’ve read my Russian blog, you may remember that Microsoft #2 Russian blogger at Microsoft for 2006 (I was #1) Boris Kogan works exactly on this kind of spam protection in the Microsoft Exchange Server.
Sure, it all looks logical and wise, only the results are still not quite on par with spammers. Just check your mailbox and see for yourself. Sure, Boris can quote how many spam messages his code can kill in a day on a single server, but the spam still comes and gets through… Besides, it’s a little too high-tech for such a basic problem. Don’t get me wrong, high-tech solutions is a good thing, but it just seems that there should be some simple and efficient solution, simple, but efficient, something that can solve the problem. Something like “he had a black belt in kun-fu, karate and taek-wan-do, but I had a gun.” Can we find one?
Let’s start with the user scenarios with a very simple minded user. Right now I am sitting in a Barnes&Noble bookstore, and there are collections of anecdotes on sale: “Redneck jokes”, “Farmer from Iowa jokes”. So, let’s see, what would an imaginary redneck from Iowa sitting on a porch of his house on a farm somewhere in Kentucky think about the spam problem?
User scenario – a farmer buys a computer
Just a necessary disclaimer, although I am using these folklore personages, I lived in Iowa myself and loved it, and I know many decent men, who proudly consider themselves “rednecks”. Not to mention that my manager, who was born in Tennessee, helped me to edit this post. Well, all the more reasons to think what they think about it and what their problem with spam is. Right?
Now, let’s hear the evidence first face (sorry, I considered simulating Southern talk, but wisely decided against such an attempt. After all, I ought not to compete with masters, I’m just fixin’ to tell ya about spam, not to present a literary masterpiece):
I sat on the porch of my house, stroke my charged gun lying on my knees, and sipped home-made whiskey from a bottle. Missy makes it clean as a tear, God bless such a wife and her father. If not for Old Jeff and his gun, I’d never have guts to marry her, and where could I find another like her? Missy is good – keepin’ the house in order, good mother, and when it comes to bed… True daughter of the South, the way I like it. Anyway, back to business.
So, I sit on the porch of my house, quietly, ain’t asking for no trouble. For now. Usually, I don’t keep a gun on my knees, no need, it hangs nearby anyway, but today I had a reason. Two days ago some city folks drove by my house in their fancy car, and yesterday they did it too, lookin’ around for somethin’. So, I decided that if they show up today, I’m fixin’ to teach them a lesson. After all, God gave man a gun to protect his property.
So, I sit, drink, and then Missy comes from the house and tells me that there’s a mail for me in the computer. Computer is a separate story. Last year the corn grew real well, we weren’t even able to save it all, some of it got wet and rot. Of course, that had a good side too – that’s why we have so much home made whiskey now, it would be a pity to waste that corn. But even what’s left made a pretty coin. Then the son of Pat from the farm down the road, behind the Snaik Crick, came from the city, where he lives, to visit his parents. I went to Pat the second day after he came to borrow a couple of cans of oil for the harvester and we talked. “You should buy a computer with Internet”, he said, “It will let you have weather forecasts for free. And it will be simpler to work with suppliers – to place an order or check if seeds or fertilizers are ready for pick up. It will also let you check with the bank, if they had the next payment for the harvester in place and what you have on the account. And it all instant and free.” So, we talked with Missy and decided that we should buy such a useful thing.
Pat’s son fixed it up for us, connected to the phone line, showed us everything, explained. “This is a monitor, which shows everything. This is keyboard and a rat to control the computer, and this is the system block with a hard drive…” Or something like that. I don’t know what does this “hard drive” drives, and why is it so hard, but who cares?
Anyway, a good buy, really useful thing. And when our son started to send photos from the Army, we even started to watch TV less. To tell the truth, it happened to be not all free as promised. That Internet, that Pat’s son spoke about, was almost twenty bucks monthly. A large bottle of real whiskey bought from a store, by the way. But it was worth it.
So, I get up and went to check what’s the mail came to me. Opening it, and what do I see? Somebody offers me Viagra, five bucks a pill! Whatcha those guys talkin’ about? Me and Missy don’t need no Viagra. What do these skunks presume?...
So, I call Pat’s son and ask, whatcha goin’ on?
And he said, “Oh, that’s spam.”
“Huh?” I say. “What’s pork got to do with it?”
“No, not pork,” he says. “That’s how people call any unasked advertising sent by email.”
Then I exploded. “What? What does this city folks permit themselves? It’s my computer, and my hard drive, I paid for them, and this Internet too, and they use it send me their junk ads? Who let them dump this junk on my property?!”
“C’mon,” Pat’s son says. “Just delete it, like I showed you, and forget about it. You don’t worry about postal junk mail in your mailbox, right?”
“Don’t worry?” I say. “Nope, no, sir! Postal mail is a different story. I don’t pay for the junk mail I get in the mail, and mailbox is the property of U.S. Postal Service. With email it’s different. I paid for the hard drive with my own money, and this Internet ain’t free either. I think these are the same skunks, who nosed around yesterday. Oughtta have figured out that I am waitin’ for them with a gun, so instead of comin’, tried to get me through the computer!”
“I don’t think so,” Pat’s son says. “They could be on another end of Earth. I’ve heard that most of the spam today comes from Russia now.”
“What?! Russians again?! What the …<skipped due to inappropriate vocabulary>… Congress is looking for?”
So, I said bye to Pat’s son and went to write letter to Congress. Here I am, sittin’ and writin’. Honorable Senator, I am a taxpayer and a registered voter, maybe you can tell me how an honest citizen can protect his property and what are you doin’ about it?
Now, let’s look at the solutions offered by the software industry. See the problem?
For some wild, completely incomprehensible reason, we believe that anybody on another end of Earth has the right to put and advertisement into your email inbox on your disk through the bandwidth that you paid for. Ok, ok, sure that’s not 100% correct, we believe that he has this right if anti-spam filter, mail server and Congress did not considered him a spammer. Still, the main presumption is the same. Somebody – not you – gave this guy the right to send you ads. Somebody else monopolized this right and tries to make you believe that this is the way. What’s worse, we (computer industry) already convinced a lot of users. But THIS DOES NOT MAKE IT RIGHT!
And now, let me describe – no, not a solution, but how email could look like if the principle of private property was firmly in place.
Solution – phase 1: Mail client
This is the first phase, which is very easy to participate even in the existing joyless state of email technology. All you need if to change existing email clients. No need to change all of them, change those under your control (if you develop one or develop add-ons to some). Whoever will be late, well, their users will simply continue to suffer from spam, that’s it. So, simply don’t be late.
The key idea is simple – it’s not a server, not a filter, not a Congress,
it’s a user who must decide, who can send letters to him/her.
A mail client like Outlook or Evolution may help the user a lot with that. The principle is simple: mail client must maintain the “white list” of permitted addresses. Any message from any other address does not show up in the inbox at all. Instead, in a special folder “Request for permissions” the user gets a short notice “Joe Smith with address firstname.lastname@example.org asks for your permission to send you email messages using SMTP server mail.smith.info.” And two buttons: “Permit” and “Ban”. Actually, you don’t even have to open that notices, it’s only for inexperienced users who double-click on them by an accident. Normally, if you need to permit somebody, you just right-click on his notice in the list and select “Permit”. Pressing “Ban” is not necessary, anything not permitted will be automatically cleaned up from you “Requests” folder in a configurable period of time, say, a week or a month. What button “Ban” does is to ban the sender forever, so that even requests for permissions from him don’t show up anymore. Considering that spammers today rarely use the same address twice, the usefulness of a “black list” is limited, but sometimes it just feels good. J
Makes sense? But what if you need the message on the first contact? What if the first message contains important information that you don’t want to loose? Say, the first message from a potential employer? No problem whatsoever. Your mail client does not actually throws the messages, it just keeps it inaccessible until you make your mind. Some advanced client may even not download it from the server until it knows what to do with it, although even downloading and keeping invisible may be just fine. The key is that mail from unidentified senders is kept for some time, but kept inaccessible! So if you decide to grant the request and permit the sender, you will get the message all right.
Now, the last strike: the “Requests” folder is located in a secondary place on the screen, never emphasized in any way even if there are new requests, and generally does not attract attention. Hence, the user comes there only if he really expects a message from a new sender, which is not too often event. Sure, some “analytical” persons will laboriously come there every day to do a “duty” of reviewing requests… they’ll learn with the time and stop doing so. “Requests” folder does not take time. You don’t go there unless you expect a message from a new sender, and even then, you just find it, approve it, and you are done. That’s it.
Well, two more points. Your business email should not require permission for senders within your company, it’s pretty obvious. And if you sent somebody an email, you automatically permitted him to reply. Now, that’s all.
“I don’t understand,” would say your vice president, “what’s the difference, if the message already saved on the disk? Why not show it? And what does it change?” And if you work for a small company, you would know that your career is over, because he already filed you forever under the label “guys, who talk too much nonsense.” Well, he is a vice president, he has enough money, he does not have to understand. But the difference is HUGE. Because there is no advertising event, no hit, no impression, your mind is not impacted. And you only go to the “Requests” folder only if you really waiting for a messages from new senders, and even then, you know specifically what you are looking for. So, on one hand, you save your time, and on another hand, spam does not fulfill its advertising promise. It’s true that spam is very cheap, BUT (!) any price is too high if it returns zero results. Makes sense?
You see? You took the very reason for existence from spammers. The user does not see even the subject: filed, which is used to lure him to open the letter, not to mention the content. Sure, they may try to use the last pieces of information they control like address and the name. However, the names like “Clarissa Hot Girl” and addresses like “Viagra4You@somewhere.com” are quite easy to filter, not to mention that you know for sure that this is spam. And even more, if you only look into “Requests” folder to approve a rare expected new sender, you even don’t look at these names and addresses.
Now, that we have covered the main scenario, let’s see what happens with boundary cases.
What if I prohibited somebody by mistake?
So, go and permit him back. Not in one click, but it should be possible. Go to the “black list” and take him out of there. And next time use the black list if somebody is really annoying. For the rest, being not permitted is quite enough, no need to specifically prohibit them.
What if I permitted a spammer by mistake?
Again, don’t see a problem. Any messages will have near the sender’s address two small buttons (or right-click options): “Revoke permission”, “Ban forever” just for this case.
What if am subscribing to some maillist?
If you subscribe to a maillist, you know what address will be used for confirmation request. Go to the “Requests” folder and permit the sender as usual.
What if my friends forget to configure their right name?
Well, privacy is a good thing, but politeness is important too. Letting people your name if you want to talk to them is generally a good idea. But in any case, it does not matter much. The request for permission will merely look like:
«Anonymous» from address xF5ak0boo31@hack.com asked for your permission to send you email messages…
If you know who this xF5ak0boo31 is and expect a mail from him, just permit it.
What about corporate email? Same procedure to get a mail from a peer in the next cubicle?
No, of course. It’s simply a normal configurable system of exceptions. Say, all messages from the domain microsoft.com that were send within the corporate network (no intermediate mail server outside), are automatically permitted.
What if my job requires me to get a lot of mail from unknown people?
Yes, that happens. In this rare case, you will have to turn the permission-based scheme off, so the President of United States and Senators will not have this luxury. However, something tells me that they are not the primary target of spammers, so it will not help spammers. And it will not hurt us, who will use it.
Besides, even public figures can transit into receiving citizens requests through a web form instead of email. In this case they will have a known SMTP server and sender, and web form has other advantages like security.
Solution – phase 2: Receiving mail server
Now, we can go to the next phase – moving the defense to the level of a mail server. Extend POP3 and IMAP protocols a bit, and the server will be able to receive from the mail client “black lists”, and hence implement the preliminary filtering before actually receiving it. And if the user will choose so, it can do the same for anything outside “white list” too.
An important change is that when refusing to receive a message, it will return a special code, that sending server and client can recognize and interpret. What it does, is lets the spammer know that this user is protected. Normally exposing extra information is considered a bad practice, but in this case, a spammer who rented email list will know that mail list vendors cheated on him. And it feels good to make spiders eat each other.
Sure, it will only make sense when there is enough clients supporting that or if the same company develops both the client and the server. But eventually, it will be there.
Another valuable improvement is that receiving server can distinct the sending SMTP server. It means that if a mail to “email@example.com” comes from smtp.spam.com, the receiving server may reject such messages as an evident spoof. Sure, mail is sent sometimes through intermediate servers, however this technology was developed long ago, when the Internet “was turned off for the night”. Today, there is no need for intermediate servers, sending server may connect directly to the receiving sever of the addressee, and if it’s not online, there is nothing bad to wait and try later. Besides, intermediate servers become more and more rare because of the spam problem anyway.
Solution – phase 3: Sending server and sending client
Once enough receiving servers support this policy, it allows to introduce a new sort of logic, when any messages not in a white list if rejected, but a request for permission is still delivered by the receiving server (unless the sender is in the black list). In this case, the sending server of the client, may retry an attempt to send later, in a reasonable period of time, when it’s expected that the other side may have reviewed the request and gave the permission. And if the receiving side will send a note with permission, resend the message again. A very good thing with that scheme is that the load goes almost completely to the sending system, which is quite logical.
What’s wrong with the today practice of, say, hotmail, to use as a white list the address book?
A lot. First, you cannot put simply an email into the address book, for some reason they ask for a name and a ton of other stuff. And if you are enraged with that write something like “A—hole” in the name field (meaning the mail provider, not your friend of course), the next time you send the mail to him “From:” field will look like:
To: “A—hole” <Bob@smith.com>
In other words, something that you don’t like to happen. Not to mention that address book system is much heavier and hard to use.
Second, your choice in such systems with an address book is to either save message from unknown senders in a special folder or delete it. If you delete it, you lose first time messages and have no clue that it happened. If you save it, you still get full impact of spam once you go into this folder. Requests for permission solves both problems.
And third, proposed scheme includes additional data like SMTP server, that are not saved in the address book.
In other words, the difference is like between Mercedes and a country-style cart. Both has four wheels, both get you to the destination, there is a place to sit, but the sensations are somewhat different.
What if it will be inconvenient for the user?
Then he will turn it off. Meanwhile, it’s inconvenient for the user to receive hundreds of irrelevant messages per day.
What if the user does not know from whom he will get a message?
You are kidding, right? And what if he does not know how to press buttons?
Of course, there are cases when receiving from unknown senders is part of the job. Congressmen, mayors, other public figures have this problem. See above about that.
What if maillist sends tons of reminders and all from different addresses?
Then the people, who created software for such a maillist, will have to update this software. That’s it.
My forums sends mail from the name of the user, who sent me a message. So, should I get the whole forum into my white list?
Again, creators of the software for your forum will have to update their software to from From: and Reply-To fields properly. From should be from the forum, and you may include it into your white list. Individual user’s address may be in the reply-to field.
I like to send greeting cards. But then, they come from another server!
If you addressee sees a mail from you from another server, he will for sure permit it, right? If not, may be it’s not a good idea to send those today either.
So, here we are. Now, was it really so complicated so that for a decade of spam nobody could think of it? I don’t believe it.
The idea is evident and trivial. It’s even not patentable – it was used long before in chats and instant messaging. I remember getting a question in chats in 96-97 “The user X wants to talk to you privately, do you agree?” It’s all invented and in place! All we need, is to implement it.