I just read an article that Larry Seltzer wrote for eWeek entitled "The Moon and the Spam Filter. In it, he takes on the classic rhetorical question of "If we can send a man to the moon, how come we can't stop spam", and he makes the point that spam is a social problem at it's root, not a technological one. He quotes an unidentified participant in the IETF's Anti-Spam Research Group saying, "Technological measures are an arms race—a good stopgap, perhaps, but still a stopgap. It will require social change, and that's slow to happen, especially when its major opponent is laziness.".
It's a valid point, and I recommend the article, but there's something I just have to comment on. Seltzer asks, again rhetorically, "After years of effort and zillions of dollars invested by major players, why is spam still dominating the e-mail landscape?" and that question itself hits a sore point for me.
The headline from a New York Times article nearly two years ago, which is pictured above, is fresh in my mind these days. It was published a day or two before I gave a presentation on spam at Lotusphere two years ago, and I'll be doing a similar presentation next week. I do plan to revisit the headline near the beginning of the presentation.
This also harkens back to my unfinished series in my old blog, inspired by that headline, called What Bil What Bill Gates Really Could Do About Spam Part 1 and Part 2. The sore point that Larry Seltzer raised for me is that the major players haven't really spent "zillions" on anti-spam measures. They've spent millions, certainly, but it seems unlikely to me that Microsoft's expenditures on anti-spam research and development go into the billions, and more importantly: they haven't been spending their millions on the right things. It's bold of me to say that, I know, but I have what I believe to be solid grounds.
The conclusion that I never wrote to that old series of essays was this: Microsoft, more than any other major player, has the resources to spend and the opportunity to do something that really will cut down spam. Gates' prediction that the spam problem would be solved by 2006 was wildly off base because none of the technologies he spoke about in relation to that prediction stood a chance. This was not merely because of the implementation problems inherent in the technologies. It was because of the fact that Gates was ignoring two fundamental facts. One is that spamers are, by definition, criminals. This was true, by the way, before the CAN-SPAM Act, though it's ever more clear now that they continue to flaunt it. The second is that basic insecurity of tens, if not hundreds of millions of desktop computers worldwide provides sanctuary and free tools for the criminals to use.
The concept of "postage", which Gates seemed to be very keen on at the time of the two year pronouncement, but which he quickly seemed to back away from, was doomed to failure simply because spammers will find ways to make other people pay it. That's true of either of the postage approaches that has been considered: computational tax, or micropayment. Spammers don't use their own machines, so they don't care much at all about computational taxes, and if they've commandeered a machine it's not going to be much of a challenge to them to commandeer a micropayment account. Another one of the key technologies that inspired Gates to make his prediction was sender authentication. These various technologies -- when combined with the types of ISP-based counter-measures that the Anti-Spam Technical Alliance have been talking about -- make it possible to identify spamming computers and shut them down. With so many insecure machines out there to work with, however, this will not be much of a hindrance to spammers at all. They'll have moved on to another machine by the time it happens.
So yes, there's a social problem. I grant that. The "zillions" that Seltzer postulates, or some lower figure, that have indeed been spent on anti-spam technology are just stopgaps until that social problem is solved. I grant that, too. All criminal behavior, however, is legitimately classed as a social problem. Actually, it's two social problems: that of the criminals themselves, and that of the people who knowingly or gullibly profit indirectly from the criminals -- e.g., by buying their stolen goods. All the locks we put on our homes, cars and possessions are just stopgaps until the social problems of criminal behavior are solved, but it's still necessary to spend zillions on those stopgaps that protect us from crime. Seltzer, I'm sure, recognizes all of this. He closes his article with the observation "But just as things like crime and poverty never really go away, I think spam will never go away." Perhaps that's pessimistic, and perhaps not, but even if it's true we need to keep on working on it -- even with stop-gaps. The key is that we need to be working on the right stop-gaps.
It may indeed take zillions, but face it: Bill Gates has got them. Microsoft is doing lots of conventional R&D on the spam problem, but I say that anyone can do that sort of research. Microsoft's anti-spam R&D money should be aimed at the one dimension of the spam problem that is uniquely in their realm of expertise -- and arguably in their realm of responsibily --, and which only they can solve.
One could argue, by the way, that Microsoft isn't the only organization that can solve the problem of millions of old and unsecured machines. The open source community could provide a secure, lightweight OS that runs on all those machines, without a doubt. There's one thing they can't do, however: provide an incentive to millions of owners of old machines to actually switch.
One of the reasons that I never finished the series of essays that I had started -- it;'s more than a year and a half ago now -- was that at the very time I was getting ready to write part 3, which was going to say what I thought Microsoft should do with those billions of dollars they had in the bank, they went and gave a bunch of it back to their shareholders. It seems to have turned out, though, that Microsoft still does have plenty of cash left, so now I'm ready to say exactly where they should spend it if they want to make a dent in the spam problem. They need to invent or adopt a technology that can secure those many millions of computers that are running Windows 95, 98, Me, NT, 2000, and XP. Longhorn and .NET aren't it, given that their hardware requirements vastly outstrip the capabilies of most of the computers in question. They need to make that technology as ubiquitous worldwide as AOL CDs once were, and they need to provide a compelling reason for every owner of every insecure machine out there to install that technology. The technology could be a new, lightweight, but secure version of Windows that runs on old hardware. It could be a lightweight VM executive that runs Windows and monitors its behavior to detect intrusions, or just to detect outbound spam for that matter. It could be a physical device that plugs into the modem or network jack to provide security. It doesn't really matter waht the technology is. What does matter is that it works, it works on old machines, that anyone can get it easily, and that there's a reward to be had -- perhaps a deep discount on your next Microsoft purchase -- if you install it and keep it installed.
I don't have to do the math to know that this would really cost Microsoft multiple billions of dollars. I'll even go with Seltzer and say it will cost "zillions", and that's the reason that I end up agreeing with Seltzer's ultimate conclusion: that the spam problem will never go away. Not until, at least, the entire current generation of computers and software is eventually replaced with truly secure machines, so that the sanctuaries and tools that spammers can use to stay ahead of all the other stop-gaps will no longer be available.
1. Chris Linfoot01/18/2006 04:06:08 AM
What you talkin' 'bout there, Rich?
Spam ended here years ago.
OK, it didn't but none of my users know that because the (very simple) technology we have lined up against the spammers works.
Call me an optimist, but I think technology is winning.
2. Richard Schwartz01/18/2006 07:27:25 AM
Care to total up the cost of the man-hours you (and everyone who has contributed helpful information directly or indirectly to yo, and everyone who runs the blacklist servers you use) have put in to achieve your results? And add to that all the time you (and/or your users) spend checking for false positives. Even if there are hardly any, it still takes some time to check.
I use your techniques, many of them anyhow. And I use some of my own, techniques, too. I capture or deflect a lot of spam, and I don't get a lot of false positives (except when I'm experimenting with settings trying to improve things). My numbers aren't quite as good as yours, and spam definitely does still get through. If the pattern holds, and if I do nothing else, I'll probably get through another year before the amount that reaches inboxes gets to the level that users find it intolderable again -- and then I'll need to raise the game to a new level. And meanwhile, the number of spam delivery attempts -- and the bandwidth consumed by them -- will continue to grow.
3. Jon Johnston01/19/2006 07:27:16 PM
There's a simple reason as to why spam won't end, and that is because it's monetarily beneficial to the people who are sending it, whether it's "email marketing" or the malicious types who would use it to infect machines. There's more money in doing it than there is risk in being caught and prosecuted.
Chris doesn't have spam because he is master of his own world. Few are that lucky.
4. Richard Schwartz01/19/2006 07:47:13 PM
@Jon: You're quite right, but if the sanctuaries provided by insecure machines weren't such a big problem, that would help increase the risk of being caught and prosecuted. There's still the big problem of the ease of electronically crossing international borders versus the difficulty of enforcing laws across them -- but that's something that ASTA could address by putting a throttle on SMTP traffic coming from routers in countries where they don't cooperate with enforcement.