Last week, IBM issued a press release describing an AlphaWorks project called FairUCE. The trade press picked up on it, and so did the mainstream press, and many bloggers -- including Chris Linfoot picked up on that coverage. The trouble is, some of that coverage was just flat out wrong. Noting the inconsistencies between various reports, I decided to follow up a bit. I got in touch with Mathew Nelson, the developer of FairUCE. He responded to my messages promptly and courteously, and he even gave me his private (non-IBM) email address, which is in fact protected by FairUCE.
Let's review some of the things that were reported about FairUCE...
CNET News.com published an article, with the following lead:
Unveiled Tuesday, the antispam technology is meant to take an aggressive swing at computers being used to deliver large volumes of unsolicited e-mail. After identifying a certain machine as an established source of spam, the software, dubbed FairUCE, bounces back any messages sent by the device in question with the intent of slowing that computer down and retarding its ability to produce more unwanted e-mail.
And a CNN/Money article that was forwarded to me said the following:
NEW YORK (CNN/Money) - IBM is set to unveil a service Tuesday that will send unwanted e-mail back to the spammers who send them, according to a published report Tuesday.
The Wall Street Journal reports that the new IBM (Research) service, to be known as FairUCE, uses a giant database to identify computers that are sending spam.
The paper reports that, using that database, e-mails coming from a computer on the spam list are sent directly back to the computer, not just the e-mail account, that sent them.
"We're doing it to shut this guy down," Stuart McIrvine, IBM's director of corporate security strategy, told the paper. "Every time he tries to send, he gets slammed again."
I wish I had access to the Wall Street Journal article that was cited, because I'm wondering whether it was the Journal or CNN/Money that totally botched the story. It's interesting to note, by the way, that a Google search for the lead sentence in the above-quote article will bring you to a very different CNN/Money article. It seems that they re-wrote their copy rather quickly it to correct the gross inaccuracies.
I don't know what that quote (or mis-quote, perhaps) about "We're doing it to shut this guy down" is all about. It's just totally wrong. Unlike Mathew, Mr. McIrvine hasn't responded at all to my email requesting clarfication. Mathew. however, indicated without any hesitation -- and he granted me permission to quote him on this -- that articles that referenced a "giant database" and "spamming spammers" were incorrect. Mathew also pointed me to an article in InformationWeek that he said got it mostly right.
What FairUCE really is is a three-layer system that first makes a "best guess" about whether the address information associated with an email has been spoofed, secondly checks whitelists and blacklists, and lastly (unless whitelisted) sends an email-based challenge to the sender. Mathew prefers to characterize it as an "inquiry" rather than a "challenge". There is, however, little to no difference in appearance between the two. Here is a reproduction (slightly edited) of the response I received when I sent a message to Mathew's personal address:
Your message entitled "Re: Questions about FairUCE" has not yet been delivered to Mathew Nelson because it arrived from comcast.net instead of rhs.com. Please select one of the following options:
I am email@example.com. Please accept my message entitled "Re: Questions about FairUCE"
I did not send you a message entitled "Re: Questions about FairUCE"
This is an automated response, please do not reply.
I got this challenge (or "inquiry") because my email messages do have some of the possible hallmarks of being potentially spoofed, and because my domain (or is it my IP address... to be honest, I'm not sure) had no pre-existing reputation within Mathew's FairUCE database. My email originates on a dynamic IP, and I must relay through my ISP's relay server because too many large ISPs blacklist SMTP sessions from all dynamic IPs. I do not, however, use my ISP's domain in the headers. I use my own domain, which happens to resolve in DNS as a CNAME record that points to the A record of the permanent hostname that my ISP has assigned to me within their domain. It's a bit of a convoluted configuration, and technically it violates some RFCs, but it works because all the standard DNS resolver libraries in common use resolve the CNAME internally without any action taken by the application code. The point of the challenge is to establish reputation for a never-before-seen domain, not to validate the sender. That's what makes it an "inquiry".
Based on what I've learned about FairUCE, I believe the key feature is that it accepts email from previously unknown senders without challenge if it does not appear to be spoofed. Almost all spam is presently spoofed, and most legitimate email doesn't appear to be spoofed, so the rejection rate is very good but the number of challenges that go out to legitimate senders is small. Those of us who run small businesses on dynamic IP addresses but use our own domain names instead of our provider's are the legitmate senders who are likely to get the challenges. Larger enterprise users generally won't receive challenges -- if their mail systems and DNS are configured properly anyhow. Bearing in mind that IBM's primary business interest is to serve these larger enterprises (despite the "new" committment to SMB that IBM seems to make on an annual basis, this is still absolutely true and likely to stay true forever), FairUCE does seem to make a certain amount of sense as a potential IBM anti-spam offering.
Unfortuntely, however, I believe FairUCE will have a fairly limited life-span as an effective anti-spam tool in its own right. The same is true for other identity technologies like SPF and DomainKeys. These are anti-spoofing tools, not anti-spam tools. This is not to say that these aren't important technologies, however. They will shift the field of battle. Effective identity tools will force spammers to change their tools so that they no longer spoof address information. They will force spammers to use the identities of the people whose computers they take over with their zombie programs, and to use the relay servers of the ISPs that those computers use for connectivity. Spam messages will be easier for recipients to trace to the source machines, and ISPs will be able to meter usage of their relays to raise early warning flags and isolate computers that are being used to send spam. That's a good thing. How it got turned into "Every time he tries to send, he gets slammed again", however, remains a mystery.
1. Devin Olson03/29/2005 05:21:18 PM
Very good commentary Richard. I have not yet heard about "FairUCE"; thanks for clearing up the confusion ahead of time.
2. ajp03/30/2005 04:59:16 AM
A great read thanks. I'm particularly interested in your comment: "Bearing in mind that IBM's primary business interest is to serve these larger enterprises (despite the "new" committment to SMB that IBM seems to make on an annual basis, this is still absolutely true and likely to stay true forever)" .
I support a number of very small Notes installations <10 users. This is only possible thanks to Express licencing which is a brilliant move by IBM and should be commended at all times. However, there is no doubt that the mindset at IBM simply does not recognise these small user bases. IBM's fixation with the big corporates is being blogged about more more often.
Reminds me of the great Talking Heads: "Once in a life time" ...
.. Same as it ever was...Same as it ever was...Same as it ever was...
Same as it ever was...Same as it ever was...Same as it ever was...
Same as it ever was...Same as it ever was...
3. Chris Linfoot03/31/2005 09:45:54 AM
Thanks for that summary. Suspicions confirmed, I am afraid and you even have a sample right there to prove it. This is a c/r system with one new twist
- If I send with a sender envelope in example.com and the host that delivers the email to the FairUCE protected system resolves in example.com, then it will be assumed not to be spoofed.
- but if (as in your case) the sender envelope and delivering mail host do not match (possible spoof), a challenge is issued.
Trouble is, that latter condition is way more common than Mr Nelson seems to think. We have four main production domains here, most users are in one of two dot com domains but the mail servers are in .co.uk. That means every one of our users would get a challenge to a legitimate email.
Add to this the fact that we have recently seen one of our domains used as the spoofed sender of a large spam run and you add injury to the previous insult. This spam run led to nearly 8,000 bounces coming to us within a single 24 hour period because target addresses did not exist. How many more bogus emails would have landed if the target sites were using FairUCE or any other c/r system?
I think you were too kind in your summing up.
As a proof of concept or technology demonstrator, FairUCE is probably very worthwhile, but it has no place whatsoever in any production environment in its current condtion.
Perhaps there is a reason they call it AlphaWorks (not Beta).
4. Gavin Bollard03/31/2005 11:41:38 PM
FairUCE seems to be a good attempt to get away from the awful content filtering problems that we seem to have.
1. Constantly playing leapfrog with spammers every time they come up with a new word or new spelling.
2. Occasionally screwing up our own systems by putting non-alphanumeric symbol in a word (eg: | $@ *).
3. Wasting processor cycles looking for words in every part of every email.
4. Blocking legitimate emails (our system blocks mail from one person with an unfortunate name - male organ as part of his firstname).
Anyone who implements fairUCE should ensure that are ready to do a bit of updating on their whitelists.
One thing bothers me though - where does DOMINO fit into this picture???
5. Richard Schwartz04/01/2005 12:34:10 AM
Domino would fit in the same way as any MTA. It would live behind FairUCE. Inbound traffic would go to FairUCE and whatever is accepted would go to Domino. That's reasonable for a proof-of-concept, which is all that FairUCE really is. If IBM ever does turn FairUCE into a real product, I .would certainly want them to do some tighter integration.