The Sender Policy Framework and Caller ID for Email

SMTP was never designed to be a secure protocol. Back in the early days of the ARPAnet (predecessor to the Internet), designers assumed that all hosts that connected to the network would be trusted and trustable; they didn't foresee the explosion in Internet connectivity driven by cheap, high-bandwidth connections and powerful sub-$500 PCs. Because of SMTP's design, there's no guaranteed way to prevent senders from forging headers or impersonating other computers. Two of the many solutions that have been proposed to this problem include the Sender Policy Framework (SPF) and Microsoft's proposed Caller ID for Email specification. (The SPF standard is described fully at http://spf.pobox.com ; Microsoft maintains a complete specification for Caller ID--as well as an explanation of how to create your own Caller ID records--at http://www.microsoft.com/mscorp/twc/privacy/spam_callerid.mspx .) These two protocols are similar in some ways but are designed to solve different problems.
SPF addresses the fact that SMTP senders can easily lie about who they are. For example, I can send out spam with a return address of [email protected], and any bounces or nondelivery reports (NDRs) that my spam generates will go to Scott's mailbox. (When a spammer specifies an innocent bystander's return address specifically with the intent of flooding the victim, it's known as a "joe-job," but more often spammers simply pick a valid return address at random.) To use SPF, you publish a DNS text (TXT) record that contains information about the Internet hosts that are permitted to send email on behalf of your domains. For example, suppose I publish an SPF record for robichaux.net. Receiving SMTP servers can look up that record and use it to make decisions about incoming messages that claim to be from my domain. SPF lets the receiving Message Transfer Agent (MTA) make a four-way determination according to the following criteria:

- If no SPF record is available or the available record is bogus or malformed, the receiving server must rely on other methods to determine the message's validity.

- If the SPF record indicates that the domain is still in the process of moving to SPF use and the record doesn't include the sender's IP address, the message is probably legitimate.

- If the SPF record indicates that the domain has been completely transitioned to full SPF use but the record doesn't include the sender's IP address, the message is from a spammer.

- If the SPF record indicates that the sender's IP address is legitimate for the domain, the message is legitimate.

The ability to make these determinations is useful but has some limitations--primarily that SPF examines the sender's IP address only and so can't make decisions based on any other headers. SPF's other major limitation is that it uses a relatively constrained format that isn't easily extensible. Microsoft's Caller ID specification addresses both of these problems.
Like SPF, Caller ID requires administrators to create DNS TXT records, but each Caller ID record includes an XML-formatted description of the domain's email policy. Caller ID filters can make decisions according to a header's entire contents. This capability provides superior flexibility, at the cost of requiring the receiving server to accept the message before making a decision about whether to keep it.
The specification's use of XML has engendered a lot of debate among antispam activists because Caller ID records are more verbose than their SPF equivalents. Microsoft cites the broad availability of XML tools and parsers, and the ease of extending the Caller ID XML namespace to add new behaviors, as reasons behind using XML. The real utility of this approach comes when you consider the possible extensions to the Caller ID specification. Imagine the ability to succinctly define a policy that specifies that you'll accept email from certain addresses only when the messages are digitally signed or that you'll accept email from a specific domain only after the sender preregisters with your server. All sorts of interesting applications are possible.
Which standard will win in the marketplace? It's hard to tell. SPF has an early lead; almost 8000 domains, including AOL (which says its use of SPF is currently "experimental"), are using SPF. But Microsoft Hotmail will soon begin using Caller ID, so the total number of messages protected by that specification is likely to shoot upward in the not-too-distant future. There's another important point to consider, one that I first saw Larry Seltzer make in eWeek: "[Microsoft's] not selling SMTP authentication, [it's] selling products and services that are made better by SMTP authentication, and it behooves [Microsoft] to use the best method available." I think ultimately that SPF, Caller ID, and Yahoo!'s DomainKeys system will coalesce into one, or possibly two, standards that will then reach critical mass.
Caller ID and SPF deployment are both easy, and I'm curious about whether you're considering adopting either or both. Drop me a line and let me know what you plan to do.