Major e-mail providers, including Google, Microsoft, and Yahoo! are teaming up with PayPal, Facebook, LinkedIn, and more, to implement a new system for authenticating e-mail senders to try to prevent the sending of fradulent spam and phishing messages.

The protocol that powers e-mail, SMTP, dates back to a more trusting era; a time when the only people who sent you e-mails were people you wanted to send you e-mails. SMTP servers are willing to accept pretty much any e-mail destined for a mailbox they know about (which is, admittedly, an improvement on how things used to be, when they'd accept e-mails even for mailboxes they didn't know about), a fact which spammers and phishers exploit daily.

Making any fundamental changes to SMTP itself is nigh impossible; there are too many e-mail servers, and they all have to interoperate with each other, an insurmountable hurdle for any major change. So what we're left with is all manner of additional systems that are designed to give SMTP servers a bit more information about the person sending the e-mail, so that they can judge whether or not they really want to accept the message.

The two main systems in use today are called SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail). Both systems use DNS to publish extra information about the e-mail sender's domain. SPF tells the receiving server which outgoing servers are allowed to send mail for a given domain; if the receiving server receives mail from a server not on the list, it should assume that the mail is fraudulent. DKIM embeds a cryptographic signature to e-mail messages and an indication of which DNS entry to examine. The receiving server can then look up the DNS entry and use the data it finds to verify the signature.

These systems are not perfect; though both are used widely, they haven't been adopted universally. This means that some legitimate mail will arrive that doesn't have SPF or DKIM DNS entries, and so mail servers can't depend on its presence. Common legitimate operations can also break them; many mailing list programs add footers to messages, which will cause rejection by DKIM, and forwarding e-mails causes rejection by SPF. As a result, failing one or other test is not a good reason to reject a message.

These systems also make it hard to diagnose misconfigurations; receiving servers will typically just swallow or ignore mails sent by systems with bad SPF or DKIM configurations.

The large group of companies, which includes the biggest Web mail servers and some of the most common corporate victims of phishing attempts, is proposing a new scheme, DMARC ("Domain-based Message Authentication, Reporting & Conformance"), in an attempt to tackle these problems. DMARC fills some of the gaps in SPF and DKIM, making them more trustworthy.

DMARC is based on work done by PayPal in conjunction with Yahoo!, and later extended to Gmail. This initial work resulted in a substantial reduction in the number of PayPal phishing attempts seen by users of those mail providers, and DMARC is an attempt to extend that to more organizations. As with SPF and DKIM, DMARC depends on storing extra information about the sender in DNS. This information tells receiving mail servers how to handle messages that fail the SPF or DKIM tests, and how critical the two tests are. The sender can tell recipient servers to reject messages that fail SPF and DKIM outright, to quarantine them somehow (for example, putting them into a spam folder), or to accept the mail normally and send a report of the failure back to the sender.

In turn, this makes SPF and DKIM much safer for organizations to deploy. They can start with the "notification" mode, confident that no mail will be lost if they have made a mistake, and use the information learned to repair any errors. DMARC also allows recipients to know if a domain should be using SPF and DKIM in the first place.

Without a global rollout, DMARC can't solve all phishing and spam problems. The companies that have signed up to support the project include major recipients of phishing attempts—the various free e-mail providers—and sites against which phishing attacks are regularly made. Mail sent between the organizations will be verified using the SPF/DKIM/DMARC trifecta. Anyone using the major mail providers and the major services should see a substantial reduction in fraudulent mail. Senders and recipients who want to receive similar protection can implement DMARC themselves by following the specification that the DMARC group is working on.

Given the constraints imposed by SMTP, we may never get an e-mail system that is entirely free of malicious and annoying junk. SMTP e-mail was never designed to be trustworthy, and systems like SPF and DKIM are constrained by the inadequacies of SMTP's design. Nonetheless, mechanisms such as DMARC can still make a big difference, and with the support of these major companies, e-mail might get that little bit safer.

Listing image by Illustration by dmarc.org