Earlier this week, one of our customers emailed us to ask why a spear phishing simulation tool they were using was able to send email to their users, despite them having set up SPF (sender policy framework) properly. It’s a great question, and while we’ve described what SPF is before, many of the organizations we speak with do not understand why it fails to provide adequate protection for stopping spear phishing attacks.
In this post, we’ll take a look at some of the more insidious problems around email authentication, and why stopping spear phishing attacks requires more comprehensive security than existing specs can provide — and we’ll even step back to the 1930s and look at why, through the lens of a certain European logician and mathematician.
HELO, Who’s There?
Email fraud is a major problem, in part because email is an incredibly complex set of systems that span the entire internet. It’s fairly impressive that any computer can, essentially in realtime, send a human-readable message to anyone in the world; unfortunately, the technological stack that enables this kind of communication to function was not built upon a foundation of trust.
In other words, it’s easy to manipulate email, and use it for nefarious purposes: credential theft, financial fraud, and even real-world crime. Last year, in fact, a study by a major antivirus company found that the odds that a major terrorist attack on the United States had a 9 in 10 chance of starting with an email-based deception — a spear phishing attack.
I Thought We Were Covered?
A common misconception we encounter when working with our clients is that they can implement certain technologies — SPF, DKIM, DMARC, and so on — to prevent these kinds of attacks from succeeding.
In theory, this should be how it works. The idea behind the SPF framework, for example, is that, “a domain may explicitly authorize the hosts that are allowed to use its domain name, and a receiving host may check such authorization.” (Via the official SPF RFC, here.)
Sadly, in practice, it’s possible to send a spoofed email to a user in such a way as cause SPF to pass, but still be entirely spoofed from the perspective of the end-user. And DKIM, DMARC, and even newer technologies like ARC are either so poorly adopted as to be essentially useless today, or are entirely untested.
Take this (slightly redacted) example, sent to a user inside of our domain:
Notice, in the red box, how this email has been authenticated via SPF. Every email provider in the world would accept this message, and display it by default. The only problem?
This is a completely spoofed message. It was sent to our user from a script that rewrote the email to look as though it was from someone they could trust (the @gmail.com address, which we’ve redacted for privacy purposes).
This underscores one of the fundamental issues with email security and authentication.
Problem 1: Authentication Is Usually Mis-Configured
One of the primary problems with email authentication technologies like SPF (which is the =pass line in the example above) are that they require work to implement properly, and then need to be maintained whenever an organization’s mail servers are updated or changed.
We see so many broken implementations of SPF in the wild when working with our customers. In a perfect world, every domain would have SPF, DKIM, and DMARC set up and running well, and it would be nearly impossible to execute a spoofing attack. In our reality, however, this is an impossible hurdle.
For example, one of the companies we work closely with had a good SPF implementation – but it was a few years out of date. They were early adopters of the spec, but whomever set up their original DNS entries for SPF either forgot to update them when they switched from a hosted Exchange server to a Google Apps environment, or was no longer employed there.
The net effect? Every message they sent was marked as a softfail — meaning that there was no technical way to differentiate a spoofed message from a legitimate one when sent from their domain.
Problem 2: The Burden Is On The Wrong People
On the receiving side of the exchange, this kind of inconsistent deployment means that it’s nearly impossible to use SPF for its intended purpose of determining whether to deliver a message or not. As a result, the default for every email provider is to allow these kinds of softfail results through – doing otherwise would likely mean that a significant number of otherwise reasonable emails would be “lost”, with the obvious cost being to their end users.
Of course, most people who receive email simply read it, respond to it, and file it away. The assumption – especially if they are using a major provider like Google or Microsoft – is that their mail is generally safe. Foundational controls for malware and spam are nearly ubiquitous today, and default protections (for example, Google’s in-email warning banners) are highly visible:
The major problem here is that these protections are, at best, imprecise. Not to pick on Google, as they do a very good job at providing baseline security against these kinds of attacks, but banners and spam filters typically work based on keyword detection. Savvy attackers know this, and know how to bypass these filters.
Recently, Google and Yahoo both announced that they would be transitioning to a more strict DMARC policy (p=reject). We celebrate the improved authentication that this will provide — and are excited to see ARC gaining traction, as well — but even with these policies in effect, when strong authentication goes up against deliverability, deliverability will win.
Problem 3: Technology Is Not The Same As Intelligence
Imagine a world where, miraculously, all spoofing attacks become impossible. Perhaps we’re wrong — although it’s a hard argument to make — and every domain in the world will implement some form of DNS-level validation within the next decade.
The complexity of the global email network means that there remain significant and highly successful attacks — remailing attacks, homograph attacks, and direct account compromises — where, even if we had a perfect authentication system, it would be possible for attackers to trick people into taking action based on mail that was inherently untrustworthy.
So, if email authentication can’t help, what can?
Abusing Gödel: The Trust Incompleteness Theorem
The answer here is one of first principles. What is missing in email exchanges – and, while not the topic of this article,other forms of cloud-based communication – is trust.
Trust must be derived from outside of the systems used to exchange email if it is to be believable.
Allow us to step, if briefly, away from email security to explain why. In the early part of the 20th century, Kurt Gödel, an Austrian mathematician, noted that (simplifying somewhat) within the realm of mathematics, no system can be both consistent (that is, free from contradictions) and complete (that is, if in respect to any given property, every formula having that property can be derived using that system).
While Gödel was writing about mathematical systems, his so-called incompleteness theorem has wide-ranging implications. Among them is the concept that for any given system — even something as comparatively trivial as sending validated messages over the Internet — there will be certain truths which cannot be proven within that system.
Trust is an example of this.
The ability to manipulate the content and headers of a message, or to impersonate a trusted individual by an attackers are the primary reasons why email security is in such a sorry state today. While increasing the complexity of these systems may deter more basic attacks – and this is very much what we expect to see as Google, Microsoft, and other email providers improve the default security they offer – trust about these communication systems must come from an external source.
Trust at the Moment of Interaction
And so, we come to what we’re building with GreatHorn.
While natively integrated with the major email providers, our ability to analyze messages, identify indicators of deception, and take automated action without requiring a decision from a recipient is based on the belief that this kind of trust must come from outside of the email exchange itself.
Take as one example our example from the start of this post:
Where an email system would rely on SPF (and the other headers) to determine if this was a spoofing attempt, and would likely get it wrong, GreatHorn has an external data set it can use to determine if this is a trustworthy message. In this case, that analysis can extend far beyond what is evident in this single message, or even across the total set of messages available to the recipient domain (greathorn.com, in this example).
Moreover, GreatHorn can look at more data than is even available to the platform provider, Google — because with over 50,000,000 new emails every quarter to analyze, across every major email vendor, the data set available in full realtime is entirely unparalleled.
And, in this case, we can immediately identify that this is a spoofing attempt despite the supposedly valid SPF record!
Given the massive financial cost for a data breach or financial fraud attack, this kind of external security is essential. Want to see if you’re at risk? We’re happy to help, and can deploy GreatHorn in under 20 minutes.