Have you ever put a metallic object into a microwave oven? You didn’t make that mistake twice, did you? Back when it was new, the microwave oven was something of a black box, but we learned a little bit about it and nowadays it’s a useful tool.
For something that revolutionized the world and the way we do business in it, the inner workings of email are still a black box to a lot of people who depend on its reliable delivery. Even on the most reliable platforms, the number of moving parts between your mailbox and that of your correspondents means the opportunities for things to go wrong en route are manifold. We hear a lot of complaints from people whose anxiety level could be dramatically reduced by a basic understanding of how email is delivered, so let’s take a quick look at how email is delivered, why some email is stopped, and what you should do when it happens to you – all in less time than it will take that potato to “bake” in the science oven.
Before we go any further, it will help to understand that the only reason email works at all is that the vast collection of servers passing it between each other on the internet enjoy absolutely no central governing authority but all conform by consensus to a set of rules (called “RFC’s”) that describe how email should flow. The first and best reason to employ professionals to configure your mail servers – or pay to use cloud-hosted ones – is that they won’t embarrass you when someone sees “this message might be a phishing email” under your name on an email, or it winds up in a spam folder because of something someone did wrong. If you’re lucky and your mail servers are misconfigured, your message won’t get delivered at all and your server will show you a non-delivery report. It’s worth noting that your recipient’s mail server should NEVER send a non-delivery report because senders’ addresses on spam are almost always falsified. In fact, if you see a non-delivery report claiming to be from your recipient’s mail server, contact ECHO sales and tell them your recipient’s IT department needs help.
Since misconfigurations are the hallmark of amateurs and spam-churning bot-nets, the vast majority of spam rejected by filters and properly configured servers gets rejected due to a misconfiguration. Spammers know this, so they find ways to hijack legitimate users’ accounts, and sometimes spammers infect the entire network leading to the compromise of the entire mail domain. Send enough spam into the world where users and automated content filters can see it and you’ll find your reputation trashed – sometimes yours, sometimes your domain’s, and sometimes the network from which you and your domain operate. A sullied reputation is the next most common reason for mail to wind up in a spam folder or be undeliverable. It’s for this reason we recommend that all of your organization’s mail leave for the internet by a spam filter that trusts you – and that will flag content before someone else does. In the process, this also helps your spam filter learn who your legitimate correspondents are, and this can keep their mail from being caught by your spam filter – no matter the content. Rare exceptions to this rule exist, but if you’re not sure this is how your email gets to the internet, you should investigate before you find out that you or your domain somehow got blacklisted somewhere.
Speaking of blacklists, there appears to be a great deal of confusion surrounding these tools and how they affect mail flow. Remember that the internet is a self-policing, highly distributed and collaborative effort. There are a myriad of blacklists running around on the internet that offer varying degrees of defense against unwanted email arriving from tainted networks, rogue servers and disreputable senders. Some blacklists are more reputable than others: we’ve seen blacklists embarrassed from time to time for blacklisting servers and whole networks that didn’t merit it. Trusting a blacklist that one day blocks Google, Microsoft, Intermedia or MX Logic is a pretty good way to get people to complain about their mail, and some blacklists are simply too prone to easily list domains without offering recourse when legitimate problems arise and are fixed. A good way to see whether your domain is on a blacklist is to visit http://mxtoolbox.com/blacklists.aspx which lists the most reputable and common ones. Speaking of MX Toolbox, their diagnostics page is a great way to check for common misconfigurations.
Whitelists are another popular area of concern for users with questions or complaints about mail flow. It should be noted that whitelists are implemented in different ways between mail systems. Some are going to result in even the most egregiously misconfigured servers being able to send mail into your mailbox or organization for a whitelisted sender or domain, while others are going to only result in letting minor misconfigurations through along with the bypassing of content-based filters. Your users should understand that their most important correspondents should be whitelisted, perhaps along with their entire domain, and they should understand the reasons for this. At the same time, they should understand that a whitelist is not necessarily a guarantee of delivery, and how to interpret non-delivery reports.
When email isn’t delivered and you see a non-delivery report, you need to understand how to interpret it. This usually means looking closely at the report itself to see who generated it. It should be generated by a server whose name is familiar to you as one that might conceivably try to deliver your mail. Again, if the server generating the report is one you’ve never heard of or one that might belong to your recipient, you’re looking at something that should almost never happen, since so much spam has a forged reply-to address. Your users should probably all be educated as to what a legitimate non-delivery report looks like and how to identify it as such. Send a message right now to a bogus recipient and look at the result if you’re not sure. When they see one, the first thing the IT department will usually ask for is to see the message headers, which they can often only do if the message is forwarded to the IT department as an attachment. The headers show each hop along the way to the failed attempt and can help show who dropped the ball – or the hot potato, as the case may be.
Finally, because we often get strange requests from people asking us to handle mail in strange ways so as to prevent users from seeing non-delivery reports or to re-write messages as they travel through the system, we need to emphasize that after years of evolution, email works the way it does for a reason. Tinkering with the messages your server forwards on to their destination by rewriting the headers or automatically responding to senders is in most cases a Really Bad Idea. Think carefully about what you’re trying to accomplish before you do this. If an executive assistant needs to be alerted to a non-delivery report in the executive’s inbox ASAP with a ticket generated on the help desk on the double – we can do that, but you probably don’t want us to do it by interfering with the way non-delivery reports work.
Email can be a bit of a black box, but like with other black boxes, it helps to understand a bit about how it works.