I'm assisting someone with their small business's Google Apps account. They're having a big problem sending and receiving emails...in that sometimes they aren't doing either. It's not all their email, just a portion of it, but sometimes they don't receive the email that other people have sent them, and sometimes other people don't receive the email my client sends out.

After looking it all over, I found that an SPF record in their domain registrar/web hosts DNS settings wasn't including Google, and had another typo. Since then I've switched what was providing DNS altogether and things seem to have improved, but it's not clear yet.

My question is this: what can I do for them going forward? They're naturally scared that they've already missed some business and that other people think they haven't been as prompt in their replies as they actually have been. They also appear to be getting spam messages from old accounts in their systems that don't even exist anymore, so I assume that someone is spoofing some of their addresses.

Do I look into DMARC? I've only come across that recently and I'm not sure I entirely understand how it works.

I'm not just asking for them, either. I've recently discovered that some of my outbound emails were not delivered because my domain was marked my my web host for spamming. Of course, this happened without any notice to me, no bounce back notifications, and no way for me to know which emails did not get sent. This disturbs me greatly, as I'm worried that I've lost business over this. I'm considering moving all my email handling away from my web host (Bluehost - actually my client's web host too) to someone else. As it is, I'm just using GMail to check a POP3 account on their servers, but now I'm thinking of going with Google Apps or Office 365, but I have no idea if that will help anything.

Anyway, sorry for the rant. I'm just frustrated. I get that spam is a problem, but I can't imagine that my output ranks with even the lowest level spammer, and the content certainly doesn't. In the end, I would be far less upset if I had known what was happening, but there was absolutely no indication, and that worries me greatly.

Do you have any access to the mail server logs, or the headers in the messages that have issues?

Beyond SPF, do you also have DKIM setup?

Have you checked your domain and your customers domain against any of the spam blacklist services?

I'm slowly plotting out my own mail server migration soon, and will likely look to research and implement DMARC when the time comes. Currently I have no experience with it, but see it rising up in importance.

I don't have access to server logs.

What do you think the headers of the outbound emails with issues would tell me?

I don't have DKIM set up. I thought that was part of DMARC.

Where can I find these blacklist services?

DMARC builds on both DKIM and SPF. This white paper explains them in a bit more detail:
http://www.mcafee.com/us/resources/solution-briefs/sb-spf-dkim-dmarc-demystified.pdf

Quick summary, SPF basically puts in DNS records that other mail servers use to verify that mail from your host is coming from the right IP. DKIM goes further by adding some signing to the outbound e-mails via public/private encryption schemes. The public key is stored in DNS as a TXT record. It was a bit more involved to set up, but it seems to have helped my domain a lot. Fixing my broken SPF setup and adding DKIM put a stop to GMail rejecting some messages from my domain. DKIM helps prevent some possible spoofing attacks that SPF wouldn't.

Looks like I ended up using this service to verify my DKIM setup, and their page shows some useful tidbits to look for in e-mail headers: http://mail.appmaildev.com/en/dkim/

I was emailing in and out of my domain to my iCloud, GMail, and Yahoo mail accounts to see the headers both ways during the DKIM setup.

Blacklist wise, it looks like this site will search a bit over 100 different mail blacklists to see if your domain or IP is on any of them: http://mxtoolbox.com/blacklists.aspx

Much of the silent failures without bounce backs are using the mail blacklists to drop mail. The idea is the SMTP server rejects the connection before the message even fully comes through. CPU usage wise, it's minor as it's several DNS lookups that can easily be cached too. It's far lower impact with much higher benefits then full on accepting the message, scanning it, and generating a bounce back response.

Woah, thanks for all of that, Tom. I'm going to need to take a while to look over it all. Do you know of a good primer on what should be in an SPF record?

For now, the simplest solution I've found is, as you might have guessed, to let Google handle it all. My client was already on Google Apps and had their domain registered through Google Domains, but their nameservers were pointed to Bluehost. I removed DNS handling from Bluehost like a child that couldn't be trusted with something delicate, and gave control to Google. So far, that seems to be really helping. I think I might end up doing the same thing for my domain and finally sign myself up for Google Apps. I've just been lazy about it and not looking forward to having two accounts that I have to check on my desktop.

I still very much want to add the other levels of assurance, though, so I'll be doing some research on the links you provided. Thanks so much.


As for my own domain, it looks like it passed on all lists, according to that site (I've used MXToolBox.com before - great site). I know for sure that some of the content from my domain is being blocked by some providers, though. For example, every single invoice I send through Freshbooks is undelivered by AOL. I've added Freshbooks to my SPF records, but they still get blocked. I can send regular emails to those same people, though, so I usually end up sending them PDFs of my invoices. Unfortunately, it doesn't look like they support DKIM.

SPF Wise, this helped me fix and build out mine: http://www.openspf.org/SPF_Record_Syntax

GMail ended up flagging my domain a bit due to two factors: the report as spam emails from the board to moderators with GMail addresses and those notices going out over IPv6. At the time my SPF records only validated for the IPv4 address of the server.

Once I added IPv6 AAAA records for my server, these two SPF records made the most sense for my setup:

Name:@ Type:TXT "v=spf1 mx mx:miniinfo.net -all"
Name:mail Type:TXT "v=spf1 a -all"

This means that if the root of the domain (miniinfo.net) record is accessed, it says anything defined as an MX record for my domain is also valid for sending mail from my domain. The MX record resolves to mail.miniinfo.net. The second spf record ensures that anything with an A record for mail.miniinfo.net is valid. It was slightly unclear if that meant it would also check AAAA records, so currently I have an ip6: entry in both records. I'll look to simplify it during a mail migration later this year.

The Freshbooks issue is understandably frustrating, since you likely can't get the server logs from their mail server that would help indicate why AOL is dropping them. Does your own SPF record contain this exactly?

I'll reach out in PM, because I think I spotted an issue with your record.

Just to follow up on this issue: my method to resolve all of this was to finally sign up for a Google Apps account and to transfer all my domains to Google Domains. I realize how typical it is for me to run to Google for everything, but it's the best solution for me. There's good synergy here because in Google Domains it takes about three clicks to set up a dozen DNS entries for Google Apps. It even set up a DKIM entry, which should help fix some of my initial problems.

I just wanted to thank Tom, who went to extraordinary lengths to help me with this issue. He even looked through my DNS entries personally to see where things might have been going wrong, and gave me advice for how to fix them. Thanks for all the help, Tom. You definitely helped my understanding of these issues, and I'll be looking through the resources you sent me in order to learn some new stuff.