Implementing DMARC: Adjusting SPF Records

It's been a couple of months since I last wrote about implementing DMARC and what comes next (review and adjustment). So I figured this would be a good time to document a few changes I've made based on the reporting data received so far.

Daily (or so) Reports

I'm glad I set up an alias account to receive the DMARC reports from vendors. The volume has been totally manageable across about a half-dozen valid, used domains (for sending), and some days there are no reports, while on others there are several.

What I've noticed thus far is that there are a couple of good stewards: Google and Yahoo. I have received reports from other providers, but mostly these two (and of them, predominantly Google). Generally speaking there's one report per domain per reporting party, but only based on activity, as expected.

The Report Contents

The lion's share of report data aligns with my expectations: good sends based on trusted configurations (DKIM and/or SPF). I have lightly ratcheted down a domain or two, whereas I have left some others more open at this point...because not everything that I expect to go through (and should be configured to do so) is being marked as passed. There have been several times when unauthorized sending takes place, and this is very obvious in the reports. Just the other day I examined a report with 34 failed records from an IP/source well outside the authorized list.

Some SPF Modifications

For some unknown reason (other than 'luck of the draw'), I have noticed a trend with one of my valid hosts not properly passing SPF. But not always. Once in a while a valid request matches SPF and is flagged as pass. In this particular case, SPF is the only option as DKIM cannot be implemented on the host, so before I ratchet down further restrictions this needs to be fixed consistently.

Ultimately I believe the problem is in the nested includes for the SPF record. It seems as though most of the time these inquiries are running up to the limit of 10 lookups rule, but the host/source IP can change and is something I cannot control. That being said, the SPF records I'm including are correct and do include the right ranges...they just seem to be returned 'too late' in the lookup chain...or somehow are exceeding the UDP packet size.

I've been using a tool from dmarcian for SPF Record Checks and recently decided to implement the 'experimental' (use at your own risk) "record flattening" rewrites. Since I understand what this is doing (condensing the records, but in so doing shifting the burden of maintenance to me versus the reference) and feel confident based on the IP lookups and references I've been using, I created a set of three SPF DNS TXT records based on the flattening suggestions. In theory this will fix the problem and will start to allow these valid messages to at least be marked SPF passed.

Early Results

So far things are promising, but in the first few days after making the aforementioned changes there was still a number of valid messages being flagged as failed, and of those not flagged as failed, even fewer making it through to their destination. The site in question runs a blog platform and the lion's share of these messages are merely moderation notifications for comments (because: comment spam). These messages are merely notifications to the site owner, and after the changes were made there was minor improvement in the deliverability of messages, but we actually discovered a larger problem: it was necessary to change both the administrator/notification address and actually implement an SMTP plugin on the platform.

Wait, What?

When we first changed the notification address, the problem became much more obvious. The "old" account/destination was outright rejecting messages based on message filtering (content/pattern matching) and not based on any DMARC implementation. This explained why very few were actually reaching their destination, even if they were passing SPF (and thus, DMARC). By switching notification addresses, we were able to better identify what was still awry and make some better changes. That process is what led us to implement an SMTP plugin for the blog platform to have more direct control over mail relay and default settings.

Ultimately, this relatively minor change has resulted in almost all outgoing messages to properly match our SPF settings, and as a result almost all messages are now properly passing both SPF and DMARC for that site/host/domain.

Remember: Review and Revise

In the example above, I managed to fix the problem with carefully curated SPF DNS records. Now that the SMTP components have been addressed, I'll keep monitoring the reports for this host for some time to ensure things are behaving...and if they do, will likely attempt to revert the record flattening business. It may work, it may not. Time will tell.

As noted several times before, it's important to periodically (or have ongoing) review of these reports to better do your part in addressing spam. These DMARC reports, while a bit clunky, can quickly identify possible issues that might be easy to fix...or point out other wonky or not-quite-right bits to fix!