Dev Ups

Published 2022-10-11 in mail-server

Improving and measuring the credibility of an outgoing mail server

I wrote about the simplest configuration to have my exim4 emails accepted. "Simplest" meaning accepted by GMX and the sender not being on their block list.

I wrote again about satisfying gmail's concerns, which were:

  1. disabling IPv6
  2. Enabling SPF

I continue looking at improving the reputation of sender and emails. Reputation and trust must be earned, but let's maximise credibility from the start.

All this post adds is DKIM.

Outlook

Outlook says I'm on a blocklist:

  SMTP|> MAIL FROM:<<localuser>@mailervps.silverbullets.co.uk> SIZE=1424
  SMTP|> RCPT TO:<realmailbox@outlook.com>
         will write message using CHUNKING
  SMTP+> BDAT 401 LAST
  SMTP>> QUIT
  SMTP<< 550 5.7.1 Unfortunately, messages from [54.38.0.0] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3140). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [BN8NAM12FT030.eop-nam12.prod.protection.outlook.com]
LOG: MAIN
  H=outlook-com.olc.protection.outlook.com [104.47.55.161] TLS error on connection (recv): The TLS connection was non-properly terminated.
  SMTP(closed)<<
LOG: MAIN
  H=outlook-com.olc.protection.outlook.com [104.47.55.161] TLS error on connection (recv): The specified session has been invalidated for some reason.
  SMTP(closed)<<
LOG: MAIN
  H=outlook-com.olc.protection.outlook.com [104.47.55.161] TLS error on connection (recv): The specified session has been invalidated for some reason.
  SMTP(close)>>
LOG: MAIN
  ** realmailbox@outlook.com R=dnslookup T=remote_smtp H=outlook-com.olc.protection.outlook.com [104.47.55.161] X=TLS1.2:ECDHE_SECP384R1__RSA_SHA256__AES_256_GCM:256 CV=yes DN="C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,CN=mail.protection.outlook.com": SMTP error from remote mail server after pipelined end of data: 550 5.7.1 Unfortunately, messages from [54.38.0.0] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3140). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [BN8NAM12FT030.eop-nam12.prod.protection.outlook.com]
LOG: MAIN

TLS

In the preceding article about having Gmail accept our emails, I demonstrated that our TLS (or lack of it) is already sufficient. I don't find any immediate benefit to upgrading the TLS herein.

You can watch the TLS protocol actions in detail for each email, if you are using the Postfix MTA, and tweak verbosity.

TLS problem

Both Gmail (even when successful) and Outlook are reporting these TLS error on connection (recv):

  • The TLS connection was non-properly terminated.
  • The specified session has been invalidated for some reason.

I tried sending the email as root to ensure this wasn't a permission problem accessing the TLS certs, but the problem remained.

SMTP>> STARTTLS and SMTP(TLS shutdown)>> proceed correctly for GMX. Perhaps "TLS" was intended more as an adjective to "connection" rather than the TLS itself being incorrect. That the connection was pre-empted, I can believe.

I found the TLS certificate locations for split configuration (which I must keep reminding myself I'm not using):

:~$ cat /etc/exim4/conf.d/main/01_exim4-config_listmacrosdefs | grep CONFDIR -n
26:.ifndef CONFDIR
27:CONFDIR = /etc/exim4
70:# defaulting to mail_spool. See CONFDIR/conf.d/transport/ for possibilities
:~$ sudo cat /etc/exim4/conf.d/main/03_exim4-config_tlsoptions  | grep CONFDIR -n
22:#                          CONFDIR/exim.crt if unset
24:#                          CONFDIR/exim.key if unset
32:MAIN_TLS_CERTIFICATE = CONFDIR/exim.crt
37:MAIN_TLS_PRIVATEKEY = CONFDIR/exim.key
# The next line changes the cert to point to where certbot is already managing them.
# :~$ sed "s|^MAIN_TLS_CERTIFICATE = CONFDIR/exim.crt|MAIN_TLS_CERTIFICATE = /etc/letsencrypt/live/mailervps.silverbullets.co.uk/cert.pem|" /etc/exim4/conf.d/main/03_exim4-config_tlsoptions  | grep /etc/ssl/ -n
# That's for split configuration, in my case the mod is to /etc/exim4/exim4.conf.template
# Repeat for the PRIVATEKEY.

The commented code at the bottom shows an attempt to change the certificate and key locations to that of the HTTPS server. I don't know how well this will work, its work the future.

If that code isn't evidence enough, we can see in exim's log:

Warning: No server certificate defined; will use a selfsigned one.

Suggested action: either install a certificate or change tls_advertise_hosts option

Is certification mutual?

It is the mail-server that has a TLS certificate. Since we are sending (not relaying, not receiving) emails here, we are acting as client. exim4 is technically a server, and we might like to enjoy it being able to accept email, encrypted in flight, which TLS gives us.

Immediately after the first paragraph of its Introduction, about in-flight encryption, RFC 3207, the RFC for Secure SMTP over TLS, adds (emphasis mine):

authenticate each others' identities. For example, a secure SMTP server might only allow communications from other SMTP agents it knows,

SMTP agent huh? That puts the A in MTA and sounds like the server-client duality I've been describing thus far.

Undeniably; we are still block-listed by Outlook, and we haven't set all the DNS records we could and should.

Testing TLS

swaks is the tool for testing SMTP. I had to install it via apt-get (including recommends) then I could run a simple test:

swaks --to realmailbox@gmx.com

That mail got delivered, so be careful not to spam anyone. I trust swaks to not share that email address, but please make your own checks.

Next test is adding the -tls flag, defined as:

Require connection to use STARTTLS. Exit if TLS not available for any reason (not advertised, negotiations failed, etc).

swaks -tls --to realmailbox@gmx.com

That succeeded, but to quote RFC 3207, "SMTP server might only allow communication from other SMTP agents it knows".

When I repeated for gmail.com I was presented with IPv6AuthError, despite having fixed that problem when sendmail is the command issued to send the email.

When I ran swaks command against an Outlook address it said I was on a block list.

Repeated for yahoo.co.uk, it made no complaints, but then sendmail to yahoo wasn't flagging TLS error on connection (recv) to start with. swaks mail went straight to spam.

With swaks, all mention of TLS connection was non-properly terminated is gone. This suggests to me that those messages came from the sendmail/exim4 software, and are probably referring to some brusque disconnection by a busy, or discreet, mail server. I think it's safe to proceed without giving exim our LetsEncrypt certificate and key.

swaks struggled to run from my dev laptop. Most mail servers reject IP addresses reserved for personal use as potential source of spam.

SPF and DKIM

This pair of records were identified as deficient before I moved my testing to a dedicated VPS. The feedback associated the lack of SPF and DKIM with the email being spam, and blocked.

I haven't seen these specific messages again on the currently dedicated VPS.

It is accepted wisdom that SPF and DKIM are boons to authenticating emails and passing spam filters.

SPF

If you followed my previous post about getting accepted by Gmail you're probably all set and could skip this SPF section. If you want to learn more, this is a little more in depth; it wasn't exactly difficult.

There's a 255 character limit to this TXT record. For me sending from one IPv4 address, this would take the form:

v=spf1 ip4:1.2.3.4 -all

My DNS provider gave me the option to create a SPF record instead of a TXT one. SPF is given 600 TTL. Here is the relevant text from my DNS zone:

mailervps                  600 IN TXT    "v=spf1 ip4:54.38.0.0 -all"
mailervps                      IN TXT    "v=spf1 ip4:54.38.0.0 -all"

I deleted the ordinary TXT record; having multiple SPF records is an error.

I checked it at dmarcanalyzer.com and it was ok. Gmail has Postmaster Tools to help get mail accepted with only an SPF record, which we've already done. Maybe it can tell us how to dodge its spam filters...

I can check locally too:

~$ dig mailervps.silverbullets.co.uk TXT | grep "ANSWER SECTION" -A2
;; ANSWER SECTION:
mailervps.silverbullets.co.uk. 600  IN  TXT "v=spf1 ip4:54.38.0.0 -all"

Note that none of this security has extended to the parent domain:

~$ dig silverbullets.co.uk TXT | grep "ANSWER SECTION" -A3
;; ANSWER SECTION:
silverbullets.co.uk.    3600    IN  TXT "google-site-verification=ZEAAjR7ofIigS1jteowuFGxFMR42noEC3kTRBEerEqI"
silverbullets.co.uk.    3600    IN  TXT "1|www.silverbullets.co.uk"

I tested sending from the parent domain. It worked, at least for Gmail. I find this surprising. We couldn't send to Gmail until we had an SPF record for the sender domain. Now we have that, Gmail lets us send from a different IP address, and we are using the parent of the domain that the SPF applies to. That's just my observation, it feels like a double standard, but I lack the resources (VPSes and domain names) to investigate further.

DKIM

Generate key and DKIM record

Using 2048 bits should be ok. 1024 is faster and takes less space in the DNS records, but is considered insecure:

cd /etc/exim4
openssl genrsa -out dkim-key.pem 2048
openssl rsa -in dkim-key.pem -pubout

The output is the public key, copy the base64 into the DKIM record, a TXT record, and paste into your DNS zone:

v=DKIM1; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtowO79V4hS3FQ4SM1DrsYIrItb/0TlobBmyUT4NkTcx7hie48lRcVhtve2SopaTXrfujs1QEqcy/y6JuhdP30gPx8Z2CEps4QiVRz0Ug42nOhzMqC8vWJPm8rW4knVKdsVZ0W9BjylxPHWZVJ46JyaeTisH6OVY5TvyBpVOM1LGUdNkIfSyuTL3YyZsl/S+PMxFIgJlP+SkOuPbZVQ6IwWD0fLiY0G37N55aXSSylpZOSorCqSZd8a/oL2IDh3Skt3n1bTiqdarHVvzC2oeK4zY9kMzlOsm3UCLbcnS/81gkyBXuANFbfGE6FJsRl7bXucIMTt6i7Ptusid1+vTgnQIDAQAB;t=s;

Base 64 is letters, numbers, +, and /. Fields are separated by semi-colons. t=s prevents the signature applying to subdomains.

exim.org allows for Ed25519 keys, which are much shorter, and newer, but less widely, and optimally, supported:

with GnuTLS 3.6.0 or OpenSSL 1.1.1 or later, be a valid Ed25519 private key

Update exim4 config

Locate "remote_smtp:" within exim's configuration files:

sudo grep -C3 -r "remote_smtp:" /etc/exim4/

I find it in two locations. /etc/exim4/exim4.conf.template is for if I was using that single file configuration, which I am not. /etc/exim4/conf.d/transport/30_exim4-config_remote_smtp, is for split configuration users, where it sits right at the top.

I replaced my:

remote_smtp:
  debug_print = "T: remote_smtp for $local_part@$domain"
  driver = smtp

with:

remote_smtp:
  debug_print = "T: remote_smtp for $local_part@$domain"
  driver = smtp
  dkim_domain = mailervps.silverbullets.co.uk
  dkim_private_key = /etc/exim4/dkim-key.pem
  dkim_canon = relaxed
  dkim_selector = x

Search RFC 4871 for the "canon". You have "simple", which is strict, or "relaxed".

I don't need to specify the subdomain, mailervps, even when sending from a VPS with an A record corresponding to the subdomain. I can have multiple DKIM records, one for each subdomain, and avoid needing to share keys.

Automate exim4 config

To automate this change we should choose any of 3 unappealing options:

1:

sed -i '/^remote_smtp:/{n;n;a\  dkim_domain = mailervps.silverbullets.co.uk\n  dkim_private_key = /etc/exim4/dkim-key.pem\n  dkim_canon = relaxed\n  dkim_selector = x 
}' /etc/exim4/exim4.conf.template

2:

sed -i '\|^remote_smtp:|{n;n;s|$|\n  dkim_domain = mailervps.silverbullets.co.uk\n  dkim_private_key = /etc/exim4/dkim-key.pem\n  dkim_canon = relaxed\n  dkim_selector = x|}' /etc/exim4/exim4.conf.template

3:

line_num=$(sed -n '/remote_smtp:/=' /etc/exim4/exim4.conf.template)
sed -i "$(($line_num + 3))i\  dkim_domain = mailervps.silverbullets.co.uk" /etc/exim4/exim4.conf.template
sed -i "$(($line_num + 3))i\  dkim_private_key = /etc/exim4/dkim-key.pem" /etc/exim4/exim4.conf.template
sed -i "$(($line_num + 3))i\  dkim_canon = relaxed" /etc/exim4/exim4.conf.template
sed -i "$(($line_num + 3))i\  dkim_selector = x" /etc/exim4/exim4.conf.template

I prefer the third since it's the least cryptic. The others still have uses; they contain different special characters.

Apply, and check (update-exim4.conf already does its checks, but belt and braces) our changes with:

sudo update-exim4.conf
exim -bP transports | grep dkim 

Test DKIM

I start testing with these two dedicated services, or better yet, email my own Gmail account.

Test changes

mail-tester.com provides an email address to send to, max 3 times per day for free. I've redacted some characters for privacy, the characters are all within the range [a-z0-9]:

mail tester landing page

Connecting to reception.mail-tester.com [94.23.206.89]:25 ...  non-TFO mode connection attempt to 94.23.206.89, 0 data
 connected
  SMTP<< 220 mail-tester.com ESMTP Postfix (Ubuntu)
  SMTP>> EHLO mailervps.silverbullets.co.uk
  SMTP<< 250-mail-tester.com
         250-PIPELINING
         250-SIZE 10240000
         250-VRFY
         250-ETRN
         250-STARTTLS
         250-ENHANCEDSTATUSCODES
         250 8BITMIME
  SMTP>> STARTTLS
  SMTP<< 220 2.0.0 Ready to start TLS
  SMTP>> EHLO mailervps.silverbullets.co.uk
  SMTP<< 250-mail-tester.com
         250-PIPELINING
         250-SIZE 10240000
         250-VRFY
         250-ETRN
         250-ENHANCEDSTATUSCODES
         250 8BITMIME
  SMTP|> MAIL FROM:<<localuser>@mailervps.silverbullets.co.uk> SIZE=1461
  SMTP|> RCPT TO:<test-rq8o9@srv1.mail-tester.com>
  SMTP>> DATA
  SMTP<< 250 2.1.0 Ok
  SMTP<< 250 2.1.5 Ok
  SMTP<< 354 End data with <CR><LF>.<CR><LF>
  SMTP>> writing message and terminating "."
  SMTP>> QUIT
  SMTP<< 250 2.0.0 Ok: queued as B0276A993E
  SMTP<< 221 2.0.0 Bye
  SMTP(close)>>
LOG: MAIN
  => test-rq8o9@srv1.mail-tester.com R=dnslookup T=remote_smtp H=reception.mail-tester.com [94.23.206.89] X=TLS1.2:ECDHE_SECP256R1__RSA_SHA256__AES_256_GCM:256 CV=yes DN="CN=mail-tester.com" C="250 2.0.0 Ok: queued as B0276A993E"
LOG: MAIN
  Completed

"Check your score" appends those unique characters to the URL and gives a score out of 10 which is based on 5 criteria. One of which says I'm not on any blacklists.

Before completing DKIM configuration I got 7.8, -1 for missing DKIM. The other -1.2 was from SpamAssassin, "Missing To: header".

I correct the DKIM (already correct above), and add a "To:" header:

echo -e "From: you@mailervps.silverbullets.co.uk\nTo: test-rq8o9@srv1.mail-tester.com\nSubject: Scoring Points\nThis is very important and definitely not spam.\n" | sendmail -v test-rq8o9@srv1.mail-tester.com

mail-tester is happy now. The same email sent to Gmail, taking care to reflect that in the "To:" header, is still flagged as spam.

Outlook - road to freedom

Concerning:

Please contact your Internet service provider since part of their network is on our block list (S3140)

I completed Outlook's form with just the bare details required: both IP addresses, the sending domain name, and website URL. Other boxes filled were:

  1. Is your server dedicated or shared? *
  2. Copy and paste any error messages: *

That first one threw me. I know I don't have a physical machine dedicated, but I think they mean, "do I share keys with anyone?". The second box I thought would be all the information they could need; I may even have confused them by including the email headers.

They summarily dismissed my entreaty with: "Not qualified for mitigation". I was invited to submit the error in detail, like I hadn't just copied and pasted that (albeit with potentially "noisy" headers).

One helpful link in the reply was Smart Network Data Services program (SNDS), "to monitor the 'health' and reputation of your registered IPs". The next step was to, "request access", to each IP where they, "determined that the following email addresses are associated with the specified network in an appropriately authoritative way".

This included 2 email addresses on my domain, plus 2 more at colocrossing.com. [I'm jumping back (on the 7th Nov 22) in time to back this update because receiving email isn't to be rushed into]. The instructions for completing the sign up were immediately sent to the address I chose. postmaster is an alias for root and required according to RFC 5321 Section 4.5.1 Minimum Implementation.

Verified, I could, "View IP status". It says all IPs have normal status. I could "view data". There was no data. They won't accept my emails.

Outlook's own literature quantifies patience requirements:

A new IP can expect to be fully ramped within a couple of weeks or sooner, depending on volume and list accuracy, and as long as their junk email complaint rates are kept at a minimum.

Next: Receiving email

It will be good to understand the perspective big email providers have when it comes to accepting emails. This insulates my personal email from this domain. A contact me form leveraging a receiving-only MTA only works until I need to reply.

I sent emails to you@mailervps.silverbullets.co.uk and <localuser>@mailervps.silverbullets.co.uk. I heard nothing until four days later when an automatic "mail delivery software" replied that both addresses failed. Exim's configuration made perfectly clear I had not elected to support incoming emails.

Configuring exim4, or any MTA, to receive, safely, will require more work. Fail2Ban filters for port 25/587, failregex are each 5, long, impenetrable lines.

Next: DMARC

DMARC requires publishing an email address. I'll need a throwaway email address once it gets published, but what's one more set of credentials at this point?

The example DMARC record, which would be a publicly available TXT record in my DNS zone:

"v=DMARC1;p=reject;pct=100;rua=mailto:<<observer_at_another_domain.com>>"

It allows the owner of the record to see things from the perspective of a receiving MTA; validation of DKIM and SPF. This helps the domain owner identify spoofing attempts, and to stem potential reputational damage.

The p=reject part of the record signals that failing emails should be rejected. I already know Gmail will reject if SPF validation fails, but I think I'd like GMX to reject mail if SPF fails. Anyway, it's only a suggestion, so no guarantee they'd oblige.