tls-report | This repository contains tools to build the TLS | TLS library

After running some testcases, I have experienced the following:

(This was done by a Outlook.com smtp server.)

• MTA-STS: Deliberately incorrect, but existing third-party mx server in mta-sts file.
• DNS: Correct mx server.

The sender was informed about the delivery failure after 24 hours.

It was explained in my local language what was going on; here information highlights:

1. That the message could was not delivered.
2. That it was tried multiple times to deliver.
3. But that the cause was being unable to connect to the remote server.
4. Advise was given to contact the recipient by phone to ask the recipient to inform the postmaster about the error.
5. It was even suggested that the problem could most likely only be solved by the postmaster.
6. (A link was provided but that wasn't really helpful. Additionally the technical bounce message was visible among it the technical words "failed MTA-STS validation").

• MTA-STS: Correct and desired mx in mta-sts file.
• DNS: Deliberately set to incorrect mx server, existing server though.

After 24 hours I received an error back. Confusingly the message state that the address did not exists in the target domain. Though this is true, it shouldn't have gotten this far. However, when reviewing the technical part the outlook-sending server mentioned 'failed mta-sts errors validation'. So the technical part contained the correct mta-sts validation error, but the human/user readable part only mentioned that the target address did not exist in the target server.

I guess if the address doesn't exists, any mta-sts errors are "less important" to report to the end-user. The user was advised to re-type and resend the e-mail and verify if the address with the recipient (phone was mentioned). However, even if the user followed the instructions, the next e-mail wouldn't have been delivered either, but that is beyond this testcase.

• MTA-STS: Correct mx in mta-sts file.
• DNS: Fake MX corrects.

After 24 hours I received an error back. The cause for not being able to deliver the message was being unable to resolve the domain location of the recipient. (Undesired result, but logical, mx were referring to nothing.)

The technical part of the message mentioned 'DNS query failed'. Nothing of mta-sts was mentioned.

• MTA-STS: Correct mx in mta-sts file.
• DNS: Incorrect but existing mx records; a cname referring to the same IP of the correct mx server (which shouldn't matter because mta-sts should compare cert with cname.)

The results, unexpected:

• One email got delivered somewhere between that 24 time-window.
• One email failed due to mta-sts validation error.

Temporary downtime of webserver might have been a factor, though that shouldn't have mattered. - Cannot explain.

I took a while to find the correct testcase as you can see. But Testcase C describes the desired behaviour. Yes, the sender is informed, after 24 hours with outlook.com as smtp-server. The user is informed in clear language. That being said, I do have an additional opinion about the timing here, mentioned below.

Staying with the facts: I did not perform a testcase with a server trying unencrypted connections. Testcase C puts the ball into the the recipient's postmaster's court, I would be curious to see where the ball (the 'todo') would be placed, in the case of unencrypted attempts, as that cannot be solved by the recipient but must be solved by the sender or sender's postmaster.

I also did not test multiple smtp servers.

That being said, MTA-STS-validation needs to be supported by the sender SMTP (correct me in comments if I am wrong*), so if a server is so old it tries do deliver an e-mail over non-encrypted connection, it will most likely not support MTA-STS so it will not validate the MTA-STS policy and simply deliver the e-mail unprotected. * Found confirmation here, from paragraph "There is a standard...")

If somebody tries to redirect some incoming e-mail by dns-poisoning, a modern smtp-server will not deliver the e-mail to an incorrect destination. So it protects against evil doing, not against legacy.

I think the feedback delay of 24 hours is too long. Testcase C reports 11 retry attempts within that 24 hour window. Though I appreciate the system not giving up, I would argue that it might be in the interest of the sender to inform him of at least a non-regular delivery.