[exim] Re: tracing SMTP conversations

Top Page
Delete this message
Reply to this message
Author: Bill Cole
Date:  
To: graeme vetterlein via Exim-users
Subject: [exim] Re: tracing SMTP conversations
On 2024-02-23 at 21:12:12 UTC-0500 (Sat, 24 Feb 2024 02:12:12 +0000)
graeme vetterlein via Exim-users <graeme.exim4@???>
is rumored to have said:

> Starting in mid February 2024, gmail tightened up its policies and I
> for one
> started getting bounced email.
>
> After about 4 lots of 12 hour days I figured out I was not setting
> errors_to= in my userforward
> so my "RCPT TO" envelope headerwas wrong/missing.'


The argument to RCPT TO is not a header. A "header" in email is a
logical line of text before the first blank line of the message with a
header name followed by ': ' and the header content.

The argument to RCPT TO is simply an email address. The argument to MAIL
FROM is an email address. Neither one is a header. Either may OR MAY NOT
match a message header's content.


> Having fixed that, I now started getting failures sent to my
> postmaster (who I'd pointed errors_to at)
>
> ------------- cut here------------
> could not be delivered to one or more of its recipients. The following
> address(es) failed:
>
>   auser@???
>     (generated from auser.home@???)
>     host smtp.forwardemail.net
> [2001:19f0:ac01:d3c:5400:4ff:fe72:47c4]
>     SMTP error from remote mail server after end of data:
>     550 From header must end with @mydomain.com
> ------------- cut here------------
>
>
> The phrase "From header" is suitably vague,


It is not in any way vague.

> do they mean the "MAIL FROM"  envelope header


Again for emphasis: that is NOT a header.

> or the From header in the mail body headers ?


That's the only place a header exists.

Another clue is that this was "after the end of data" which means that
the server already gave a positive reply to MAIL FROM and saved this
error until it has actually seen the headers.

> This only seems to effect mails that have been forwarded via a a users
> ~/.forward.


Because that does nothing to modify the headers.

>
> What I REALLY REALLY would like to do is watch the SMTP conversation
> between my exim4 and its smarterhost
>
> (it's probably something I'm sending, so I want as much to see *my*
> side of the conversation)


I strongly doubt this is a technical problem in what Exim is doing in
SMTP. This reads like a policy issue: your account is only allowed to
send mail with a From header using your own domain.

Talk to the people who run forwardemail.net. Or read their docs. Or
change your account config with them. In any case, this is THEIR
POLICY.

> I can see over many years people have been asking to do this and been
> told to use wireshark....which I eventually
> did. However , in the past this was fine but nowadays it does a
> STARTTLS and I can see no more.
>
> Is there really no way I can see the SMTP conversation? If I could
> just see what I sent it before said "error"
> I feel I could save another block of long nights.


It sent the message, after getting a 'go ahead' response to DATA. It
ended the message with <CRLF>.<CRLF>, which is referred to in the error
message as "end of data"

The underlying cause here is that traditional transparent forwarding
(using .forward or /etc/aliases) does absolutely nothing to the message
data, but simply relays the message using the same envelope addresses
that the message arrived with. This has become increasingly incompatible
with the efforts to reduce spam by enhancing the authenticability of
messages. The first step was SPF, which allows domain owners to publish
a limited set of IP addresses that are authorized to use the domain in
MAIL FROM and how to interpret violations of that limit. SPF breaks
traditional forwarding. It has always been known to do so. There is a
mechanism for forwarding MTAs called SRS which was developed with SPF to
allow MTAs to rewrite envelope senders to the MTA's domain while
preserving bounce-ability. If you use transparent forwarding for mail
originating from a domain with a restrictive SPF record, it will fail.

We've also got DKIM to sign messages and DMARC which requires either SPF
or DKIM to succeed AND for the succeeding mechanism to use the domain
in the From header. Google and Yahoo have both said that they will be
requiring mail from large senders to be authenticatable via those
mechanisms to be accepted at all. It is expected that with those
behemoths getting tough, many others will finally start enforcing

In short: the mechanism you are using to forward email in the simplest
and oldest way is not sustainable.

--
Bill Cole
bill@??? or billcole@???
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@???
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/