[exim] Callout cache duplicates + case invertions

Top Page
Delete this message
Reply to this message
Author: Evgeniy Berdnikov
Date:  
To: exim-users
Subject: [exim] Callout cache duplicates + case invertions
Hi guys.

I have several mail relays with Exim-4.92 on Debian, running with
"verify = sender/callout=300s". Today one incoming mail passed this
sender verify procedure on 1st relay (highest priority MX), then
hit greylisting (got 451) and came to the 2nd relay, where it was
rejected with

Aug 15 16:04:25 citrine exim[25920]: 2019-08-15 16:04:25.148 [25920] H=relay.seldon.ru [185.179.199.32]:49727 I=[192.168.10.13]:25 sender verify fail for <prvs=113054f968=username@???>: relay.seldon.ru [185.179.199.32] : SMTP error from remote mail server after RCPT TO:<prvs=113054f968=username@???>: 550 5.1.1 Backscatter Protection detected an invalid or expired email address

(real user address is mangled). Investigation shows that sender's MX host
does not pass sender verify if address contain uppercase letters:

220 relay.seldon.ru ESMTP MDaemon 18.0.2; Thu, 15 Aug 2019 18:05:39 +0300
[...]
mail from: <>
250 2.1.0 Sender OK
rcpt to: <prvs=113054f968=username@???>
250 2.1.5 Recipient OK
rcpt to: <prvs=113054f968=username@???>
550 5.1.1 Backscatter Protection detected an invalid or expired email address

Well, this is obvious violation of the case insensitivity principle for
domain part. But MDaemon bugs are not targets for this discussion.
I was surprised that sender's MTA came to my 1st relay with lowercase
domain in env_from address (seldon.ru), which passed sender verify, and
since several seconds came to came to 2nd relay with uppercase letters
in domain part. Logs have different register for env_from domain
of the same mail.

Maybe this is some other MDaemon bug resulting in case change during
relay's traversal, but this scenario does not explain all facts below.
First, I see something strange with callout cache on the 2nd relay:

# exim_dumpdb /var/spool/exim4 callout | fgrep -i seldon
15-Aug-2019 16:04:24 prvs=113054f968=username@??? callout=accept
15-Aug-2019 16:04:25 prvs=113054f968=username@??? callout=reject

Only one mail was logged around 16:04:25 from this address, and it was
rejected. Why callout cache have TWO different entries with very close
timestamps but different cases for domain part and different results?

I made some experiments, feeding some env_from addresses to my relays
and watching network traffic. Results are:
1. Callout entries for mail addresses have the same case as in MAIL FROM.
2. Callout entries for host names are always in low case.
3. Single env_from results in a single SMTP callout.
I failed to reproduce duplication of callout records and case inversions.
However, there are duplicates in cache, for different domains.
Examples:

# exim_dumpdb /var/spool/exim4 callout | fgrep -i noreply@???
25-Jul-2019 15:06:55 noreply@??? callout=accept
25-Jul-2019 15:06:56 noreply@??? callout=accept

# exim_dumpdb /var/spool/exim4 callout | fgrep -i club@???
25-Jul-2019 11:15:52 club@??? callout=accept
25-Jul-2019 11:15:52 club@??? callout=accept

# exim_dumpdb /var/spool/exim4 callout | fgrep -i osp.ru
18-Jul-2019 17:39:36 bounce-264727097@??? callout=accept
18-Jul-2019 17:39:35 bounce-264727097@??? callout=accept
18-Jul-2019 17:39:36 bounce.email.osp.ru callout=accept postmaster=unknown random=unknown

Logs for these 3 mails contain addresses with lowcase domain parts.
Case inversions+duplicates are not rare, they count to ~ 25% mails
on my hosts:

# exim_dumpdb /var/spool/exim4 callout | fgrep -v random= | egrep '[A-Z]\.' | wc -l
1340

# exim_dumpdb /var/spool/exim4 callout | fgrep -v random= | egrep -v '[A-Z]\.' | wc -l
5951

Have anybody idea why could it happen? Is it a bug?
--
Eugene Berdnikov