Hello Heiko,
another long mail, it seems that primary_hostname in the header is
replaced via some other parts of exim memory. Only if more than one mail
is delivererd via one TCP session (actually I see corruption on second
mails via one connection).
On 06.06.2018 22:00, Heiko Schlittermann via Exim-users wrote:
>> You are right, the X-SA-Exim-Scanned header is truncated (after "on", I
>> missed that before) it is set by sa-exim (code snipplet from sa-exim.c
>> with line numbers):
> Ah, that *I* didn't see, that there's a fragment of the header to be
> added. Hm. The 's-' is part of the primary hostname?
s- is not part of the primary hostname (primary hostname is
"deep-thought").
By digging in deeper, it seams that instead of the primary_hostname some
other (possibly random memory parts) are inserted. I disabled sa-exim
spam scanning for linux-kernel mailing list. However errors occur (on
same header inserted by spamassassin).
Strangest example I have is following (as said without active scanning),
where parts of my alias definitions are inserted (SPAMASSASSIN headers
from spamc invocation via procmail during delivery). This looks like
major parts of /etc/aliases (not complete though some lines of the
beginning of the file are missing). Some blank lines in there are
replaced by 0. End of the sa-exim header "); SAEximRunCond expanded to
false" is fully in there (only primary_hostname) is replaced:
--cut
Return-path: <linux-kernel-owner@???>
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on
deep-thought.ursa-minor-beta.org
X-Spam-Flag: YES
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.9 required=5.0
tests=HEADER_FROM_DIFFERENT_DOMAINS,
MAILING_LIST_MULTI,RAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RCVD_IN_BL_SPAMCOP_NET,
URIBL_MW_SURBL autolearn=no autolearn_force=no version=3.4.1
X-Spam-Relay-Country: US ES KR
X-Spam-Report:
* 1.2 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
* [Blocked - see <http://www.spamcop.net/bl.shtml?221.163.32.101>]
* 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail
* domains are different
* 1.3 URIBL_MW_SURBL Contains a URL listed in the MW SURBL blocklist
* [URIs: glasneck.de]
* 2.4 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level above 50%
* [cf: 100]
* 1.7 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
* -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list
* manager
Envelope-to: linux-kernel@???
Delivery-date: Thu, 07 Jun 2018 21:40:22 +0200
Received: from vger.kernel.org ([209.132.180.67]:56086)
by deep-thought with esmtp (Exim 4.90_1)
(envelope-from <linux-kernel-owner@???>)
id 1fR0li-0007Lf-Bh
for linux-kernel@XXX; Thu, 07 Jun 2018 21:40:22 +0200
Received: (majordomo@???) by vger.kernel.org via listexpand
id S1752861AbeFGTkU (ORCPT <rfc822;linux-kernel@???>);
Thu, 7 Jun 2018 15:40:20 -0400
Received: from japeto.mep.pandasecurity.com ([92.54.27.188]:51624 "EHLO
japeto.mep.pandasecurity.com" rhost-flags-OK-OK-OK-OK)
by vger.kernel.org with ESMTP id S1751468AbeFGTkU (ORCPT
<rfc822;linux-kernel@???>);
Thu, 7 Jun 2018 15:40:20 -0400
X-Greylist: delayed 4470 seconds by postgrey-1.27 at vger.kernel.org;
Thu, 07 Jun 2018 15:40:20 EDT
Received: from [221.163.32.101] (helo=10.0.0.30)
by japeto.mep.pandasecurity.com with esmtpsa
(TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256)
(Exim 4.80)
(envelope-from <alvarez@???>)
id 1fQzbU-0001Oc-LR
for linux-kernel@???; Thu, 07 Jun 2018 20:25:45 +0200
X-Envelope-From: alvarez@???
Date: Fri, 08 Jun 2018 03:25:45 +0900
From: Emma Aiden <alvarez@???>
To: linux-kernel@???
Message-ID: <393970332.201867182545@???>
Subject: Pay Invoice
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_NextPart_000_00D0_B7B9959F.A17E6B58"
X-CTCH-IPCLASS: G2
X-SPF-Received: 4
X-Spamina-Bogosity: Ham
Sender: linux-kernel-owner@???
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@???
X-SA-Do-Not-Run: Yes
Received-SPF: none client-ip=209.132.180.67;
envelope-from=linux-kernel-owner@???; helo=vger.kernel.org
X-SA-Exim-Connect-IP: 209.132.180.67
X-SA-Exim-Rcpt-To: linux-kernel@XXX
X-SA-Exim-Mail-From: linux-kernel-owner@???
X-SA-Exim-Scanned: No (on
daemon: root
bin: root
sys: root
sync: root
games: root
man: root
lp: root
news: junk
clamav: root
uucp: root
proxy: root
majordom: root
postgres: root
www-data: root
backup: root
msql: root
operator: root
list: root
irc: root
gnats: root
nobody: root
logcheck: root
0hostmaster: root
usenet: root
news: root
webmaster: root
www: root
ftp: root
abuse: root
noc: root
security: root
0mailer-daemon: postmaster
munin: root
fail2ban: thomas
0root: system@XXX, benny@XXX
kernel@XXX); SAEximRunCond expanded to false
------=_NextPart_000_00D0_B7B9959F.A17E6B58
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
=0DThis is the invoice for your new order.=0DPlease review this information=
and confirm this is good to go.
http://glasneck.de/DOC/Customer-Invoice-IG-1757272/
=0DBest Regards,
Emma Aiden
=0DOffice: 503.798.1252=0DFax: 503 566-6659
------=_NextPart_000_00D0_B7B9959F.A17E6B58--
--cut
Exim mainlog show the corrupted message as second mail sent over one TCP
connection (linux kernel mailing list server is the only server that
sends more than one mail per TCP connection, other servers do not send
those volumes). I do not follow all messages on the list, thus there may
be other errors/corruptions (the queue error I had initially are the
most obvious, other corruption which do not lead to technical errors).
Quick grep in the Mail dir shows significant number of messages which
seem to have some unexpected strings in the header. I see corruption in
this specific header for other messages as well, all have in common that
there was one than one message sent over one single TCP connection.
I am setting a debug header containing $primary_hostname in an acl
stanza to see if there is some corruption in this heades as well.
--cut exim mainlog
2018-06-07 21:38:36.153 [15531] SMTP connection from
[209.132.180.67]:56086 I=[176.X.X.X]:25 (TCP/IP connection count = 1)
2018-06-07 21:38:36.717 [28251] 1fR0k0-0007Lf-Jr DKIM:
d=baylibre-com.20150623.gappssmtp.com s=20150623 c=relaxed/relaxed
a=rsa-sha256 b=2048 [verification succ
eeded]
2018-06-07 21:38:36.736 [28251] 1fR0k0-0007Lf-Jr SA: Debug:
SAEximRunCond expand returned: '0'
2018-06-07 21:38:36.736 [28251] 1fR0k0-0007Lf-Jr SA: Action: Not running
SA because SAEximRunCond expanded to false (Message-Id:
1fR0k0-0007Lf-Jr). From <linux-kernel-owner@???>
(host=vger.kernel.org [209.132.180.67]) for linux-kernel@XXX
2018-06-07 21:38:36.782 [28251] 1fR0k0-0007Lf-Jr <=
linux-kernel-owner@??? H=vger.kernel.org
[209.132.180.67]:56086 I=[176.X.X.X]:25 P=esmtp K S=4546 M8S=8
id=1528400289-28004-1-git-send-email-clabbe@??? from
<linux-kernel-owner@???> for linux-kernel@XXX
2018-06-07 21:38:38.665 [28255] 1fR0k0-0007Lf-Jr => thomas
<linux-kernel@XXX> F=<linux-kernel-owner@???>
P=<linux-kernel-owner@???> R=procmail T=procmail_pipe S=4751
QT=2.050s DT=1.864s
2018-06-07 21:38:38.666 [28255] 1fR0k0-0007Lf-Jr Completed QT=2.050s
2018-06-07 21:40:22.425 [28251] 1fR0li-0007Lf-Bh SA: Debug:
SAEximRunCond expand returned: '0'
2018-06-07 21:40:22.425 [28251] 1fR0li-0007Lf-Bh SA: Action: Not running
SA because SAEximRunCond expanded to false (Message-Id:
1fR0li-0007Lf-Bh). From <linux-kernel-owner@???>
(host=vger.kernel.org [209.132.180.67]) for linux-kernel@XXX
2018-06-07 21:40:22.469 [28251] 1fR0li-0007Lf-Bh <=
linux-kernel-owner@??? H=vger.kernel.org
[209.132.180.67]:56086 I=[176.X.X.X]:25 P=esmtp K S=2921 M8S=8
id=393970332.201867182545@??? from
<linux-kernel-owner@???> for linux-kernel@XXX
2018-06-07 21:40:23.096 [28251] SMTP connection from vger.kernel.org
[209.132.180.67]:56086 I=[176.X.X.X]:25 closed by QUIT
2018-06-07 21:40:24.754 [29248] 1fR0li-0007Lf-Bh => thomas
<linux-kernel@XXX> F=<linux-kernel-owner@???>
P=<linux-kernel-owner@???> R=procmail T=procmail_pipe S=3126
QT=2.391s DT=2.266s
2018-06-07 21:40:24.754 [29248] 1fR0li-0007Lf-Bh Completed QT=2.391s
--cut
> Ok, the spool wire format is off, you said. I'm not sure about the
> mechanigs of sa_exim, that is, I do not have any clue *which* file it
> sees and modifies. And/or if we built some optimisations which assume
> that the spooled files (spooled in $spooldir/scan) are not altered.
>
> For better theories about what's going on we need to know which files
> sa_exim accesses.
>
> If this is important and worth to be solved,, it would need some further
> investigation.
I did not have the time to dig deeper into that yet. Thus I think it is
worth, will try to
> @Jeremy: Maybe we should announce that sa_exim will have
> some end-of-life in the near future?
I am looking into this, as sa-exim is (in my humble opinion) one of the
best SPAM checking and greylisting integrations as it combines both to
get the most out of it.
> I do greylisting based on the announced content size. But your approach
> might be good too.
>
> I wrote some Perl function(s) to support greylisting in Exim, these
> functions work reliable for years already. Tell me, if you're
> interested, I"d update the docs and the scripts a bit and publish it.
>
> (To be true, it is published already, but the docs are outdated.)
I'd appreciate some best practices/real life approaches on this. I
tested some combination of greylistd and spamd integration and wrote a
small poc spamassassin plugin to additionally rate on greylisting
status. However this was not as straight-forward as I expected it to be
in the first place (greylistd-integration works fine on acl_rcpt but not
on acl_data as I expected it to work). Actually this might be because I
am not that deep in the topic any more (exim is running rock-solid
stable for years *thanks a lot for that great job guys* and I had to pay
much more attention to other topics).
Best regards,
Thomas
