Re: [exim] mail delivery to smarthost very slow

Αρχική Σελίδα
Delete this message
Reply to this message
Συντάκτης: Jonas Meurer
Ημερομηνία:  
Προς: Evgeniy Berdnikov
Υ/ο: exim-users, Exim-users
Αντικείμενο: Re: [exim] mail delivery to smarthost very slow
Hi,

first, thanks a lot for your answers. It's much appreciated.

Am 2015-11-16 12:26, schrieb Evgeniy Berdnikov:
> On Mon, Nov 16, 2015 at 10:50:12AM +0000, Jeremy Harris wrote:
>> On 16/11/15 10:06, Jonas Meurer wrote:
>> > As you can see, the mails are sent to the smarthost one by one, with
>> > a new connection for each mail. Is it possible to make Exim4 deliver
>> > several mails together to the smarthost in one connection?
>>
>> Exim normally reuses a connection for multiple deliveries; it takes
>> specific configuration to make it not do so. This is a Debian
>> config problem; you need to ask in a Deb-specific place. The Deb
>> docs should tell you where.
>
> Debian Exim package has no specific options for remote_smtp_smarthost
> transport, limiting number of messages per connection.


The remote_smtp_smarthost transport configuration here is exactly the
following (macros already expanded):

remote_smtp_smarthost:
   debug_print = "T: remote_smtp_smarthost for $local_part@$domain"
   driver = smtp
   hosts_try_auth = <; ${if exists{/etc/exim4/passwd.client} \
         {\


${lookup{$host}nwildlsearch{/etc/exim4/passwd.client}{$host_address}}\
         }\
         {} \
       }


No connection_max_messages is set in the whole Exim4 configuration at
all, so the default of 500 should apply to the remote_smtp_smarthost
transport, no?

> And I see no proof of single-delivery statement in the initial post,
> its logs do not show such behaviour. However, some mails are not sent
> immediately but queued due smtp_accept_queue_per_connection=100, with
> record in log "no immediate delivery: more than 100 messages received
> in one connection". Maybe the reason of delays is that some mail
> bundles
> are sent to smarthost during scheduled queue runs (once 30 min or
> more).


Indeed, most messages are queued due to smtp_accept_queue_per_connection
being set to 100.

The mailserver in question is quiet most of the time, but several times
a week it receives something like 15k-20k similar newsletter (but not
identical) messages from a web application. To my understanding, the
first
messages are sent immediately until 100 delivery processes are active.
all remaining messages are queued and sent later by the exim4 queue
runner.

Is it a good idea to raise smtp_accept_queue_per_connection to a very
high value or disable it by setting it to 0 in order to make Exim
deliver all messages immediately?

Apart from that, I don't get, why the queue runner sends queued messages
one by one each with a new connection to the smarthost. At least, that's
how I read the logs. How can I distinguish from the logs whether
messages
are sent bundled in one connection or with a new connection each? To me,
the logs look like the latter is true in my case. No asterisks next to
H=... and TLS connection details in each delivery log.

Here's another example of Exim sending out some messages from the queue
to the smarthost:

[...]
2015-11-13 14:31:25 1ZxERp-0004ep-DA <=
melin+N7-IDNL1447420608-C0@??? H=localhost
(nabu-newsletter) [127.0.0.1] P=esmtp S=60573
id=1744905331.167881447421485398.JavaMail.root@nabu-newsletter
2015-11-13 14:31:25 1ZxERp-0004ep-DA no immediate delivery: more than
100 messages received in one connection
[...]
2015-11-13 14:34:20 1ZxEUe-00059s-D7 <=
melin+N7-IDNL1447420608-C0@??? H=localhost
(nabu-newsletter) [127.0.0.1] P=esmtp S=112449
id=805524646.201721447421660403.JavaMail.root@nabu-newsletter
2015-11-13 14:34:20 1ZxEUe-00059s-D7 no immediate delivery: more than
100 messages received in one connection
[...]
2015-11-13 14:34:51 1ZxEV9-0005JT-OM <=
melin+N7-IDNL1447420608-C0@??? H=localhost
(nabu-newsletter) [127.0.0.1] P=esmtp S=95763
id=1632431573.208141447421691753.JavaMail.root@nabu-newsletter
2015-11-13 14:34:51 1ZxEV9-0005JT-OM no immediate delivery: more than
100 messages received in one connection
[...]
2015-11-13 14:35:24 1ZxEVf-0005JT-WE <=
melin+N7-IDNL1447420608-C0@??? H=localhost
(nabu-newsletter) [127.0.0.1] P=esmtp S=91838
id=898528339.213911447421723997.JavaMail.root@nabu-newsletter
2015-11-13 14:35:24 1ZxEVf-0005JT-WE no immediate delivery: more than
100 messages received in one connection
[...]
2015-11-13 15:22:21 1ZxEVf-0005JT-WE Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxEVf-0005JT-WE => john.doe@??? R=smarthost
T=remote_smtp_smarthost H=smtp2.eu [195.34.176.148]
X=TLS1.0:RSA_AES_128_CBC_SHA1:128 DN="OU=GT46541191,OU=See
www.rapidssl.com/resources/cps (c)14,OU=Domain Control Validated -
RapidSSL(R),CN=*.inet.de" C="250 2.0.0 tADEMKtb017818 Message accepted
for delivery"
2015-11-13 15:22:21 1ZxEVf-0005JT-WE Completed
2015-11-13 15:22:21 1ZxEUe-00059s-D7 => jane.doe@??? R=smarthost
T=remote_smtp_smarthost H=smtp2.eu [195.34.176.148]
X=TLS1.0:RSA_AES_128_CBC_SHA1:128 DN="OU=GT46541191,OU=See
www.rapidssl.com/resources/cps (c)14,OU=Domain Control Validated -
RapidSSL(R),CN=*.inet.de" C="250 2.0.0 tADEMKhr017821 Message accepted
for delivery"
2015-11-13 15:22:21 1ZxEUe-00059s-D7 Completed
2015-11-13 15:22:21 1ZxER6-0004ep-1p Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxETe-0004v9-0M Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxEV9-0005JT-OM Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxETJ-00059s-HF Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxEV9-0005JT-OM => another.john.doe@???
R=smarthost T=remote_smtp_smarthost H=smtp2.eu [195.34.176.148]
X=TLS1.0:RSA_AES_128_CBC_SHA1:128 DN="OU=GT46541191,OU=See
www.rapidssl.com/resources/cps (c)14,OU=Domain Control Validated -
RapidSSL(R),CN=*.inet.de" C="250 2.0.0 tADEMKIG017817 Message accepted
for delivery"
2015-11-13 15:22:21 1ZxEV9-0005JT-OM Completed
2015-11-13 15:22:21 1ZxETS-00059s-16 Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxETJ-00059s-HF Spool file is locked (another
process is handling this message)
2015-11-13 15:22:21 1ZxERp-0004ep-DA => another.jane.doe@???
R=smarthost T=remote_smtp_smarthost H=smtp2.eu [195.34.176.148]
X=TLS1.0:RSA_AES_128_CBC_SHA1:128 DN="OU=GT46541191,OU=See
www.rapidssl.com/resources/cps (c)14,OU=Domain Control Validated -
RapidSSL(R),CN=*.inet.de" C="250 2.0.0 tADEMK55017822 Message accepted
for delivery"
2015-11-13 15:22:21 1ZxERp-0004ep-DA Completed

Cheers
jonas