Re: [exim] Smarthost + queue worker keep alive the connectio…

Góra strony
Delete this message
Reply to this message
Autor: Jeremy Harris
Data:  
Dla: exim-users
Temat: Re: [exim] Smarthost + queue worker keep alive the connection
On 25/01/2020 17:46, Jeremy Harris via Exim-users wrote:
> On 22/01/2020 12:43, Graeme Fowler via Exim-users wrote:
>> It took 35 seconds to do the initial non-delivering queue run while parsing and routing each message on the queue, then started delivering them.
> So that's 35ms per message.


> I guess in the longterm, that's a possible point of attack. We could
> parallelize the scan over that fsync time.


Development testing of a patch gives:

Testcase: 80 messages queued, "exim -d+all" instrumented for the time
until completion of the 1st phase.

4-core x86_64, consumer SSD:
Baseline: approx 1 sec (12.5 ms/msg)
2-ply:           613ms (7.66 ms/msg, +63% speed)
10-ply:          207ms (2.59 ms/msg, +380% or 4.8x speed)


8-core aarch64, rotating disk:
Baseline:        4.8 sec (60 ms/msg)
2-ply:           3.5 sec (44 ms/msg, +36% speed)
10-ply:          3.4 sec (42.5 ms/msg, +42% speed)



Looks like the aarch system is into diminishing returns; unfortunately
because both the disk and the cpu change across the two test platforms
we don't know which was the limiting factor. However, the speedup looks
worthwhile so I think I'll make a guess at 4-ply for an initial commit.

It can't be used when queue_run_in_order is active.



Bleeding-edge self-builders may also be interested in 9438970c97:
"ACL: control = queue/first_pass_route". It provides for SMTP-sourced
messages the same facility that "-oqds" does for command-line source.
--
Cheers,
Jeremy