Re: [exim] Smarthost + queue worker keep alive the connectio…

On 22/01/2020 12:43, Graeme Fowler via Exim-users wrote:
> I’ll send you a link to one separately.

Thanks.

> In short: generated a queue of just over 1000 messages with identical sender/recipient couplet but subject and message body varying, with Exim doing no queue running, then ran ‘exim -qq’ in full debug mode.
>
> It took 35 seconds to do the initial non-delivering queue run while parsing and routing each message on the queue, then started delivering them.
So that's 35ms per message.

Of interest in that debug output - there's a consistent 6 or 7ms pause
just after "Renaming spool header file". This is where exim has just
written a spool file under a temporary name; it renames it to the final
name and then does an fsync() on the directory.

[ You can build exim with an option to disable that fsync(). I don't
advise it if your mail is valuable to you. ]

6ms sounds like a 10k RPM disk rotation. Is your spool on rotating
rust, and could you consider an SSD?


I guess in the longterm, that's a possible point of attack. We could
parallelize the scan over that fsync time.



There's also a consistent 9 - 11ms pause just before that "Renaming"
output line, after the previous line (which is a note about the last
recipient entry in the file, just after it was written to the stdio
buffer). The code between those two places flushes the stdio buffers
and fsync()s to get the file data out on stable storage. That also
sounds amenable to parallel operations.
--
Cheers,
Jeremy