On Fri, 3 Dec 2004, John W. Baxter wrote:
> This "feels like" another great opportunity for per OS and per OS version
> code to creep into Exim (and in the case of Linux, possibly
> per-distribution).
Given Nigel's subsequent post, this could probably only be done as an
option, which doesn't seem to good an idea.
Brooding over the original discussion over the weekend, I can see that,
for situations where there is a queue of any size, a queue runner that
makes ONE list of messages and then tries to deliver several at once
would waste less time looking at messages unncessarily that several
queue runners. This is borne out by Michael's experiments with his own
"external queue runner". Exim really does work best when one queue
runner can finish its work before the next one starts, because queue
runners were never intended to be a primary delivery mechanism.
A more radical idea would be to invent yet another hint. When a
message fails to be delivered and is left on the queue, the hints for
various delivery actions (hosts etc) are updated together at the end of
the delivery process. It should be straightforward to remember the
earliest retry time for any of them. So one could make a hint record,
keyed by message id, that contained this time. A queue runner could
consult this file before forking and trying to deliver a message.
The problem with this is the bottleneck of the hints database. The queue
runner would have to keep opening and closing it so that delivery
processes that were finishing could update it. This leads us on into the
territory of database concurrent usage. I believe that BDB4 has
mechanisms for this, but it would be a radical departure to insist that
all Exim users use BDB4.
My current position is that this is an area that might in the future be
played with, but it isn't something for the short-term. And ideally,
someone who has these huge queues needs to do experiments - it isn't
something I can simulate.
--
Philip Hazel University of Cambridge Computing Service,
ph10@??? Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book
