Re: [exim] Small modification for queue runners?

On Fri, Dec 03, 2004 at 04:46:07PM +0000, Philip Hazel wrote:
> Looks like your test shows that it *is* the redundant trying of messages
> that have just been tried that is your problem when you run so many
> queue runners.

Yes. I badly need simultaneous deliveries to make good use of the system,
so I used multiple queue runners. If I have 20000 mails on the queue, 100
of them being deliverable right now, although slow, one queue runner can
not deliver them before more piles up, but I have a ratio of 19900 failed
deliveries vs. 100 successful ones. That's ok. Using 10 queue runners,
I have a ratio of 199900 failed deliveries cs. 100 successful ones.

Using 10 spawned deliveries from a central queue runner, I can deliver
10 times as much, yet I have the ratio of a single Exim queue runner.

> > Philip: Could you shed some light on why the queue runner needs the
> > pipe? How does the process tree look like when delivering multiple
> > messages down the same channel? Does one delivery fork and exec a new
> > delivery, passing it the channel?
>
> Yes.
>
> > If so, why can't it just exec it?
>
> The reason it can't just exec it is that the original queue runner needs
> to wait until the entire sequence of deliveries has happened. Otherwise
> it would not be following the rule "one queue runner does one delivery
> at a time"[*]. The original process that the queue runner creates may
> finish long before the entire chain. The pipe is a convenient way of
> detecting when all the forked processes have terminated.
>
> [*]Actually, the rule is already broken if one message has deliveries to
> more than one host, and there are other messages waiting for both of
> them.

As I see it, the queue runner parent keeps a pipe open, because its
child terminates without waiting for grandchildren it spawns before right
before terminating. If it would not fork grandchildren, but exec them,
the parent would still wait for the same process, thus not needing a pipe.

But I am probably missing something in this picture.

Michael