Re: [Exim] exiscan and delivery process explosion

Top Page
Delete this message
Reply to this message
Author: Tom Kistner
Date:  
To: Graeme Wilford
CC: exim-users
Subject: Re: [Exim] exiscan and delivery process explosion
On Thu, Apr 19, 2001 at 01:48:17PM +0100, Graeme Wilford (g.wilford@???) wrote:

> We've been using exiscan-0.99 for a while now (since I last emailed you
> to discuss a possible race condition) and have been very happy with it.
>
> Recently, we had a series of power and system failures that left us with
> exim queues containing several thousand successfully scanned emails.
>
> When starting 'exim -bd -q' without exiscan under these conditions, a
> max of queue_run_max queue runners (default: 5) are spawned. However,
> when exiscan is used to handle de-queueing, it spawns an explicit exim
> delivery process for each and every spooled email. Our mail servers
> would boot and then run out of resources as 2000+ exim processes fought
> to deliver outstanding emails.


I see. Thats a design error on my part. I do not check how many 'exim -Mc'
type processes are already running.

> Can exiscan be modified to fork delivery processes as children and use
> wait to monitor and limit the maximum amount running at any one time?


yes, exiscan could fork itself and the child would wait for the "exim -Mc"
system call to finish, while the parent could happily continue scanning.
The child could then quit and the parent would reap it with a
(non-blocking) waitpid call while running through the main loop the next
time ... Parent would hold a child counter and compare it against a MAX
setting in the config file ...

> I might be able to spend a little time looking at this if it doesn't
> clash with other exiscan development and if you aren't already looking
> into this...


It won't clash with the 1.0 exiscan series as I have begun work on a
much-improved rewrite, focusing on cleaner code instead of featurism
(the current features will be in, of course).

This issue will be fixed in the rewrite, however I would kindly ask if you
could fix it for 0.99 / 1.00 ? I would place it on my exiscan page as a
diff, with credits :) I'm swamped with real life (paid) work at the moment :)


cheers,

    tom


--
Tom Kistner <tom@???> | ICQ 1501527