[Exim] pipes and system load

Top Page
Delete this message
Reply to this message
Author: Marc Haber
Date:  
To: exim-users
Subject: [Exim] pipes and system load
Hi!

My home machine is running exim 3.02 and is getting fed from a POP3
mailbox with fetchmail. I get quite some amount of mailing list
traffic and would like to use exim to pipe that mail into mail2news to
be able to read my mailing lists like newsgroups.

My first attempt was an absolute desaster: I simply added a .forward
file to the account that receives the mailing list mail:

|root@torres[2/502]:/mnt/main6/home/mh$ cat /home/mh-lists/.forward
|# Exim filter <<== do not edit or remove this line!
|
|testprint "return_path == $return_path"
|if $return_path is "<canossa-bounce@???>" then
|        deliver "mh-lists-mbox"
|        pipe "mail2news -n \"gf1.lists.canossa\""

|else
|        deliver "mh-lists-mbox"
|        pipe "mail2news -n \"gf1.lists.unknown\""

|endif
|root@torres[3/503]:/mnt/main6/home/mh$

When a bunch of e-mail came in, the different exim processes called a
bunch of mail2news processes which in turn called a bunch of inews
which in turn made the inn quite busy. As a result, my machine ran out
of file descriptors and crashed badly.

So I believe that I need to serialize the pipes. This could easily be
accomplished by having exim queue _every_ mail that is being processed
and then later to call a queue runner to deliver the mail. Since the
queue runner is serially processing the mail, load is kept low.

However, this approach has a number of serious disadvantages:

- queueing everything delays delivery of all messages.
- Using a queue runner to deliver remote mail, mailing list mail is
also processed in that run, delaying remote delivery which results
in my machine staying online for longer periods of time.

Can I selectively queue mail addressed to mh-lists while delivering
other mail at once?

Can I queue mail that is destined to a pipe while doing normal
deliveries at once?

Can I have a queue runner process mail selectively (for example: "You,
new queue runner, are only to process remote mail, while you, queue
runner number two, are only to process local mail)?

Would "pipe 'nice mail2news...'" also cause the inews and posting
processes to run niced, stopping exim from wrecking my system?

Does anybody have ideas to solve this problem in an elegant way?

BTW, I really feel that exim should have an option to stop certain
actions (like pipes) if system load is getting too high. I know that
there are some options to do so, but I did not find an option to stop
pipes if load is getting too high.

Greetings
Marc

-- 
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber          |   " Questions are the         | Mailadresse im Header
Karlsruhe, Germany  |     Beginning of Wisdom "     | Fon: *49 721 966 32 15
Nordisch by Nature  | Lt. Worf, TNG "Rightful Heir" | Fax: *49 721 966 31 29