Re: [exim] Should queue processing be rewritten in Exim?

Top Page
Delete this message
Reply to this message
Author: Marc Perkel
Date:  
To: exim-user
Subject: Re: [exim] Should queue processing be rewritten in Exim?


Peter Bowyer wrote:
> On 02/07/2008, Marc Perkel <marc@???> wrote:
>
>> Suppose Exim had 2 queues instead of one. We'll call the first queue the
>> primary queue which is a ram drive and the secondary queue which is a
>> disk drive. All this is of course optional and user settable.
>>
>
> You're proposing a solution before defining the requirements. What's
> the high-level use case here?
>
> Peter
>
>

Hi Peter,

I'll use myself as an example. I am in the front end spam filtering
business. (http://www.junkemailfilter.com) I have about 4000 domains
pointed to a single Exim server. Email comes in - I filter it - and I
send the good email on to the customer's existing server. This machine
is running Exim only. Spamassassin is running on a several other
computers as well as MySQL and caching DNS servers. This one box, which
is the box most everything hits first is all Exim. I have several other
Exim servers on the next level of MX for backup and overload handling.

Additionally I have a a spam out channel where spam is forwarded to
another server and is delivered to several other places that process my
spam. It goes off to spam blocking companies for mining and to URIBL
list processors to feed their block lists. These messages are very low
priority and if they are lost or rejected there is no loss.


Most email hits this server and if it's good is instantly relayed to the
customer's server for delivery. Most spam is forwarded to my spam
processing server and if it fails it's deleted. No retry on spam. If the
customer's server is down or overloaded and I get a 4xx error on
delivery the message is transferred to a retry server which will attempt
to deliver the message for 4 days. So the idea of this main exim server
is not to hold anything in it's own queue. Whatever comes in goes
somewhere because it is the fast bulk processor.

The bottleneck is the queue and I've noticed that if I use a ram disk
for the queue that I can process many times as much mail with var lower
load levels that using disk based queue. Ideally if I had a battery back
up ram disk card that would be ideal but I can't find that anywhere.

So - that's the problem I wish to solve.