On Fri, 28 Jul 2006, Peter D. Gray wrote:
> I am about to implement an exim configuration whereby mail
> tagged as spam by our gateways (not exim) is
> frozen in the queues for later delivery. I intend to
> deliver it at night. I use the queue_only option to
> put all mail into the queue in any case, and I have
> qrunners which perform actual delivery. The spam
> mail will be frozen by a global filter.
>
> Obviously, the size of the mail queue is about to get
> real large. I anticipate having 20K to 100K messages
> in the queue. I am running the split_queue_directory option.
Exim was never designed to operate efficiently with large queues. The
whole philosophy was to move mail quickly without hanging on to it. In
our environment, fewer than 5% of messages have to be retried. (See
chapter 3 of my book. Or you should have come on my course last week.
Oh, I see you are in .au; a bit far. :-) Of course, 5% of our volume is
nowadays a lot more messages than it was 11 years ago when I first
thought of Exim.
If you do as you suggest, you are going to be mixing two different
kinds of message in the same queue: those you are holding on to and
those that are having delivery problems. This makes the administration
harder.
> It would be nice if exim understood anternate queues, that
> way I could use the filter action to change the mail
> queue and freeze at the same time (RFE please).
There is a Wish List item about his, but as it is a major re-design, it
ain't going to happen quickly, if at all.
> My question I guess is just how the size of the queue
> will impact the qrunner performance. For example,
> can it tell without reading the files in the queue
> that a message is frozen?
No. Performance degrades with large queues.
> Or will it have to open and read each file just to ignore it if it is
> frozen?
Yes.
> It would be nice is the files did not need to be opened and read.
Indeed, but the underlying design principle is *not* to keep separate
lists of message information, because that leads to the possibility of
the lists getting out of step with the actual messages (something we'd
seen on a previous MTA we once ran). In Exim, the files ARE the queue.
It makes writing the software easier, and I do prefer to keep things
simple. :-) Not that Exim is simple, especially the way it has
grown... perhaps I should say "simpleR".
> If this is going to be a big problem, I will need to move items out of
> the queue that are frozen and move them back in later (or have an
> alternate config with the different queue).
I suggest you treat this in a similar way to storing mail for dial-up
hosts. The advice I give for this is to deliver such messages in BSMTP
format into files (per user, per domain, or whatever). In the dial-up
situation, this allows you to limit the volume, time out individual
messages, etc.
Then shovel them back into Exim using -bS when you want them delivered,
with an appropriate indication (e.g. protocol setting) so they don't end
up back in files again.
--
Philip Hazel University of Cambridge Computing Service
Get the Exim 4 book: http://www.uit.co.uk/exim-book
