Re: [Exim] Performance bottleneck scanning large spools.

> Enormous queues are something for which Exim was not designed, I'm
> afraid. It seems to me that the problem discussed in this thread is
> caused by Exim reaching the limit of what it can handle (which, I must
> confess, is far more than I ever envisaged when I started). I'm not an
> expert on discs and their operation, but maybe the best solution (while
> retaining Exim) is somehow to speed up the directory scanning, if that
> is possible.

Could someone define "enormous queues" in terms of numbers? Perhaps it
may be wise to start with measuring some parameters as kernel CPU time
and disk throughput to find out where exactly the problem is. Is it
reading the directories? Is it getting the inodes of all directory
entries? Is it possibly lock contention between exim processes?

I increased my inode cache to take many actively used files into
account and I think I increased the dentry cache as well. I had
to increase the SYN table as well (note that Linux 2.2 does that with
/proc/sys/net/ipv4/tcp_max_syn_backlog, not with the listen parameter).
I monitor CPU load, network and disk throughput as well as queue sizes
with MRTG. Disk traffic is mostly less than network traffic and overall
performance is not an issue, but my queues rarely hold more than a few
thousand mails. I am afraid that I can't shutdown outgoing delivery to
see how things might change with a growing queue. ;-)

Michael