On 8 Jan 2000, Kai Henningsen wrote:
> Exim has this db directory.
Nice thought, but ...
> You could keep a copy of the spool *directory* in a database - Exim
> already knows how to do stuff like that, and how to keep it up-to-date.
Not exactly. All the data in the db directory is *hints*, and not
considered vital. For example, if an attempt to update one of those
databases fails, Exim just ignores the error. Also, it is not a major
problem if any of the files are lost.
If Exim started relying on a list of messages in such a file, then it
would have to be made reliable, and the usual heavier-weight apparatus
of transactions, journals, etc. for updating a database it would have to
be implemented. You couldn't just use the existing functions as they
stand.
> And you can have it do a directory rescan once in a while to make sure no
> message got left out
This is what worries me about all of these ideas. It was part of the
basic design of Exim that the contents of the spool directory *are* the
queue - "one fact in one place" - there is no secondary list that can
get out of sync. The operating system knows how to cope with contention
for different processes accessing files in the same directory - Exim
doesn't have to reproduce this kind of code.
> No need to learn about shared memory. It's slower, but should still be
> significantly faster than scanning directories. And you can cache whatever
> info the queue runner needs to decide which message to tackle next.
Again, the whole philosophy of Exim is that it doesn't try to be clever
about managing its queue. A queue-runner just works through all the
messages on the queue, in random order. There's no rule about "which
message to tackle next". Note that this means that, even if a cache of
the spool directory were implemented, a queue-runner would still have to
read every -H file on the queue. So you might not gain as much as you
think.
Enormous queues are something for which Exim was not designed, I'm
afraid. It seems to me that the problem discussed in this thread is
caused by Exim reaching the limit of what it can handle (which, I must
confess, is far more than I ever envisaged when I started). I'm not an
expert on discs and their operation, but maybe the best solution (while
retaining Exim) is somehow to speed up the directory scanning, if that
is possible.
--
Philip Hazel University of Cambridge Computing Service,
ph10@??? Cambridge, England. Phone: +44 1223 334714.
