Re: [Exim] Understanding queue runners

> Exim is not designed to operate with long queues. It's a point I try to
> make again and again. However, a queue runner forks only one process at
> a time. Are you sure it is the processes that have the impact? I thought
> a cheap fork() was supposed to be one of the good things about
> Unix/Linux systems. I would have thought it much more likely to be the
> disk accesses. A queue runner has to read every -H file in the queue.

I don't know if the fork() or something else chews up CPU time in
system mode so badly, but it is some system call that makes queue
runners expensive. It is not user space CPU time or disk accesses.

The -H files are small and I have enough RAM, and 3ware controllers
perform great, so disk accesses are no problem, in particular read
accesses.

> > But how could the queue runner be modified not to fork a process if
> > the forked process would not try a real delivery anyway?
>
> With very great difficulty. It doesn't fit the underlying paradigm for
> all sorts of reasons. (And as I said above, I'm not convinced it would
> actually help.)

I am wondering about a bunch of solutions or work-arounds:

o A purely message-based retry database, omitting the transport retry
databases. I don't know if there is a killer argument against this
option.

o Multiple queues or an easy to get message attribute that tells the
queue runner to not even look at a message for some time. I don't
know how frequently my queue runners actually visit the same message
to determine which interval would help.

> As for the -J files, not creating them when not needed isn't too easy, I
> suspect. Remember that Exim does deliveries in subprocesses that run
> under different uids. The -J file is created at the outer level.

The parent could open a pipe and collect data for a -J file, creating
one and storing the data in it, if it receives any. Or does the client
also need to read the file?

Michael