Re: [exim] Why would this happen?

On 2009-04-26 at 13:31 -0500, Robert Nicholson wrote:
> So my ISP uses exim and I use .forward to call a perlscript to further
> filter my mail.
>
> What I observe is that whenever my ISP has an issue with their mail
> server a queue will build and all the senders of the messages that
> couldn't be delivered to me will eventually retry delivery. What
> surprises me is that although I would normally expect to see messages
> eventually arrive on their own when their queue is flushed.
> Occasionally it requires me to send myself a test message to get
> things moving. That suggests to me some misconfiguration of exim such
> that the queue or delivery of messages blocks at some point and
> unblocks when I send myself a test message. So whenever I send myself
> a test message I notice a lot more mail arrive out of sequence than
> would have had I not had sent myself the message.

This looks like normal behaviour, except for your expectation that mail
arrive in-sequence. If mail is not immediately delivered, it is put in
Exim's spool, which is often known as a queue but is not ordered. Mail
comes out in unspecified order from that.

Exim maintains hints databases to avoid repeated retries to a
destination which is down. The mails have been routed and are in the
relevant hints DBs, so that when a connection is established all the
mail for that destination can be quickly found and delivered, normally
down the one connection.

When a destination is marked as unreachable, the retry hints times are
used to retry -- these have back-offs, as configured in the retry
section of the config. When a new mail comes in, an attempt is made to
deliver. If it fails, the relevant hints are updated and the mail is
left in the spool. If it succeeds, then the hints are used to find all
the other mail needed.

The only misconfiguration is that if the mail server problems really are
at the ISP end then they don't have procedures in place to manually try
a queue flush post-recovery immediately with a special Exim run, using
exim -qf (or, if their customers match a common unique substring, then
exim -Rf instead).

You might find it useful to re-read:

3.13 Delivery in detail
3.14 Retry mechanism

It might also be worth checking to see if the ISP implements ETRN
anywhere, so that you can nudge things without a test mail.

-Phil