Auteur: Charlie Elgholm Datum: Aan: Exim-users Onderwerp: Re: [exim] "exim -bpc", but only for messages not waiting for retry
On 2018-12-07 14:01, Jeremy Harris wrote: > In what sense "not tried"? Not yet delivery-tried, that is => "has only been queued as of yet, not
tried for delivery".
> What's wrong with the usual queue-runner auto-ramp? We have Exim both for normal emails and bulk emails for one of our clients
employees (internal information).
For normal emails, and hence normal queue handling, the normal "auto-ramp"
works fine (we have ours at "30s", if that's what you mean).
Port 25 uses the normal queue-mechanism, where we try to delivery up to 5
emails directly, if the load average permits, and queue the rest.
But for the bulk-sending, we generate a couple of thousand (individual)
emails in batches, queue them all, and then would like to process the queue
as quickly as possible.
We use another port, which only listens internally, for the "queue only"
(bulk) process.
I can manually monitor the queue, and issue a bunch of "exim -q &" to start
a bunch of queue runners.
But sometimes our clients Baltic smtp-servers (they have many offices) are
offline or tells us "too many emails", and then the queue-runner of course
flags that individual message as "try later".
The "problem" is that these messages are of course included in "exim -bpc",
so I don't really know how many of the emails in the queue are actually
waiting to be retried or have not been tried.
We fill the queue at a rate of 1000 emails per minute, since that is how
many individually generated emails our application server and database
manages to spit out.
We generate around 8000 to 50000 emails, depending on what kind of
information is to be shared. Which means we keep filling the queue for
about 8 to 50 minutes, and we of course want Exim to start processing the
queue as soon as the first emails arrive, and "build up" queue-runners
faster than our normal 30s window.
If "exim -bpc" gives me a count 3000, is that 3000 emails to be tried, or
is it 2 950 emails retry-waiting an hour or two, and only 50 emails to be
tried?
I would like to know, since then I can implement my queue-runners smarter.
There's no need for me to start 50 queue-runners to process 2 950 emails
that are retry-waiting for an hour, and no need for 50 queue-runners to
send 50 emails, where 2-3 should suffice.
The problem, as the script is written now, is that I only check "exim
-bpc", which gives me a total queue-count.
I would like to have queue-count/100 runners, that is for 5000 mails in the
queue I would like to have 50 queue-runners.
So far, so good, I autostart so that I always have queue-count/100 runners,
since some die of after a queue-run but more emails have been filled in the
queue.
The problem is when the queue "settles", and by that I mean that I get
2-4000 emails "retry-waiting" in the queue.
My system has not way of telling, so it keep starting 20-40 queue-runners
(every minute), trying to process the queue - which of course is futile
since the retry-timeout hasn't occured yet.
