Re: [exim] Fwd: Rate-limit queue-processing per domain

Top Page
Delete this message
Reply to this message
Author: Charlie Elgholm
Date:  
To: exim-users
Subject: Re: [exim] Fwd: Rate-limit queue-processing per domain
Sorry, I forgot to answer the list, that's why that happened.

It's not that they don't trust us. We've been sending this mail for 10
years, and we never get flagged as SPAM or unsolicited, since every mail is
something the customer actively has requested. We're also a registered
sender with outlook.com and Gmail, so we get reports etc about our
mailings. We just notice that many, and I mean many, mailservers out there
have started throttling all their incoming mail. And it would be nice if
Exim could easily play along in that scheme, and skip those domains for a
while, but still process the rest of the queue in an orderly manner. Our
main client throttle everything themselves! Which actually is hilarious
since their employees want our mail as fast as possible, but their own IT
department refuses to make special rules for us - even though we are their
sole supplier of these emails to them! :)

Maybe it would seem inefficient to look at an email in the queue, and then
disregard it based on something, but I see it as inefficient to run 2 or
more queues as well, and have one of the queues trying to squeeze in mail
in the "delivery"-queue (as others have suggested). With the now suggested
prearranged queue that would not need to happen, which is good.

As it is now, a queue-runner opens a mail, tries to deliver it, gets a
"back off!"-message from their server, opens another mail (to the same
domain), gets the same "back off!"-message, opens another....etc... hardly
efficient.

But sure, naming different queues, spooling to them based on domain, and
manually delivering from some of them is probably the clean
"Exim-solution". I just have to figure out how to make Exim skip those
queues when starting automatic queue-runners.

It would be nice to have Exim automatically "back off" when it sees a lot
of errors from a domain though. That way we can handle this dynamically,
instead of having to manually identify all the domains who need throttling.
I thought I had configured this some years ago, but it must have been for
some other product.

Again, sorry for mailing you two times. Hope it doesn't happen again.

ons 18 okt. 2017 kl. 21:54 skrev Jeremy Harris <jgh@???>:

> I don't need duplicate individual mail to those on the ML.
> Please don't.
>
> On 18/10/17 17:51, Charlie Elgholm wrote:
> > 2017-10-18 16:41 GMT+02:00 Jeremy Harris <jgh@???>:
> >> On 18/10/17 11:42, Charlie Elgholm wrote:
> >>> Is there some way to have Exim rate-limit the queue runners, so they
> only
> >>> process - let's say 10 messages per minute - for a domain (like
> >>> outlook.com/live.com) - _regardless_ of how the message was put in the
> >>> queue.
> >>
> >> No. The queue-runners just don't do that.
> >>
> >> You'd need to DIY; something like "take the first 10 id's from the
> >> spooldir, once a minute, and run 'exim -M'". Possibly using
> >> a named-spool for each class of such mails you wanted to keep separate.
> >
> > Ok, that's sad.
> >
> > The infrastructure for it all is already on the code base:
> > 1. There's something returning true/false if a mail should be
> > processed in the queue (since frozen-messages is skipped).
> > 2. There's a rate-limit command already, highly configurable.
> > 3. There's a "rule-system" (executed for the ACLs).
>
> You could certainly have a redirect router first in the
> chain rigged to expand ::defer:: for anything after the
> first few items. With current released Exim.
>
> But you'd still be pointlessly doing this for all of the
> spool that you weren't going to deliver. Hardly efficient.
>
>
> (And if the big providers don't trust you, it's likely
> because you have not built up a good reputation with them yet)
> --
> Jeremy
>
> --
> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
> ## Exim details at http://www.exim.org/
> ## Please use the Wiki with this list - http://wiki.exim.org/
>

--
Regards
Charlie Elgholm
Brightly AB