Re: [exim] Rate-limit queue-processing per domain

Top Page
Delete this message
Reply to this message
Author: Charlie Elgholm
Date:  
To: exim-users
Subject: Re: [exim] Rate-limit queue-processing per domain
> 2017-10-18 17:37 GMT+02:00 Viktor Dukhovni <exim-users@???>:
>> On Oct 18, 2017, at 6:42 AM, Charlie Elgholm <charlie@???> wrote:
>> Is there some way to have Exim rate-limit the queue runners, so they only
>> process - let's say 10 messages per minute - for a domain (like
>> outlook.com/live.com) - _regardless_ of how the message was put in the
>> queue.


> If you forward the mail to a Postfix outbound relay, you can create
> a clone of the "smtp" transport called "slow", select "slow" as the
> transport for outlook.com and live.com and get at most 10 messages
> per minute to each by setting:


Thank you for this answer, since it's a workaround. But I rather not
put more products in the pot, since that also requires training,
support, security-considerations, etc, etc, yada-yada... My main
customer is a bit picky. =)

>    slow_destination_rate_delay = 6s

>
> To prevent over-eager dead-host detection, you should also
> have:
>
>    slow_destination_concurrency_failed_cohort_limit = 10

>
> If you decide to avoid rate limits and instead set a low
> concurrency limit, you would then have something like:
>
>    # The default concurrency limit is 20
>    slow_destination_concurrency_limit = 5
>    slow_destination_concurrency_negative_feedback = 1/8
>    slow_destination_concurrency_positive_feedback = 1/3


Thank you for this. Excellent. It's certainly worth experimenting
with. But I would probable go "Full-Postfix" then, and try to rewrite
all the rules I have for our Exim-installation, instead of being
dependent on both products.

> Exim has more flexible built-in input processing, so you would
> continue to receive email via Exim, but then use a Postfix
> relay as an outbound "smarthost" for at least some domains.


Yepp, this is what most people end up doing I have noticed, since
Exim's queue-runners are "stupid" compared to other products. (Which I
personally don't believe, since Exim's configuration format and many
enhancements should be implemented on the queue-runners as well.)

> If you want the rate-limit to apply collectively to any of
> outlook.com/live.com/hotmail.com, you can declare a common
> nexthop for these:
>
>         outlook.com   slow:outlook.com
>         live.com      slow:outlook.com
>         hotmail.com   slow:outlook.com

>
> based on the observation that they presently (and will likely
> continue to) have the "same" set of MX hosts:
>
> $ dig +short -t mx outlook.com | sort -n
> 5 outlook-com.olc.protection.outlook.com.
> 10 mx1.hotmail.com.
> 10 mx2.hotmail.com.
> 10 mx3.hotmail.com.
> 10 mx4.hotmail.com.
>
> $ dig +short -t mx live.com | sort -n
> 2 live-com.olc.protection.outlook.com.
> 5 mx1.hotmail.com.
> 5 mx2.hotmail.com.
> 5 mx3.hotmail.com.
> 5 mx4.hotmail.com.
>
> $ dig +short -t mx hotmail.com | sort -n
> 2 hotmail-com.olc.protection.outlook.com.
> 5 mx1.hotmail.com.
> 5 mx2.hotmail.com.
> 5 mx3.hotmail.com.
> 5 mx4.hotmail.com.
>
> The highest priority MX on each list points to a DNS load-balanced
> name, which draws addresses from the same pool in all three cases,
> but any one lookup will often yield slightly different addresses.
>
> See:
>
> http://postfix.1071664.n5.nabble.com/Balancing-destination-concurrency-rate-delay-td54147.html
> http://postfix.1071664.n5.nabble.com/dealing-with-Yahoo-slowness-td35340.html
>
> --
>         Viktor.


Thank you, Viktor, sincerely. There's nothing better than to have one
recommend something, and then also showing the commands behind it and
a good explanation of why and how. Perfect!

I haven't given up on Exim as of yet, but your solution is a
workaround I can live with myself. I just couldn't explain it to my
customer as of now. =(

--
Regards
Charlie Elgholm
Brightly AB