Tony Finch wrote:
> I have been working on some improvements to Exim's ratelimit ACL
> condition. I think it basically works, so I thought it would be helpful to
> get some feedback on it before the next release. Here's the draft of the
> NewStuff entry, and the patch is after my .sig:
>
> The code now checks that the per_* options are not used in ACLs where it
> does not make sense, for example using per_mail in acl_smtp_helo or any
> other time when Exim is not handling a message.
>
> The /leaky, /strict, and /noupdate options are now alternatives.
> (Previously the /noupdate option was in addition to /strict or /leaky.)
> Also, the update mode is no longer recorded in the database, which may
> cause clashes if you are using /leaky and /strict with the same key.
>
> The first new feature is the /count= option. This is a generalization
> of the per_byte option, so that you can measure the throughput of other
> aggregate values. For example, the /per_byte option is now equivalent
> to /per_mail/count=${if <{0}{$message_size} {0} {$message_size} }, and
> when it isn't used in acl_smtp_rcpt, the per_rcpt option is equivalent
> to /per_mail/count=$recipients_count.
>
> The other new feature is a mechanism for counting the rate of unique
> events. The new /per_addr option counts the number of different
> recipients that someone has sent messages to in the last time period.
> Like the /count= option this is a general mechanism, so the /per_addr
> option is equivalent to /per_rcpt/unique=$local_part@$domain. You can,
> for example, measure the rate that a client uses different sender
> addresses with the options /per_mail/unique=$sender_address.
>
> For each ratelimit key Exim stores the set of /unique= values that it
> has seen for that key. The whole set is thrown away when it is older
> than the rate smoothing period, so each different event is counted at
> most once per period. In /leaky mode, an event that causes the client
> to go over the limit is not added to the set, in the same way that the
> client's recorded rate is not updated in the same situation.
>
> The /unique= mechanism needs more space in the ratelimit database than
> the other ratelimit options in order to store the event set. The number
> of unique values is potentially as large as the rate limit, so the
> extra space required increases with larger limits.
>
> The uniqueification is not perfect: there is a small probability that a
> new event will appear to have happened before. For rates less than the
> limit it is more than 99.9% correct. However in /strict mode the
> measured rate can go above the limit, and this can cause Exim to under-
> count events by a significant margin. Fortunately, if the rate is high
> enough (2.7 times the limit) that the false positive rate goes above
> 9%, then the over-full event set will be thrown away before the
> measured rate falls below the limit. Therefore the only harm should be
> that exceptionally high sending rates are logged incorrectly; any
> countermeasures you configure will be as effective as intended.
>
> The exim_dumpdb utility does not display /unique= event sets because they
> are represented in a form that makes a human-readable representation
> impossible. However you can use exim_fixdb to test membership of a set or
> to add events to it.
>
> Tony.
Is there | will there be, a [more] sophisticated matching debug feature
of the sort that says WTTE:
'current ratelimit rules can be expected to <do whatever they do at [set
points]>
... in Peter-Rabbit English?
i.e. - extrapolating from the config, not needing a test-suite to be run
for 'tuning'.
ELSE .. this could become (is already?) a bit of a foot-shooter in less
experienced hands, as it is not easy to grok as-is [1].
Otherwise - looks to be quite useful wanker-curbing for ISP's and other
shared-service providers.
Thanks,
Bill
[1] Given that this one feature is perhaps more complex than some entire
smtpd's.
;-)
