Re: [exim] Rate Limiting?

Top Page
Delete this message
Reply to this message
Author: Ian FREISLICH
Date:  
To: torsten
CC: exim-users
Subject: Re: [exim] Rate Limiting?
wrote:
> Dear all,
>
> is rate limiting a part of Exim in the meanwhile?
>
> If not, did anyone implement such a thing and would be willing to share
> his / her code?
>
> I found an old e-mail in the archives (dated 1999) where someone said he'd
> written some perl script to keep analyzing the Exim mainlog for that.


I wrote an embedded perl function to do rate limiting (I'm not sure
if a ${run ..} expansion is cheaper or more expensive than ${perl
...}, although for multiple reciepts you only have to link in perl
once). It uses a circular buffer of timestamps to calculate the
rate. The number of items in the buffer and the elapsed time between
the current insertion and the tail of the buffer is used to calculate
the rate. It supports resizing of the buffer.

The arugments are:
${perl {rate_limit}{key}{number}{time}}
key: some value for referencing the buffer.
number: the length of the buffer.
time: the threshold.

Return values: "yes" for rate limit exceeded. "no" for any other
condition.

${perl {rate_limit}{foo}{100}{30}}
Will check that key foo against a limit of 100 in 30 seconds.

/etc/exim/rates must exist and be writable by the exim user.

Here's my ACL fragment for pipes to /usr/bin/sendmail:

acl_not_smtp:
  #Rate limiting
  warn     log_message  = local rate limit exceeded.
           condition    = USE_RATE_LIMIT
           condition    = ${perl {rate_limit}{local} \
                {RATE_LIMIT_RECPIENTS}{RATE_LIMIT_PERIOD}}
           control      = freeze


accept

USE_RATE_LIMIT is a knob to turn the feature on or off. It starts
freezing messages in the queue if more messages (or recipients
depending on where you run the sprocedure) than RATE_LIMIT_RECPIENTS
are recieved in RATE_LIMIT_PERIOD seconds.

It freezes so we have evidence if we want to suspend or terminate
the account.

Ian

--
Ian Freislich


# Copyright 2005, Hetzner Africa.  All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND ITS EMPLOYEES
# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT
# HOLDER OR EMPLOYEES BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


use strict;

sub rate_limit
{
    use Fcntl ':flock';


    my ($key, $length, $delay) = @_;
    my $now = time();


    if (!open (FILE, "+</etc/exim/rates/$key")) {
        #file does not exist
        open (FILE, "+>/etc/exim/rates/$key.$$") || return("no");
        printf FILE "%08d %08d\n", $length, 0;
        for (my $i = 0; $i < $length; $i++) {
            printf FILE "%012d\n", $i;
        }
        if (!link("/etc/exim/rates/$key.$$", "/etc/exim/rates/$key")) {
            unlink("/etc/exim/rates/$key.$$");
            return("no");
        }
        unlink("/etc/exim/rates/$key.$$");
    }
    flock(FILE,LOCK_EX);
    seek(FILE, 0, 0);
    my $line = <FILE>;
    my $start = tell(FILE);
    chomp($line);
    my ($len, $offset) = split(/\s+/, $line);
    if ($len != $length) {
        # resize the buffer: it grows gracefully, but needs to be
        # forcefully shrunk
        if ($length < $len) {
            my @data = (<FILE>);
            seek(FILE, $start, 0);
            for (my $i = $offset, my $a = $length; $a--; $i++) {
                $i = 0 if ($i == $len);
                printf FILE "%012d\n", $data[$i];
            }
            truncate(FILE, $start + 13 * $length);
            $offset = 0;
        }
    }
    die "Mangled offset in rate-limit" if ($offset < 0 || $offset >= $len);


    seek(FILE, $start + 13 * $offset, 0);
    my $last = <FILE>;
    chomp($last);
    seek(FILE, $start + 13 * $offset, 0);
    printf FILE "%012d\n", $now;
    $offset++;
    $offset = 0 if ($offset == $length);
    seek(FILE, 0, 0);
    printf FILE "%08d %08d\n", $length, $offset;
    close(FILE);
    if ($now - $last <= $delay) {
        return("yes");
    }
    else {
        return("no");
    }
}