[exim-cvs] cvs commit: exim/exim-doc/doc-txt ChangeLog NewSt…

Góra strony
Delete this message
Reply to this message
Autor: Tony Finch
Data:  
Dla: exim-cvs
Temat: [exim-cvs] cvs commit: exim/exim-doc/doc-txt ChangeLog NewStuff exim/exim-src/util README ratelimit.pl
fanf2 2005/09/13 18:51:06 BST

  Modified files:
    exim-doc/doc-txt     ChangeLog NewStuff 
    exim-src/util        README 
  Added files:
    exim-src/util        ratelimit.pl 
  Log:
  Add util/ratelimit.pl to assist with choosing ratelimit settings.


  Revision  Changes    Path
  1.230     +2 -0      exim/exim-doc/doc-txt/ChangeLog
  1.70      +4 -0      exim/exim-doc/doc-txt/NewStuff
  1.2       +7 -0      exim/exim-src/util/README
  1.1       +137 -0    exim/exim-src/util/ratelimit.pl (new)


  Index: ChangeLog
  ===================================================================
  RCS file: /home/cvs/exim/exim-doc/doc-txt/ChangeLog,v
  retrieving revision 1.229
  retrieving revision 1.230
  diff -u -u -r1.229 -r1.230
  --- ChangeLog    13 Sep 2005 15:40:07 -0000    1.229
  +++ ChangeLog    13 Sep 2005 17:51:05 -0000    1.230
  @@ -1,4 +1,4 @@
  -$Cambridge: exim/exim-doc/doc-txt/ChangeLog,v 1.229 2005/09/13 15:40:07 ph10 Exp $
  +$Cambridge: exim/exim-doc/doc-txt/ChangeLog,v 1.230 2005/09/13 17:51:05 fanf2 Exp $


   Change log file for Exim from version 4.21
   -------------------------------------------
  @@ -187,6 +187,8 @@
         avoiding the need to install special zones in a real server. This is
         backwards compatible; if it can't find the fake resolver, it drops back.
         Thus, both old and new test suites can be run.
  +
  +TF/02 Added util/ratelimit.pl



Exim version 4.52

  Index: NewStuff
  ===================================================================
  RCS file: /home/cvs/exim/exim-doc/doc-txt/NewStuff,v
  retrieving revision 1.69
  retrieving revision 1.70
  diff -u -u -r1.69 -r1.70
  --- NewStuff    13 Sep 2005 11:13:27 -0000    1.69
  +++ NewStuff    13 Sep 2005 17:51:05 -0000    1.70
  @@ -1,4 +1,4 @@
  -$Cambridge: exim/exim-doc/doc-txt/NewStuff,v 1.69 2005/09/13 11:13:27 ph10 Exp $
  +$Cambridge: exim/exim-doc/doc-txt/NewStuff,v 1.70 2005/09/13 17:51:05 fanf2 Exp $


   New Features in Exim
   --------------------
  @@ -150,6 +150,10 @@
         command (compare $smtp_command_argument - see PH/07 above). This makes it
         possible to distinguish between HELO and EHLO, and also between things
         like "MAIL FROM:<>" and "MAIL FROM: <>".
  +
  +TF/01 There's a new script in util/ratelimit.pl which extracts sending
  +      rates from log files, to assist with choosing appropriate settings
  +      when deploying the ratelimit ACL condition.



Exim version 4.52

Index: ratelimit.pl
====================================================================
#!/usr/bin/perl -wT
#
# $Cambridge: exim/exim-src/util/ratelimit.pl,v 1.1 2005/09/13 17:51:06 fanf2 Exp $

use strict;

  sub usage () {
    print <<END;
  usage: ratelimit.pl <period> <regex> logfile


The aim of this script is to compute clients' peak sending rates
from an Exim log file, using the same formula as Exim's ratelimit
ACL condition. This is so that you can get an idea of a reasonable
limit setting before you deploy the restrictions.

This script isn't perfectly accurate, because the time stamps in
Exim's log files are only accurate to a seond whereas internally Exim
computes sender rates to the accuracy of your computer's clock
(typically 10ms).

The log files to be processed can be specified on the command line
after the other arguments; if no filenames are specified the script
will read from stdin.

The first command line argument is the smoothing period, as defined by
the documentation for the ratelimit ACL condition. The second argumetn
is a regular expression.

Each line is matched against the regular expression. Lines that do not
match are ignored. The regex may contain 0, 1, or 2 () capturing
sub-expressions.

If there are no () sub-expressions, then every line that matches is
used to compute a single rate. Its maximum value is reported when the
script finishes.

If there is one () sub-expression, then the text matched by the
sub-expression is used to identify a rate lookup key, similar to the
lookup key used by the ratelimit ACL condition. For example, you might
write a regex to match the client IP address, or the authenticated
username. Separate rates are computed for each different client and
the maximum rate for each client is reported when the script finishes.

  If there are two () sub-expressions, then the text matched by the
  first sub-expression is used to identify a rate lookup key as above,
  and the second is used to match the message size recorded in the log
  line, e.g. " S=(\\d+) ". In this case the byte rate is computed instead
  of the message rate, similar to the per_byte option of the ratelimit
  ACL condition.
  END
    exit 1;
  }


  sub iso2unix (@) {
    my ($y,$m,$d,$H,$M,$S,$zs,$zh,$zm) = @_;
    use integer;
    $y -= $m < 3;
    $m += $m < 3 ? 10 : -2;
    my $z = defined $zs ? "${zs}1" * ($zh * 60 + $zm) : 0;
    my $t = $y/400 - $y/100 + $y/4 + $y*365
          + $m*367/12 + $d - 719499;
    return $t * 86400
         + $H * 3600
         + $M * 60
         + $S
         - $z;
  }


  my $debug = 0;
  my $progress = 0;
  while (@ARGV && $ARGV[0] =~ /^-\w+$/) {
    $debug = 1    if $ARGV[0] =~ s/(-\w*)d(\w*)/$1$2/;
    $progress = 1 if $ARGV[0] =~ s/(-\w*)p(\w*)/$1$2/;
    shift if $ARGV[0] eq "-";
  }


usage if @ARGV < 2;

my $progtime = "";

my $period = shift;

my $re_txt = shift;
my $re = qr{$re_txt}o;

my %time;
my %rate;
my %max;

  sub debug ($) {
    my $key = shift;
    printf "%s\t%12d %8s %5.2f %5.2f\n",
      $_, $time{$key}, $key, $max{$key}, $rate{$key};
  }


  while (<>) {
    next unless $_ =~ $re;
    my $key = $1 || "";
    my $size = $2 || 1.0;
    my $time = iso2unix
      ($_ =~ m{^(\d{4})-(\d\d)-(\d\d)[ ]
                (\d\d):(\d\d):(\d\d)[ ]
                (?:([+-])(\d\d)(\d\d)[ ])?
              }x);
    if ($progress) {
      my $prog_now = substr $_, 0, 14;
      if ($progtime ne $prog_now) {
        $progtime = $prog_now;
        print "$progtime\n";
      }
    }
    if (not defined $time{$key}) {
      $time{$key} = $time;
      $rate{$key} = 0.0;
      $max{$key} = 0.0;
      debug $key if $debug;
      next;
    }
    # see acl_ratelimit() for details of the following
    my $interval = $time - $time{$key};
    my $i_over_p = $interval / $period;
    my $a = exp(-$i_over_p);
    $i_over_p = 1e-9 if $i_over_p <= 0.0;
    $time{$key} = $time;
    $rate{$key} = $size * (1.0 - $a) / $i_over_p + $a * $rate{$key};
    $max{$key} = $rate{$key} if $rate{$key} > $max{$key};
    debug $key if $debug;
  }


  print map {
    " " x (20 - length) .
    "$_ : $max{$_}\n"
  } sort {
    $max{$a} <=> $max{$b}
  } keys %max;


# eof

  Index: README
  ===================================================================
  RCS file: /home/cvs/exim/exim-src/util/README,v
  retrieving revision 1.1
  retrieving revision 1.2
  diff -u -u -r1.1 -r1.2
  --- README    24 May 2005 11:20:40 -0000    1.1
  +++ README    13 Sep 2005 17:51:06 -0000    1.2
  @@ -1,4 +1,4 @@
  -$Cambridge: exim/exim-src/util/README,v 1.1 2005/05/24 11:20:40 ph10 Exp $
  +$Cambridge: exim/exim-src/util/README,v 1.2 2005/09/13 17:51:06 fanf2 Exp $


The "util" directory in the Exim distribution
---------------------------------------------
@@ -24,6 +24,13 @@
A Perl script for a converting flat file into a format that is suitable for
processing by cdbmake into a cdb file. It has some advantages over the
cdbmake-12 awk script.
+
+ratelimit.pl
+------------
+
+A Perl script for computing peak sending rates from a log file. This is for
+use with the ratelimit ACL condition, so that you can get some idea of what a
+reasonable limit would be before deploying the feature.

unknownuser.sh
--------------