[exim] Fwd: Rejecting over quota at RCPT time - revisited

Góra strony
Delete this message
Reply to this message
Autor: Nikita Koshikov
Data:  
Dla: exim-users
Temat: [exim] Fwd: Rejecting over quota at RCPT time - revisited
I have used perl for many years for current task. Here is my code, one of
it's task is to check smtp time RCPT TO. There are 3 different functions -
one to calculate quota from maildir file, another one to use - doveadm
exec() logic - it's bad and slow. And final one - to use dovecot internal
protocol to obtain current quota. Also I did not honour current message
size - the logic - if we can't deliver message_size_limit - user is over
quota. The function can be written for any storage engine and quota
mechanism.

#!/usr/bin/perl -w

use Sys::Syslog;
use Data::Dumper;
use Socket;
use IO::Handle;
use strict;

my %conf = (
        'debug'                                 => '1',
        'debug_prefix'                  => 'perl',
        'debug_delimeter'               => ' -> ',
        'quota_mail_path'               => '/data/mail',
        'quota_control'                 => 'data',
        'quota_file'                    => 'maildirsize',
        'quota_get_cmd'                 => '/usr/bin/doveadm -f tab quota
get -u ',
        'quota_socket'                  =>
'/var/run/dovecot/doveadm-server',
        'doveadm_password'              => 'AGRvdmVhZG0AbnVvSGVpSjk=',
        'message_size_limit'    => '1'


);

sub set_conf {
    my $key = shift;
        my $value = shift;


    if (defined $conf{$key}) {
        $conf{$key} = $value;
                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Set config: $key=$value\n");
                return 0;
    }


    die "Unknown option:$key";


}
sub check_maildir_quota{
        my $local_part = shift;
        my $domain = shift;


        my
$full_path="$conf{'quota_mail_path'}/$domain/$local_part/$conf{'quota_control'}/$conf{'quota_file'}";


        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Got: local_part=$local_part domain=$domain\n");
        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Quota file for $local_part\@$domain is $full_path"."\n");


        #Check file with quota exists, if not - there can be first delivery
- so trying to store message. If yes - begining to count.
        if (not stat "$full_path"){


Exim::log_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Can't find quota file for $local_part\@$domain -->
$full_path");
                return "yes";
        }else{
                open(QOUTA,'<',$full_path) or die "$!";
                my $quota_limit = <QOUTA>;
                my $current_quota = 0;
                my $message_size_from_quota=0;
                ($quota_limit) = split(/\s+/,"$quota_limit");
                $quota_limit =~ s/[A-Za-z]//;


                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Quota limit for $local_part\@$domain is
$quota_limit"."\n");


                #If quota is unlimited. 0 - means this
                if ( $quota_limit == 0){
                        close(QOUTA);
                        return "yes";
                }


                #Count current quota
                while (<QOUTA>){
                        chomp;
                        ($message_size_from_quota) = split(/\s+/);
                        $current_quota += $message_size_from_quota;
                }


                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Current quota size for $local_part\@$domain is
$current_quota"."\n");


                close(QOUTA);


                #Adding to quota max message size(message_size_limit)
                $current_quota += $conf{'message_size_limit'};


                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Current quota size for $local_part\@$domain with
max message is $current_quota"."\n");


                #Returning to exim our decision.
                if ($current_quota >= $quota_limit){
                        return "no";
                }else{
                        return "yes";
                }
        }
}


#Parsing
#Quota name=Mailbox quota Type=STORAGE Value=608908 Limit=1048576 %=58
#Quota name=Mailbox quota Type=MESSAGE Value=17727 Limit=- %=0
sub check_doveadm_quota {
        my $local_part = shift;
        my $domain = shift;



        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Got: local_part=$local_part domain=$domain\n");


        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Quota check command:
$conf{'quota_get_cmd'}$local_part\@$domain\n");


        my @quota_data=`$conf{'quota_get_cmd'} $local_part\@$domain`;


        #Everything is OK?
        if ($? != 0) {


Exim::log_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Error running doveadm for
user=$local_part\@$domain: $!\n");
                #Not ok - but trying to deliver
                return "yes";
        }


        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Doveadm returns:\n @quota_data");


        my (@current)=split(/\t/,"$quota_data[1]");
        my $quota_limit = $current[3];
        my $current_quota = $current[2];


        #Quota is unlimited
        if ( $quota_limit == '-'){
                $quota_limit='unlimited';
                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Current quota is $current_quota. Max is
$quota_limit\n");
                return "yes";
        }
        #Cast to bytes
        $quota_limit*=1024;
        $current_quota*=1024;
        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Current quota is $current_quota. Max is $quota_limit\n");


        #Adding max message size(message_size_limit)
        $current_quota += $conf{'message_size_limit'};


        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Current quota size with max message is
$current_quota"."\n");


        #Returning to exim
        if ($current_quota >= $quota_limit){
                return "no";
        }else{
                return "yes";
        }


}

#Function uses internal dovecot protocol to obtain data from socker
doveadm-a.
#doveadm_password must be set for auth.
#We use base64 coding value = user:"doveadm" pass:value from dovecot.conf
sub check_socket_quota {

        my $local_part = shift;
        my $domain = shift;


        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Got: local_part=$local_part domain=$domain \n");
        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                "Got: socket=$conf{'quota_socket'}
pass=$conf{'doveadm_password'}\n");


        socket(TSOCK, PF_UNIX, SOCK_STREAM,0);
        connect(TSOCK, sockaddr_un("$conf{quota_socket}"));


        if ($? != 0) {


Exim::log_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Error connecting to $conf{'quota_socket'}: $@");
        }


        #After connection to socker, dovecot returns "+" или "-". Simple
check.
        if (defined(my $answer = <TSOCK>)) {


                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Debug ANSWER:\n$answer");
                print TSOCK "VERSION\tdoveadm-server\t1\t0\n";
                print TSOCK "PLAIN\t$conf{'doveadm_password'}\n";
                TSOCK->flush;
                #'+' here
                $answer=<TSOCK>;
                print TSOCK "\t$local_part\@$domain\tquota get\n";
                TSOCK->flush;


                my $quota_data = <TSOCK>;


                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Dovecot returned:\n$quota_data");


                #"+" here
                $answer = <TSOCK>;


                close TSOCK;
                my (@current)=split(/\t/,"$quota_data");
                my $quota_limit = $current[3];
                my $current_quota = $current[2];


                #Quota is unlimited
                if ( $quota_limit eq '-'){
                        $quota_limit='unlimited';
                        $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                                "Current quota is $current_quota. Max is
$quota_limit\n");
                        return "yes";
                }
                #Cast to bytes
                $quota_limit*=1024;
                $current_quota*=1024;
                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Current quota is $current_quota. Max is
$quota_limit\n");


                #Adding max message size(message_size_limit)
                $current_quota += $conf{'message_size_limit'};


                $conf{'debug'} &&
Exim::debug_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "Current quota size with max message is
$current_quota"."\n");


                #Returning to exim
                if ($current_quota >= $quota_limit){
                        return "no";
                }else{
                        return "yes";
                }
        } else {


Exim::log_write($conf{'debug_prefix'}.$conf{'debug_delimeter'}.(caller(0))[3].$conf{'debug_delimeter'}.
                        "No data avaliable after connecting to
$conf{'quota_socket'}. Check dovecot service");
                #Default police - trying to deliver message
                return "yes";
        }


}


In exim.conf you should set
MESSAGE_SIZE_LIMIT=31457280
perl_startup = do 'EXIM_ROOT/perl/exim_helper.pl';\
                        set_conf('debug','0');\
                        set_conf('message_size_limit','MESSAGE_SIZE_LIMIT');
perl_at_start


Router section:
quota_check:
    driver = redirect
    domains = +local_domains
    allow_defer
    allow_fail
        #condition = ${if eq{1}{0}}
    condition = ${if \
                eq{${perl{check_socket_quota}{$local_part}{$domain}}} {no}\
            }
    data = :fail: Account is under quota



I have used all 3 functions in the past - the last is speed champion. I
used it now. My setup is not so big - about 800 active users and ~20k
message deliveres per day(+15k rejects). Perl is working fine here. Also
debug=1 is valuable for testing and troubleshuting.


On Thu, Nov 29, 2012 at 5:47 AM, Todd Lyons <tlyons@???> wrote:

> Tomorrow, I will be experimenting with jamming the following into a
> perl function (I use the embedded perl a lot) and call it as a
> condition for a verify_only router, and then defer with an appropriate
> quota related message is customer is already over quota to stop
> further incoming emails from clogging the queue.
>
>
> use strict;
> use warnings;
> # Prints out 0 if over quota (means "not ok")
> # Prints out 1 if not over quota
> my $local_part = shift();
> my $domain     = shift();
> print_and_exit(1) if (!defined $local_part || !defined $domain);
> # We use hashed subdirectories for mailstores
> my $home = "/netapp3/mail/maildirs";
> (my $tmp = $domain) =~ s/^(...).*/$1/;
> foreach my $char (split(//,$tmp)) {
>   $home .= '/';
>   $home .= $char =~ /\w/ ? $char : '_';
> }

>
> my $file="$home/$domain/$local_part/Maildir/maildirsize";
> open(my $fh, "<", $file) or print_and_exit(1);
> my @lines = <$fh>;
> close($fh);
> print_and_exit(1) if (scalar @lines == 0);
>
> my ($quota,$qcount);  # count is calculated, but ignored
> my @quota = split(/,/,$lines[0]);
> if (scalar @quota > 0) {
>   $quota  = substr($quota[0],0,-1)
>     if (substr($quota[0],-1) eq 'S');
>   $qcount = substr($quota[0],0,-1)
>     if (substr($quota[0],-1) eq 'C');
> }
> print_and_exit(1) if (!defined $quota);

>
> my $line=my $size=my $msgs=0;
> do {
> $line++;
> (my $msgsize, my $msgcount) = split(" ", $lines[$line]);
> $size+=$msgsize; $msgs+=$msgcount;
> } while ($line < $#lines);
>
> if ($size > $quota) {
> print_and_exit(0); # Over quota!
> } else {
> print_and_exit(1); # Still has room
> }
>
> sub print_and_exit {
> my $rv = shift() || 0;
> print $rv, "\n"; # Need the newline?
> exit;
> }
>
>
> Expanded upon original work at
>
> http://www.kutukupret.com/2011/05/18/check-disk-quota-usage-by-parsing-maildirsize/
>
> ...Todd
>
>
> On Wed, Nov 28, 2012 at 6:22 PM, Robert Blayzor <rblayzor.bulk@???>
> wrote:
> > On Nov 28, 2012, at 8:32 PM, Jeremy Harris <jgh@???> wrote:
> >> The "quota" option on an appendfile transport is an expanded string;
> >> you can basically do what you want with it given a bit of creativity.
> >
> >
> > Well, yes, it is. But that string is expanded regardless if the user is
> over quota or not. The idea is to trap the over quota condition and act on
> it.
> >
> > Perhaps another string thats only expanded only if the over quota
> condition happens.
> >
> > --
> > Robert Blayzor
> > INOC, LLC
> > rblayzor@???
> > http://www.inoc.net/~rblayzor/
> >
> >
> >
> >
> > --
> > ## List details at https://lists.exim.org/mailman/listinfo/exim-users
> > ## Exim details at http://www.exim.org/
> > ## Please use the Wiki with this list - http://wiki.exim.org/
>
>
>
> --
> The total budget at all receivers for solving senders' problems is $0.
> If you want them to accept your mail and manage it the way you want,
> send it the way the spec says to. --John Levine
>
> --
> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
> ## Exim details at http://www.exim.org/
> ## Please use the Wiki with this list - http://wiki.exim.org/
>