Hello,
There's an e-mail below that was submitted to the list sometimes back. I am testing the stuff this
gentleman gave but it seems that something isn't correct with my filter, which is a mod
of what he had originally. When I analyze the logs with the perl script, the output is such
that I do not get any counts on e-mails received by the domains in question. Here is a sample
output of the analysis:
[snip]
Results - 03-10-2001 - 04-10-2001 for All Domains
Email Address # in KBytes in # out KBytes out
adeptsystems.co.ke 0 0 K 20 1 M 572 K
1234 0 0 K 8 1 M 356 K
eval 0 0 K 1 3 K
kshadmte 0 0 K 4 174 K
naftalie 0 0 K 7 38 K
africaninfluence.com 0 0 K 2 164 K
infoinfluence 0 0 K 2 164 K
artemiskenya.com 0 0 K 14 22 K
ckirituru 0 0 K 13 21 K
jmburiuki 0 0 K 1 1 K
barotagencies.com 0 0 K 6 45 K
vijitesh 0 0 K 6 45 K
[/snip]
Okay, spammers wouldn't reap from those addresses because they are thoroughly mangled ;)
Now what I think is illogical is the fact that the KBytes in is all zero for all the domains.
I'd like to attribute that to my filter because even in the log file that this is generated
from I don't see any stats for incoming mail. My filter is below:
# Do some logging (under test)
if ("${lookup {$sender_address_domain} partial-lsearch*@{/usr/local/etc/exim/relay} {1}}" is 1 or
"${lookup {$sender_address_domain} partial-lsearch*@{/usr/local/etc/exim/static} {1}}" is 1)
then
logfile /var/log/exim/domainemail 0644
log "$tod_log $sender_address $message_size >> $recipients_count"
endif
if foranyaddress $header_to ("${lookup {$thisaddress} partial-lsearch*@{/usr/local/etc/exim/relay} {1}}" is 1 or
"${lookup {$thisaddress} partial-lsearch*@{/usr/local/etc/exim/static} {1}}" is 1)
then
logfile /var/log/exim/domainemail 0644
log "$tod_log $thisaddress $message_size << "
endif
#
It seems that the second portion of the filter is a dead duck! Or maybe I need to direct that to a different
logfile?? What I'd like help with right now is if someone can help me verify that the second filter will do
something meanigful.
The original craft from Andrew had $recipient instead of $header_to but I changed that since I did not find
the variable $recipient in the filter specs. I also thought the substitution would not change the original
intended goal.
The other thing that is worth mentioning is that I have receiver_verify ON and sender_verify OFF (this is for
some reason) while Andrew had both of those ON.
Thanks in advance.
#################### Andrew's original e-mail is below ##############################
----- Forwarded message from Andrew D <awd@???> -----
From: "Andrew D" <awd@???>
To: "Exim users list" <exim-users@???>
Date: Sat, 28 Jul 2001 16:16:30 +0930
Subject: [Exim] Domain Logging
X-Mailer: Microsoft Outlook Express 5.50.4522.1200
G'Day all,
The attachment contains a perl script as well as a system filter component
for logging of domain specific email statistics (ie number and size in and
out per user).
It produces 3 types of reports : HTML, Text, or in a form that could be
imported into a database.
It can also email in any of the 3 formats.
I have been using the script with no problems for a couple of weeks on Suse
linux 6.2, perl 5.6 and running exim 3.22 at home and at work the only diff
is its Suse 7.1.
The only thing is that I have receiver_verify and sender_verify
turned ON. If you don't, then you may see users listed that aren't actually
in that domain.
Please forgive the hacked up perl, though it does seem to run pretty
quick :)
if u feel like cleaning it up, ur welcome to, just send me a copy plz :)
Also any suggestions welcome.
Thanks for your time, and have a top day/night.
cya
Andrew
This file contains a perl script as well as a system filter component
for logging of domain email statistics.
It produces 3 types of reports
HTML
TEXT
or in a form to import into a database type thingo
it can email in any of the 3 formats
optional command line options
-add=<email address> address to email to for the email option
-dom=<domain> domain to show
-td show todays only
-html html
-hum human readable
The default is the database input, though u can change the default
You probably will have to change some paths though.
#!/usr/bin/perl -w
#
# Script to parse mail domain logs to produce a report
# for incoming and outgoing email
# written by Andrew Duxorth 2001 awd@???
# log file to parse
$LOG = "/var/log/exim/domainemail";
# default email address
$emailadd = "postmaster";
# change to 1 to make the default report html
$htmld = 0;
# path to exim binary
$EXIM = "/usr/exim/bin/exim";
# character to use for database import
$DBCHAR=":";
# optional command line options
# -add=<email address> address to email to for the email option
# -dom=<domain> domain to show
# -td show todays only
# -html html
# -hum human readable
#
## No need to change anything below this line
###############################
# the subs
#
sub getinfo {
open (STAT , "$LOG") or die "could not find or open file $LOG : $!";
my @N_STAT = <STAT>;
close (STAT);
foreach $line (@N_STAT) {
($date,$a,$email,$size,$direct,$number)=split " ",$line ;
$email = lc $email;
if ($number eq ""){$number=1;}
$size=$size*$number;
if ($date eq ""||(!($date =~ /$today/))||(!($email =~ /$dom/))) {next;}
$dated=$date;$dated=~s/\-//g;
if ($dated < $dateoldest){$dateoldest = $dated;}
$num=0; $found=0; $numberin=0;
$sizein=0; $numberout=0; $sizeout=0;
foreach (@users){
local ($domain,$user,$numberin,$sizein,$numberout,$sizeout)=split ":",$users[$num];
if ($user eq $email){
$found=1;
if ($direct eq ">>"){$numberout += $number; $sizeout += $size;}
else {$numberin += 1; $sizein += $size;}
$users[$num]= "$domain:$user:$numberin:$sizein:$numberout:$sizeout:\n";
}
$num=$num+1;
}
if ($found==0){
if ($direct eq ">>"){$numberout += $number;$sizeout += $size;
} else {$numberin += 1;$sizein += $size;
}
my ($blah,$domain) = split "\@",$email;
$users[$num]= "$domain:$email:$numberin:$sizein:$numberout:$sizeout:\n";
}
}
@users = sort (@users);
if ((!$htmld) && (!$hum)){ foreach $line (@users){
$line =~ s/:/$DBCHAR/g;
print "$line";};}
}
###########################
sub printr {
my $oldomain="";
my $dateold = substr $dateoldest,0,4;
my $dateold1 = substr $dateoldest,4,2;
my $dateold2 = substr $dateoldest,6,2;
$slat ="";
$dateoldest = "$dateold2-$dateold1-$dateold";
if ($today ne "-"){$dates ="$mday-$mon-$year";}else{ $dates= "$dateoldest - $mday-$mon-$year";}
if ($dom eq ".") {$dom = "All Domains";}
if ($htmld){$hbody="<BODY><H2><center>"; $htab="</center></H2><table border=1 align=center>";
$htab1="</table>"; $htd ="</TD><TD>"; $htr ="<TR><TD>"; $htr2 = "</TD></TR>";
$htr3 = "<TR align=center><TD>"; $hh ="<H3>"; $hh1 ="</H3>";
$hc ="Content-Type: text/html\n";$hcom="<!--";$hcom2="-->";}
if ($hum) {$hbody="\n"; $htab="\n";
$htab1=""; $htd =" \t"; $htr =""; $htr2 = "";
$htr3 = ""; $hh =""; $hh1 ="";$hc ="";$hcom="";$hcom2="";
$slat="\t";
}
if ($emaild) {
$dev = "SENDMAIL";
open ($dev,"|$EXIM $emailadd");
print $dev "Subject: $today2 Results E-Mail - $dates for $dom \n$hc\n";
} else {$dev = STDOUT;}
@command = sort (@users);
print $dev "$hbody $today2 Results - $dates for $dom$htab\n";
print $dev "$htr Email Address$slat$htd # in$htd KBytes in$htd # out$htd KBytes out$htr2\n";
local $blah="";
foreach $line (@command){
$line =~ s/\n//g;
my ($domain,$email,$numin,$datain,$numout,$dataout)= split ":",$line;
if ($oldomain eq "") {$oldomain=$domain;}
if ($line =~ /^#/) {next};
($email) = split "\@",$email;
if ($domain ne $oldomain) {
my $tind= convert($tin);
$tind=newlength($tind,1);
my $toutd= convert($tout);
my $dataoutd=convert($dataout);
my $dataind=convert($datain);
$dataind=newlength($dataind,1);
$oldomain=newlength($oldomain,2);
print $dev "\n$htr3$hh$oldomain$hh1$htd$nin$htd$tind$htd$nout$htd$toutd$htr2\n";
print $dev $blah;
$emaile = newlength($email,3);
$tin=$datain;$nin=$numin;$tout=$dataout;$nout=$numout;
$blah ="$htr3$emaile$htd$numin$htd$dataind$htd$numout$htd$dataoutd$htr2\n";
} else {
$dataoutd=convert($dataout);
$dataind=convert($datain);
$tin+=$datain;$nin+=$numin;$tout+=$dataout;$nout+=$numout;
$dataind=newlength($dataind,1);
$emaile = newlength($email,3);
if ($email){ $blah .="$htr3$emaile$htd$numin$htd$dataind$htd$numout$htd";
$blah .="$dataoutd$htr2 \n";
}
}
$oldomain = $domain;
}
$tind= convert($tin);
$tind=newlength($tind,1);
$toutd= convert($tout);
$oldomain=newlength($oldomain,2);
print $dev "\n$htr3$hh$oldomain$hh1$htd$nin$htd$tind$htd$nout$htd$toutd$htr2 \n";
print $dev $blah;
print $dev "$htab1\n";
($us,$sy) = times;
$a=$us+$sy;
print $dev "$hcom It took $a seconds to create this report $hcom2\n";
}
###################
sub convert {
local ($b)=@_;
local $dat = int $b/1024;
if ($dat > 1024) {
$mb= int $dat/1024;
$kb= $dat -($mb *1024);
$dat = "$mb M $kb ";
}
return "$dat K";
}
##################
sub newlength{
local ($b,$c)=@_;
if ($c eq 1){ if (length $b < 8){$b .="$slat";return $b;}else {return $b;}}
if (length $b < 8){$b .="$slat$slat";}
elsif (length $b < 16){$b .="$slat";}
return $b;
}
########################################
# The script starts here
($mday,$mon,$year) = (localtime(time))[3,4,5];
$year +=1900;
$mon=$mon+1;
if (length $mday==1){$mday="0$mday";}
if (length $mon==1){$mon="0$mon";}
$dom=".";$dateoldest="22009999";
$today="-";$today2="";
foreach $_ (@ARGV){
if ($_ =~ /^(-|--)(h|help)$/){
print STDERR "$0 Options: \n\t-td\t\t\ttodays email report\n\t";
print STDERR "-html\t\t\thtml format report\n\t-email\t\t\tsend report via email\n\t";
print STDERR "-dom=<domain>\t\tDomain to search for\n\t-add=<email address>\tEmail address to send to\n";
exit 1;}
if ($_ =~ /^-email$/){ $emaild=1;}
if ($_ =~ /^-html$/){ $htmld=1;}
if ($_ =~ /^-hum$/){ $hum=1;}
if ($_ =~ /^-td$/){ $today="$year-$mon-$mday";$today2="Daily"}
if ($_ =~ /^-dom=/){($a,$dom)=split "=", $_;}
if ($_ =~ /^-add=/){($a,$emailadd)=split "=", $_;}
}
if (($htmld) && ($hum)) { print STDERR "error incompatiable options\n\n";exit 1;}
&getinfo();
if (($htmld) || ($hum) || ($emaild)) {&printr();}
exit 0;
# end of perl script
the file /usr/exim/virtual/loggingb contains domains that you wish to log
ie
*@domain1.com
*@domain2.com
The originator_uid is the user that runs amavis and I don't want to include
email that has gone through that as then the user will get double :(
you could also use the id for a pipe that you don't wish to include
# Exim Filter
#require this b4 logging
if not first_delivery
then
finish
endif
logfile /var/log/exim/domainemail
if ("${lookup {$sender_address_domain} partial-lsearch*@ \
{/usr/exim/virtual/loggingd} {1}}" is 1 and
$originator_uid is not 58)
then
log "$tod_log $sender_address $message_size >> $recipients_count"
endif
if foranyaddress $recipients ("${lookup {$thisaddress} partial-lsearch*@ \
{/usr/exim/virtual/loggindb} {1}}" is 1 and
$originator_uid is not 58)
then
log "$tod_log $thisaddress $message_size << "
endif
# End of filter bit
----- End forwarded message -----
-Wash
--
Odhiambo Washington
Wananchi Online Ltd.,
wash@??? 1st Flr Loita Hse.
Tel: 254 2 313985 Loita Street.,
Fax: 254 2 313922 PO Box 10286,00100-NAIROBI,KE
A friendship can weather most things and thrive in thin soil; but it needs a
little mulch of letters and phone calls and small, silly presents every so
often - just to save it from drying out completely.
-Pam Brown
