Rather than suborning ~/exim/paniclog as a convenient place to write things with
use of Exim's built-in 'logwrite' and 'log_message' tools...
- Might we have one (or even three?) 'spare', blank, 'freeform' logs.
"blank" as in not even timestamped unless $tod_epoch or such is specified.
Rationale:
- Calling an external script, compiled or interpreted executable, or even the
embedded perl or a filter raises resource load, locking and privilege issues,
not to mention where (smtp-time or not) it can be applied. Essentially
re-inventing the wheel AND the tire.
- By adding a 'wheel' with no tire, we can use the existing logging mechanism.
Present-day example:
deny
!hosts = /var/mail/IP-white
message = Sender $sender_host_address using dynamic IP for mail server
dnslists = dul.dnsbl.sorbs.net
logwrite = :panic:,DYN,$sender_host_address,$sender_host_name
On an RBL 'hit' the paniclog might contain:
2006-09-23 20:30:25 ,DYN,195.14.185.10,ip-195-14-185-10.bnk.lt
grep ",DYN," /var/log/exim/paniclog > /path/to/file
and we have a .csv file uploadable to an SQL DB, spreadsheet - whatever.
We then use that (in this case) to put chronic offenders into a local blockfile,
/var/mail/IP-black.
As only a few hundred of sorbs' brazillions of listed portable offenders hit our
site in a given month, and as Exim does not cache between 'child' processes,
keeping the worst recent offnders 'local' saves thousands of RBL callouts every day.
What is wanted is to omit the timestamp and trailing space, optionally adding
'$tod_epoch' (or similar such replacement..) as the last field, and to NOT have
to interfere with existing logs, their naming, syslog interface, or rotation.
To wit, where 'scratch(n)' is the new logfile name:
deny
!hosts = /var/mail/IP-white
message = Sender $sender_host_address using dynamic IP for mail server
dnslists = dul.dnsbl.sorbs.net
logwrite = :scratch1:$sender_host_address,$sender_host_name,$tod_epoch
or:
deny
!hosts = /var/mail/IP-white
message = Sender $sender_host_address using dynamic IP for mail server
dnslists = dul.dnsbl.sorbs.net
logwrite = :scratch2:$sender_host_name:$sender_host_address
In use:
- 'Exceptions' - (fewer than a dozen here) driven by customer needs, are placed
into /var/mail/IP-white. A 'hit' will skip the above tests.
- Chronic, long-term offenders are ID'ed with external analysis tools, placed
into /var/mail/IP-black and also tested before hitting the above acl clauses.
No new RBL callout needed.
- The top few hundred 'new' offenders listed here, built up throughout the day
and also massaged by external tools each night - are placed into
/var/mail/IP-brown and tested *before* making the above RBL callout.
Over time, denizens of /var/mail/IP-brown will either be discarded or migrated
from the brownlist to the blacklist (a few perhaps even to the whitelist).
As exim already 'owns' the log files and has them open for writing, the right
format could even make :scratch(n): directly useful in an acl before end-of-day
external massaging.
Not sure if this would always intercept the second hit before RBL callout, but
it should catch the 3rd or 4th, and zombies, dictionery attacks, etc. *DO* seem
to come in waves more ofhten than not.
Result, BTW, is a reduction in RBL callouts from several per second to a few
every hour or so. 80-20 rule (or 98/2, really).
A 'free form' extra log file could be a very handy item, and probably easier to
implement than a general-purpose write-anything from anywhere-at-anytime facility.
Bill Hacker
