Re: [exim] Reminder: ClamAV 0.95 minimum from 2010-04-15

Top Page
Delete this message
Reply to this message
Author: W B Hacker
Date:  
To: exim users
Subject: Re: [exim] Reminder: ClamAV 0.95 minimum from 2010-04-15
e.sand+exim-users@??? wrote:
>> IMNSHO, that's bass-ackwards.
>
> Quite possibly - and I agree, but I am currently "lazy", thus removing ClamAV
> since it had fewer "hits" than SA does in my logs is the most logical choice
> for me (in terms of least work required) at the moment.


Well the 'biggie' is that the few times ClamAV gets a hit (damn seldom with all
Zombots blocked) what it hits on is almost always a dangerous (read
cross-platform) phish.

WHEREAS - what SA catches is more often some damn fool composing in html or
purveying some WinWOES-proprietery attachment that *could* carry vermin - but
seldom is.

So ClamAV is stopping real bullets, even if less than one a month, while what SA
hits are mostly mere nuisances.

> I would rejoice at
> the ability to remove SA from my system however, as the mere thought of
> depending on a Perl program with hundreds of uncompiled RE's makes my
> boy-bits hide.


LOL! .. even that's OK .. long as they aren't hiding in dodgy places...

>
>> SA - implemented in an interpreted language, even one that is arguably a
>> very good fit to the task, but making *seriously* complex tests, requires
>> perhaps several orders of magnitude more resources than ClamAV's slender
>> compiled binary performing a more straightforward job.
>
> I would have thought so too - and perhaps this is maybe just my config for
> ClamAV, but I found it to appear to be using more cpu power than SA was when
> scanning messages...


To be fair - less than 10% of 'offered' incoming traffic ever makes it to the
stage where we would call EITHER ClamAV or SA. Further, as a 100% ABM shop, we
run ClamAV on our own outbound, but not SA. Mac and *BSD boxen are not noted
for generating gratuitous crud, malformed messages, or becoming zombified.

> at least I think that's what I remember (might have been
> memory - but I'm fairly sure it was using more of at least one of the two
> than SA was).


Easy enough to check (observe K and M distinctions, of course..):


CAVEAT: Lighty loaded boxen, one with really short uptime:

=====

conducive# uptime
8:58PM up 18 days, 19 hrs, 2 users, load averages: 0.07, 0.03, 0.01

   PID USERNAME     THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND


SA:

   830 root           1  44    0 24220K 21480K select   7:08  0.00% perl5.8.8
  8851 spamd          1  44    0   121M   103M select   3:49  0.00% perl5.8.8


Exim:

   897 eximd          1  44    0  8008K  2636K select   0:38  0.00% exim-4.69-3


ClamAV:

   892 clamav         1  20    0  4120K  2188K pause    0:29  0.00% freshclam
   881 clamav         2  44    0 91240K 88284K select   0:00  0.00% clamd


====
triligon# uptime
5:03AM up 346 days, 7:01, 1 user, load averages: 0.49, 0.15, 0.06

   PID USERNAME     THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND


No SA installed:

ClamAV:

   797 clamav         2  44    0 91016K 87928K select   0:00  0.00% clamd
   802 clamav         1  20    0  5144K  3004K pause    3:33  0.00% freshclam


Exim:

70467 eximd          1  45    0  7308K  2928K select   4:24  0.00% exim-4.69-3


> In either case however, I would ultimately really prefer to
> have Exim running alone - not really requiring the aid of any 3rd party
> programs to perform message scanning.


Message (content) scanning can indeed be dispensed with these days since it is
now so easy to block zombots, malware, and 'professional' advertisers.

Thereafter, if the sending network/MTA entity is 'clean' then the only marginal
content will be from honest folks (or plain fools) and most of your clients will
want/need to see that sort of stuff from Win-Victim friends and family *anyway*.

JFWIW, I haven't had a single ClamAV hit in over six months. Zero in 346 days
for the second server above.

Why so?

Most Wincrobes arrive from compromised/zombot Winboxen.

As these fail rDNS, and/or are in Dynamic-IP or other RBL or LBL's, they never
make it past the early stage of acl_smtp_connect. A long, long way from waking
up a scanner...

> I haven't had the time or opportunity
> however to do anything more than think of how nice it would be.
>


It IS nice. Look at the average server load when you don't let zombots even
stay for a full second. Both servers are 1U VIA C6 and drawing very little power
from our expensive UPS budget.

>> We've always run SA stripped-down to only such tests as cannot be done more
>> 'cheaply' by Exim (or ClamAV) and are having good success a year into the
>> experiment with dropping SA altogether on one of our servers.
>
> That statement is extremely intriguing. I would be interested (if it's not a
> company secret) in learning what measures you take within Exim's config to
> block out most of your unwanted email. As I stated just earlier - if I could
> ultimately remove my last dependency of SA from Exim, I would be a happy man.
>
>
> Eli.
>
>


No secret.

90% of the work is done with one test:

CAVEATS:

- Apply to port 25 (outsiders) but not port 587 (your user's AUTH'ing to submit)

- Note these are the THIRD test in acl_smtp_connect. The first two check
Whitelists by IP and/or (IF resolvable) whitelists by the domain.tld of the said
connecting IP)

- uncomment the logwrites only for testing


====

# CONNECT_3: Tests for rDNS - delayed to allow whitelisting

   # CONNECT_3A first test for simple absence of a PTR RR
   # *** CAVEAT WBH change to ~/src/host.c required for this to work! ***
     deny
   # logwrite    = C3A Checking for PTR RR on IP=$sender_host_address \
   #         $sender_address $sender_host_name
     condition   = ${if eq{$interface_port}{25}}
     !hosts      = : +relay_from_hosts
     condition   = ${if match{$sender_host_name}{$sender_host_address}{yes}{no}}
     log_message = C3A no valid PTR RR for $sender_host_address $sender_address \
         $sender_host_name


CAVEAT: Skip the above unless you want to recompile. Exim does exactly the same
thing with the second test, below. All the above does is make it clear whether a
PTR was missing or just present but did not link to A or MX RR useful for smtp,
as in the 'generic' PTR RR with hyphenated IP used by some ISP networks for ADSL
clients..

   # CONNECT_3B If a PTR RR was not found, we don't get this far.
   # If a PTR exists, test to see if it is of any *USE*
   #
   deny
     message     = Sender on $sender_address not a mail server. \
                   DNS PTR RR fails to match to valid host name.\
   # logwrite    = C3B rDNS lookup for $sender_host_address $sender_address \
   #          $sender_host_name
     condition   = ${if eq{$interface_port}{25}}
     !hosts      = : +relay_from_hosts
     !verify     = reverse_host_lookup
     log_message = C3B rDNS fail for $sender_host_address $sender_address \
         $sender_host_name



A handful of LBL/RBL tests, a check for forged headers or failed HELO match to
FQDN (soft fail here), insistance on verifying recipients, a limit on incoming
conections to no more than 2 or 3 from any given IP, and you are in clover.

IF you have no affnity-groups, extended family, or company staff pools, you MAY
have a preponderance of singleton arrival, and singleton messages PER arrival.

In that caee you can also defer second and subsequent recipient which allows you
to apply per-recipient preferences in the DATA phase and still reject while
in-session if need be. IOW no need to generate a post-session bounce.


HTH,

Bill