On Thu, 2007-02-08 at 09:10 +0100, Adrian Phillips wrote:
> One problem I had (setting up David's sqlite greylisting) was
> understanding that "warn condition = ${lookup" can defer on "some
> database error" but "set acl = ${lookup" doesn't.
I've only just got round to looking at the sqlite version, which was put
together by someone else based on my original evil hack with lsearch and
${run echo foo >> db.txt}.
There were a few problems with it -- mostly iirc that in contradiction
of the comments, it didn't accept a message when it came from a
different host the second time round. It required $sender_host_address
to match too. So if you get a mail which is tried again from a different
IP address, it'll attempt to add that as a 'new' mail to the database.
But since the primary key for the database is the mail 'ident' hash, and
(deliberately) doesn't include the IP address that insertion will fail
because of a key collision. The mail is still deferred though, and this
will happen repeatedly. AFAICT it'll _never_ accept mail if the original
host sends it on to a fallback 'queue host' after the initial refusal.
I've sorted out the underlying bug, made it _accept_ mail when the
database insertion fails (because that failure mode is particularly
sucky even though it should never normally happen) and also made it use
a tuple of { IP, HELO } for the known resenders list, as we discussed
here a few weeks ago.
I'm shipping it in an 'exim-greylist' subpackage of the Fedora Exim
package. It currently looks like this.
(Btw, is there a clever way of doing 'SELECT foo,bar,baz FROM whatever'
and getting them into three separate $acl_m_foo, $acl_m_bar, $acl_m_baz
variables? I can either select them and then split the single text
string, or fetch them separately as I do below. Did I miss a trick?
In fact, I can't even see how to split a foo|bar|baz string and then set
acl_m_foo=$1, set acl_m_bar=$2 etc., because aren't $1,$2 etc. cleared
by the time you get to the next ACL modifier? You have to use them
_inside_ the ${sg...} IIRC?)
# $Id: exim-greylist.conf.inc,v 1.6 2007/02/08 10:31:24 dwmw2 Exp $
GREYDB=/var/spool/exim/db/greylist.db
# ACL for greylisting. Place reason(s) for greylisting into a variable named
# $acl_m_greylistreasons before invoking with 'require acl = greylist_mail'.
# The reasons should be separate lines of text, and will be reported in
# the SMTP rejection message as well as the log message.
#
# When a suspicious mail is seen, we temporarily reject it and wait to see
# if the sender tries again. Most spam robots won't bother. Real mail hosts
# _will_ retry, and we'll accept it the second time. For hosts which are
# observed to retry, we don't bother greylisting again in the future --
# it's obviously pointless. We remember such hosts, or 'known resenders',
# by a tuple of their IP address and the name they used in HELO.
#
# We also include the time of listing for 'known resenders', just in case
# someone wants to expire them after a certain amount of time. So the
# database table for these 'known resenders' looks like this:
#
# CREATE TABLE resenders (
# host TEXT,
# helo TEXT,
# time INTEGER,
# PRIMARY KEY (host, helo) );
#
# To remember mail we've rejected, we create an 'identity' from its sender
# and recipient addresses and its Message-ID: header. We don't include the
# sending IP address in the identity, because sometimes the second and
# subsequent attempts may come from a different IP address to the original.
#
# We do record the original IP address and HELO name though, because if
# the message _is_ retried from another machine, it's the _first_ one we
# want to record as a 'known resender'; not just its backup path.
#
# Obviously we record the time too, so the main table of greylisted mail
# looks like this:
#
# CREATE TABLE greylist (
# id TEXT,
# expire INTEGER,
# host TEXT,
# helo TEXT);
#
greylist_mail:
# First, accept if it there's absolutely nothing suspicious about it...
accept condition = ${if eq{$acl_m_greylistreasons}{} {1}}
# ... or if it was generated locally or by authenticated clients.
accept hosts = :
accept authenticated = *
# Secondly, there's _absolutely_ no point in greylisting mail from
# hosts which are known to resend their mail. Just accept it.
accept hosts = sqlite;GREYDB SELECT host from resenders \
WHERE helo='${quote_sqlite:$sender_helo_name}' \
AND host='$sender_host_address';
# Generate a hashed 'identity' for the mail, as described above.
warn set acl_m_greyident = ${hash{20}{62}{$sender_address$recipients$h_message-id:}}
# Attempt to look up this mail in the greylist database. If it's there,
# remember the expiry time for it; we need to make sure they've waited
# long enough.
warn set acl_m_greyexpiry = ${lookup sqlite {GREYDB SELECT expire FROM greylist \
WHERE id='${quote_sqlite:$acl_m_greyident}';}{$value}}
# If the mail isn't already the database -- i.e. if the $acl_m_greyexpiry
# variable we just looked up is empty -- then try to add it now. This is
# where the 5 minute timeout is set ($tod_epoch + 300), should you wish
# to change it.
warn condition = ${if eq {$acl_m_greyexpiry}{} {1}}
set acl_m_dontcare = ${lookup sqlite {GREYDB INSERT INTO greylist \
VALUES ( '$acl_m_greyident', \
'${eval10:$tod_epoch+300}', \
'$sender_host_address', \
'${quote_sqlite:$sender_helo_name}' );}}
# Be paranoid, and check if the insertion succeeded (by doing another lookup).
# Otherwise, if there's a database error we might end up deferring for ever.
defer condition = ${if eq {$acl_m_greyexpiry}{} {1}}
condition = ${lookup sqlite {GREYDB SELECT expire FROM greylist \
WHERE id='${quote_sqlite:$acl_m_greyident}';} {1}}
message = Your mail was considered suspicious for the following reason(s):\n$acl_m_greylistreasons \
The mail has been greylisted for 5 minutes, after which it should be accepted. \
We apologise for the inconvenience. Your mail system should keep the mail on \
its queue and retry. When that happens, your system will be added to the list \
genuine mail systems, and mail from it should not be greylisted any more. \
In the event of problems, please contact postmaster@$qualify_domain
log_message = Greylisted <$h_message-id:> from <$sender_address> for offences: ${sg {$acl_m_greylistreasons}{\n}{,}}
# Handle the error case (which should never happen, but would be bad if it did).
# First by whining about it in the logs, so the admin can deal with it...
warn condition = ${if eq {$acl_m_greyexpiry}{} {1}}
log_message = Greylist insertion failed. Bypassing greylist.
# ... and then by just accepting the message.
accept condition = ${if eq {$acl_m_greyexpiry}{} {1}}
# OK, we've dealt with the "new" messages. Now we deal with messages which
# _were_ already in the database...
# If the message was already listed but its time hasn't yet expired, keep rejecting it
defer condition = ${if > {$acl_m_greyexpiry}{$tod_epoch}}
message = Your mail was previously greylisted and the time has not yet expired.\n\
You should wait another ${eval10:$acl_m_greyexpiry-$tod_epoch} seconds.\n\
Reason(s) for greylisting: \n$acl_m_greylistreasons
# The message was listed but it's been more than five minutes. Accept it now and whitelist
# the _original_ sending host by its { IP, HELO } so that we don't delay its mail again.
warn set acl_m_orighost = ${lookup sqlite {GREYDB SELECT host FROM greylist \
WHERE id='${quote_sqlite:$acl_m_greyident}';}{$value}}
set acl_m_orighelo = ${lookup sqlite {GREYDB SELECT helo FROM greylist \
WHERE id='${quote_sqlite:$acl_m_greyident}';}{$value}}
set acl_m_dontcare = ${lookup sqlite {GREYDB INSERT INTO resenders \
VALUES ( '$acl_m_orighost', \
'${quote_sqlite:$acl_m_orighelo}', \
'$tod_epoch' ); }}
logwrite = Added host $acl_m_orighost with HELO '$acl_m_orighelo' to known resenders
accept
--
dwmw2
