Re: [exim] Exim Consolidating users

Top Page
Delete this message
Reply to this message
Author: Jonathan Gilpin
Date:  
To: exim-users@exim.org
Subject: Re: [exim] Exim Consolidating users
On 24 Feb 2010, at 22:54, Phil Pennock wrote:

> On 2010-02-24 at 19:10 +0000, Jonathan Gilpin wrote:
>> On 24 Feb 2010, at 18:51, Phil Pennock wrote:
>>
>>> On 2010-02-24 at 18:16 +0000, Jonathan Gilpin wrote:
>>>> I'm not using smtp... I have a catchall alias of *@domain going to a maildir. I then end up with a message with two addresses in the Envelope-To: E.G
>>>
>>> So can you *please* provide the relevant configuration (Router and
>>> Transport) so that we have something to look at and work from to
>>> diagnose the problem, instead of working blind?
>>
>> For the routers I have:
>
> [ Snip 4 routers, none of which is the Router causing the problem ]
>
>> A mail to: gg@??? and ga@??? both if which do not exist are caught by the catchall, one email is delivered as shows in the logs:
>>
>> 2010-02-24 18:15:02 1NkLlE-000AJ2-Pu SA: Action: scanned but message isn't spam: score=2.9 required=4.0 (scanned in 2/2 secs | Message-Id: E1NkLlE-000AJ2-Pu@???). From <root@???> (local) for gg@???, ga@???
>> 2010-02-24 18:15:02 1NkLlE-000AJ2-Pu => postmaster <gg@???> R=virtual_local_mailbox T=virtual_local_md_delivery
>
> Okay, so you've rewritten the recipient in another Router which you
> haven't shown us. Two of the SMTP RCPT recipients have been rewritten
> to the same SMTP recipient, and Exim is not delivering twice to the
> *same* recipient, as routed.
>
> This is a very different problem to the one described.
>
> Your catchall is causing you to only try to deliver one copy.
>
> Instead of using a catchall "redirect" router, use a catchall "accept"
> router, which stores the mail in the catchall folder directly, but
> routed for the original address.


Hi, I am using an accept not a redirect. I have included the full config below. Hope you can spot something...

######################################################################
#                    MAIN CONFIGURATION SETTINGS                     #
######################################################################


# MySQL defines
MYSQL_SERVER=localhost
MYSQL_USER=mailnull
MYSQL_PASSWORD=******
MYSQL_DB=exim
MYSQL_EMAILTABLE=emailtable
MYSQL_DOMAINTABLE=domaintable
MYSQL_DOMAINRTABLE=relaytable
MYSQL_WHITETABLE=whitelist
MYSQL_BLACKTABLE=blacklist
MYSQL_AUTHTABLE=emailtable

local_interfaces = /snipped/

# MySQL queries
MYSQL_Q_ISAWAY=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='${quote_mysql:$local_part}' AND is_away='yes'
MYSQL_Q_AWAYTEXT=SELECT away_text FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='${quote_mysql:$local_part}'
MYSQL_Q_FORWARD=SELECT forward FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='${quote_mysql:$local_part}' AND forward != ''
MYSQL_Q_CC=SELECT cc FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='${quote_mysql:$local_part}'
MYSQL_Q_ALIASES=SELECT substring(address, locate('@', address) +1) FROM MYSQL_EMAILTABLE,aliases WHERE emailtable.email = aliases.destination and (aliases.address = '${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}')
#MYSQL_Q_ALIASES=SELECT destination FROM MYSQL_EMAILTABLE,aliases WHERE emailtable.email = aliases.destination and (aliases.address = '${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}')
MYSQL_Q_LOCAL=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND (local_part='${quote_mysql:$local_part}' or local_part='*') AND mail != ''
MYSQL_Q_WCLOCAL=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='*' AND forward != ''
MYSQL_Q_WCLOCFW=SELECT forward FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='*' AND forward != ''
MYSQL_Q_LDOMAIN=SELECT DISTINCT domain FROM MYSQL_DOMAINTABLE WHERE domain='$domain'
MYSQL_Q_RDOMAIN=SELECT DISTINCT domain FROM MYSQL_DOMAINRTABLE WHERE domain='$domain'
MYSQL_Q_RELAYIP=SELECT ip from relayhosts
MYSQL_Q_BOXNAME=SELECT mail FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND (local_part='${quote_mysql:$local_part}' or local_part='*')
MYSQL_Q_ABOXNAME=SELECT mail FROM MYSQL_EMAILTABLE,aliases WHERE emailtable.email = aliases.destination and (aliases.address='${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}')
MYSQL_Q_SPAMC=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND (local_part='${quote_mysql:$local_part}' or local_part='*' ) AND opt_spamscan='yes'
MYSQL_Q_ASPAMC=SELECT substring(address, locate('@', address) +1) FROM MYSQL_EMAILTABLE,aliases WHERE emailtable.email = aliases.destination and (aliases.address = '${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}') AND opt_spamscan='yes'
#MYSQL_Q_ASPAMC=SELECT destination FROM MYSQL_EMAILTABLE,aliases WHERE emailtable.email = aliases.destination and (aliases.address = '${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}') AND opt_spamscan='yes'
MYSQL_Q_VSCAN=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='${quote_mysql:$local_part}' AND opt_virscan='yes'
MYSQL_Q_SPAMPURGE=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND (local_part='${quote_mysql:$local_part}' or local_part='*') AND opt_spampurge='yes'
#MYSQL_Q_DISCLAIMER=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND (local_part='${quote_mysql:$local_part}' or local_part='Postmaster') AND disclaimer='1'
MYSQL_Q_DISCLAIMER=SELECT domain FROM MYSQL_EMAILTABLE WHERE email='${quote_mysql:$authenticated_id}' AND disclaimer='1'
MYSQL_Q_DISABLED=SELECT domain FROM MYSQL_EMAILTABLE WHERE domain='${quote_mysql:$domain}' AND local_part='${quote_mysql:$local_part}' AND is_enabled='no'
MYSQL_Q_WHITELIST=SELECT DISTINCT MYSQL_WHITETABLE.address FROM MYSQL_WHITETABLE WHERE '${quote_mysql:$sender_address}' LIKE whitelist.address
MYSQL_Q_BLACKLIST=SELECT DISTINCT MYSQL_BLACKTABLE.address FROM MYSQL_BLACKTABLE WHERE '${quote_mysql:$sender_address}' LIKE blacklist.address
#MYSQL_Q_AUTHPWD1=SELECT concat(local_part,'@',domain) FROM MYSQL_AUTHTABLE WHERE email ='$2' AND password='$3'
MYSQL_Q_AUTHPWD1=SELECT concat(local_part,'@',domain) FROM MYSQL_AUTHTABLE WHERE concat(local_part,'@',domain) ='$2' AND password='$3'
#MYSQL_Q_AUTHPWD2=SELECT concat(local_part,'@',domain) FROM MYSQL_AUTHTABLE WHERE email = '$1' AND password='$2'
MYSQL_Q_AUTHPWD2=SELECT concat(local_part,'@',domain) FROM MYSQL_AUTHTABLE WHERE concat(local_part,'@',domain) = '$1' AND password='$2'
# MySQL connection
hide mysql_servers = "MYSQL_SERVER/MYSQL_DB/MYSQL_USER/MYSQL_PASSWORD"

# starting 'normal' config

#
#delay_warning = 0s
#deliver_queue_load_max = 99
#ignore_bounce_errors_after = 0s
#return_size_limit = 10000
#queue_only_load = 3
#queue_run_max = 5
#smtp_accept_queue = 10
#strip_trailing_dot
#split_spool_directory
#remote_max_parallel = 40
#
primary_hostname = hostname

domainlist local_domains = mysql;MYSQL_Q_LDOMAIN
pid_file_path = /var/run/exim.pid

domainlist relay_to_domains = mysql;MYSQL_Q_RDOMAIN
hostlist relay_from_hosts = ${lookup mysql{MYSQL_Q_RELAYIP}{${sg{$value}{\\n}{ : }} }}
#mysql;MYSQL_Q_RELAYIP

domainlist noreverse = various domains

acl_smtp_rcpt = acl_check_rcpt
acl_smtp_data = acl_check_data
av_scanner = clamd:/var/run/clamav/clamd.sock

qualify_domain = fluent18.fluent.ltd.uk
# qualify_recipient =

# allow_domain_literals
smtp_accept_max = 500

never_users = root
#trusted_users = list # : amavis
trusted_users = mailnull

host_lookup = *
rfc1413_hosts = *
smtp_enforce_sync = false
rfc1413_query_timeout = 2s

check_spool_space = 50M
check_log_space = 20M
return_size_limit = 20k
message_size_limit = 20M

# sender_unqualified_hosts =
# recipient_unqualified_hosts =

# percent_hack_domains =

ignore_bounce_errors_after = 2h
timeout_frozen_after = 4d

deliver_queue_load_max = 8
queue_only_load = 10
remote_max_parallel = 15

#tls_certificate =
#tls_privatekey =
#tls_advertise_hosts = *

######################################################################
#                       ACL CONFIGURATION                            #
#         Specifies access control lists for incoming SMTP mail      #
######################################################################


begin acl

acl_check_rcpt:
  accept  hosts = :
  deny    domains       = +local_domains
          local_parts   = ^[.] : ^.*[@%!/|]
  deny    domains       = !+local_domains
          local_parts   = ^[./|] : ^.*[@%!] : ^.*/\\.\\./
  accept  local_parts   = postmaster
          domains       = +local_domains


#  deny    message       = Reverse DNS lookup failed for host $sender_host_address.
#          log_message   = Reverse DNS lookup failed
#      domains    = !+noreverse
#      hosts        = !+relay_from_hosts
#      !authenticated = *
#          !verify       = reverse_host_lookup
  drop    log_message   = match sbl-xbl.spamhaus.org
      !authenticated = *
      dnslists     = sbl-xbl.spamhaus.org
  deny    message     = Access denied - $sender_host_address listed by $dnslist_domain\n$dnslist_text
      !authenticated = *
          dnslists     = dnsbl.sorbs.net 
#  require verify        = sender
  accept  domains       = +local_domains
          endpass
          verify        = recipient
  accept  domains       = +relay_to_domains
          endpass
          verify        = recipient
  deny      message    = $sender_host_address is not allowed to send mail from $sender_address_domain
      log_message    = SPF check failed
           spf        = fail
  accept  hosts         = +relay_from_hosts


  accept  authenticated = *
      add_header    = X-AuthenticatedID: $authenticated_id
        add_header    = X-OriginalFromAdr: $sender_address
  deny    message       = relay not permitted


acl_check_data:
  deny      message       = X-Quarantine-Me-Malware: $malware_name
          log_message   = malware: $malware_name
          demime        = *
          malware       = */defer_ok
  accept



######################################################################
#                      ROUTERS CONFIGURATION                         #
#               Specifies how addresses are handled                  #
######################################################################
#     THE ORDER IN WHICH THE ROUTERS ARE DEFINED IS IMPORTANT!       #
# An address is passed to each router in turn until it is accepted.  #
######################################################################


begin routers

fail_router:
driver = redirect
domains = ${lookup mysql {MYSQL_Q_DISABLED}{$value}}
data = ":fail:"
allow_fail

##uncomment, if you plan to use mailman (www.list.org)
#mailman_aliases:
# driver = redirect
# allow_fail
# allow_defer
# data = ${lookup{$local_part}lsearch{/etc/aliases.mailman}}
# file_transport = address_file
# pipe_transport = address_pipe

#amavis_director:
#  driver = manualroute
#  domains = ${lookup mysql {MYSQL_Q_VSCAN}{$value}}
#  condition = "${if or { \
#        {eq{$received_protocol}{scanned-ok}} \
#        {eq{$received_protocol}{spam-scanned}} \
# if you like to scan mails from localhost too, then comment the following line (not good for big mailman lists...)
#        {eq{$sender_host_address}{127.0.0.1}} \
#      } \ 
#    {0}{1}}"
#  route_list = "* localhost byname"
#  transport = amavis
#  verify = false


#check_malware:
#  driver                = redirect
#  condition             = ${if def:h_X-Quarantine-Me-Malware: {1}{0}}
#  headers_add           = X-Quarantined-Malware: $h_X-Quarantine-Me-Malware:
#  headers_remove        = X-Quarantine-Me-Malware
#  data                  = /var/db/malware/malware.$tod_logfile
#  file_transport        = address_file


check_disclaimer:
driver = dnslookup
# domains = ! +local_domains :
# domains = ${lookup mysql {MYSQL_Q_SPAMPURGE}{$value}}
# domains = ${lookup mysql {MYSQL_Q_DISCLAIMER}{$value}}
senders = ${lookup mysql {MYSQL_Q_DISCLAIMER}{$value}}
# domains = !+rfwrutherfords
transport = disclaimer_pipe
same_domain_copy_routing = yes
no_more

dnslookup:
driver = dnslookup
domains = !+local_domains
transport = remote_smtp
ignore_target_hosts = 0.0.0.0 : 127.0.0.0/8
no_more

blacklist_router:
driver = manualroute
senders = ${lookup mysql {MYSQL_Q_BLACKLIST}{$value}}
condition = "${if !def:h_X-Spam-Flag: {1}{0}}"
headers_add = X-Spam-Flag: YES
route_list = * localhost
self = pass

system_aliases:
driver = redirect
allow_fail
allow_defer
data = ${lookup{$local_part}lsearch{/etc/aliases}}
# user = exim
file_transport = address_file
pipe_transport = address_pipe

virtual_forward_director:
driver = redirect
data = ${lookup mysql {MYSQL_Q_FORWARD}{$value}}

virtual_cc_director:
driver = redirect
data = ${lookup mysql {MYSQL_Q_CC}{$value}}
unseen

spamcheck_director:
  driver = manualroute
  domains = ${lookup mysql {MYSQL_Q_SPAMC}{$value}}
  senders = ! ${lookup mysql {MYSQL_Q_WHITELIST}{$value}}
  condition = ${if and { \
    {!eq {$received_protocol}{spam-scanned}} \
    {!eq {$received_protocol}{local}} \
    } {1}{0}}
  headers_remove = X-Spam-Flag
  route_list = "* localhost byname"
  transport = spamcheck
  verify = false


#spamcheck_alias_director:
#  driver = manualroute
#  domains = ${lookup mysql {MYSQL_Q_ASPAMC}{$value}}
#  senders = ! ${lookup mysql {MYSQL_Q_WHITELIST}{$value}}
#  condition = ${if and { \
#    {!eq {$received_protocol}{spam-scanned}} \
#    {!eq {$received_protocol}{local}} \
#    } {1}{0}}
#  headers_remove = X-Spam-Flag
#  route_list = "* localhost byname"
#  transport = spamcheck
#  verify = false



spampurge_director:
driver = manualroute
domains = ${lookup mysql {MYSQL_Q_SPAMPURGE}{$value}}
condition = "${if eq{$h_X-Spam-Flag:}{YES} {1}{0}}"
route_list = "* localhost byname"
transport = devnull_transport
verify = false

vacation_director:
driver = accept
domains = ${lookup mysql {MYSQL_Q_ISAWAY}{$value}}
transport = vacation_autoreply
unseen

mysql_aliases:
driver = redirect
file_transport = address_file
pipe_transport = address_pipe
data = ${lookup mysql{SELECT destination FROM aliases,emailtable WHERE emailtable.email = aliases.destination and (aliases.address = '${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}')}}


spamcheck_alias_director:
  driver = manualroute
  domains = ${lookup mysql {MYSQL_Q_ASPAMC}{$value}}
  senders = ! ${lookup mysql {MYSQL_Q_WHITELIST}{$value}}
  condition = ${if and { \
    {!eq {$received_protocol}{spam-scanned}} \
    {!eq {$received_protocol}{local}} \
    } {1}{0}}
  headers_remove = X-Spam-Flag
  route_list = "* localhost byname"
  transport = spamcheck
  verify = false


#mysql_aliases:
#  driver = redirect
#  file_transport = address_file
#  pipe_transport = address_pipe
#  data = ${lookup mysql{SELECT destination FROM aliases,emailtable WHERE emailtable.email = aliases.destination and (aliases.address = '${quote_mysql:$local_part}@${quote_mysql:$domain}' or aliases.address = '*@${quote_mysql:$domain}')}}
  #add_header    = X-Envelope-To: ${quote_mysql:$local_part}@${quote_mysql:$domain} 


virtual_local_spam:
driver = accept
condition = "${if eq{$h_X-Spam-Flag:}{YES} {1}{0}}"
domains = ${lookup mysql {MYSQL_Q_LOCAL}{$value}}
transport = virtual_local_spam_delivery

virtual_local_mailbox:
driver = accept
domains = ${lookup mysql {MYSQL_Q_LOCAL}{$value}}
transport = virtual_local_md_delivery

#virtual_wclocal_redirect:
# driver = redirect
# driver = accept
# domains = ${lookup mysql {MYSQL_Q_WCLOCAL}{$value}}
# data = ${lookup mysql {MYSQL_Q_WCLOCFW}{$value}}
# transport = virtual_local_md_delivery

virtual_alias:
driver = accept
domains = ${lookup mysql {MYSQL_Q_ALIASES}{$value}}
transport = virtual_local_alias_delivery

######################################################################
#                      TRANSPORTS CONFIGURATION                      #
######################################################################
#                       ORDER DOES NOT MATTER                        #
#     Only one appropriate transport is called for each delivery.    #
######################################################################


begin transports

disclaimer_pipe:
driver = smtp
debug_print = "T: add_company_details_remote_smtp for $local_part@$domain"
transport_filter = /usr/local/bin/altermime --log-syslog --input=- --disclaimer=/usr/local/mail/disclaimers/rfwrutherfords.com.txt --disclaimer-html=/usr/local/mail/disclaimers/rfwrutherfords.com.html
size_addition = 1

remote_smtp:
driver = smtp
# interface = 195.78.94.248

devnull_delivery:
driver = appendfile
file = /dev/null
group = mail

address_pipe:
driver = pipe
return_output

address_file:
driver = appendfile
delivery_date_add
envelope_to_add
return_path_add

address_reply:
driver = autoreply

#amavis:
# driver = pipe
# command = "/usr/sbin/amavis <${sender_address}> ${pipe_addresses}"
# message_prefix =
# message_suffix =
# check_string =
# escape_string =
## for debugging change return_output to true
# return_output = false
# return_path_add = false
# user = amavis
# group = mail
# path = "/bin:/sbin:/usr/bin:/usr/sbin"
# current_directory = "/var/amavis"

spamcheck:
driver = pipe
command = /usr/local/sbin/exim -oMr spam-scanned -bS
use_bsmtp = true
transport_filter = "/usr/local/bin/spamc"
home_directory = "/tmp"
current_directory = "/tmp"
user = mailnull
group = mail
log_output = true
return_fail_output = true
return_path_add = false
message_prefix =
message_suffix =


virtual_local_alias_delivery:
driver = appendfile
directory = ${lookup mysql {MYSQL_Q_ABOXNAME}{$value}}
maildir_format
user = mailnull
group = mail
mode = 0660
directory_mode = 0770
envelope_to_add
return_path_add


virtual_local_md_delivery:
driver = appendfile
directory = ${lookup mysql {MYSQL_Q_BOXNAME}{$value}}
maildir_format
user = mailnull
group = mail
mode = 0660
directory_mode = 0770
envelope_to_add
return_path_add

virtual_local_spam_delivery:
driver = appendfile
directory = ${lookup mysql {MYSQL_Q_BOXNAME}{$value}}/.Spam
maildir_format
user = mailnull
group = mail
mode = 0660
directory_mode = 0770
envelope_to_add
return_path_add

vacation_autoreply:
driver = autoreply
to = ${sender_address}
from = "vacation@${domain}"
subject = "AutoResponder for ${local_part}@${domain}"
text = ${lookup mysql {MYSQL_Q_AWAYTEXT}{$value}}

devnull_transport:
driver = appendfile
file = /dev/null
user = mailnull

######################################################################
#                      RETRY CONFIGURATION                           #
######################################################################


begin retry

# This single retry rule applies to all domains and all errors. It specifies
# retries every 15 minutes for 2 hours, then increasing retry intervals,
# starting at 1 hour and increasing each time by a factor of 1.5, up to 16
# hours, then retries every 6 hours until 4 days have passed since the first
# failed delivery.

# Domain               Error       Retries
# ------               -----       -------


*                      *           F,2h,15m; G,16h,1h,1.5; F,4d,6h




######################################################################
#                      REWRITE CONFIGURATION                         #
######################################################################


# There are no rewriting specifications in this default configuration file.
begin rewrite



######################################################################
#                   AUTHENTICATION CONFIGURATION                     #
######################################################################


# There are no authenticator specifications in this default configuration file.

begin authenticators

fixed_plain:
driver = plaintext
public_name = PLAIN
server_condition = ${lookup mysql{MYSQL_Q_AUTHPWD1}{1}fail}
server_set_id = $2

fixed_login:
driver = plaintext
public_name = LOGIN
server_prompts = "Username:: : Password::"
server_condition = ${lookup mysql{MYSQL_Q_AUTHPWD2}{1}fail}
server_set_id = $1

######################################################################
#                   CONFIGURATION FOR local_scan()                   #
######################################################################


# If you have built Exim to include a local_scan() function that contains
# tables for private options, you can define those options here. Remember to
# uncomment the "begin" line. It is commented by default because it provokes
# an error with Exim binaries that are not built with LOCAL_SCAN_HAS_OPTIONS
# set in the Local/Makefile.

# begin local_scan


# End of Exim configuration file