[follow up on exim-users please -- let's try to keep this on one list]
On 11 October 2002, I said:
> I've just spent the last couple of hours beating the README.EXIM
> document from Mailman's current CVS tree into shape. (This is derived
> from Nigel Metheringham's "HOWTO - Using exim and Mailman together"
> document, which can be found at www.exim.org.)
OK, here's the latest version. I wrote about receiver verification, and
SMTP callback based on Marc's suggestion. Rewrote his text and
streamlined the config a bit to improve readability -- hope it still
works.
I also actually tested my suggested transport, and the version here is
known to work with Exim 4. ;-)
------------------------------------------------------------------------
Using Exim and Mailman Together
===============================
[This is derived from Nigel Metheringham's "HOWTO - Using exim and
Mailman together", which covers Mailman 2.0.x and Exim 3. It
has been updated to cover Mailman 2.1 and Exim 4. The original
document is here:
http://www.exim.org/howto/mailman.html]
Mailman configuration
---------------------
There is no Mailman configuration needed other than the standard options
detailed in the Mailman install documentation. The Exim configuration
is transparent to Mailman. The uid/gid settings for Mailman must match
those in the config fragments given below.
Exim configuration
------------------
The Exim configuration is built so that a list created within Mailman
automatically appears to Exim without the need for defining any
additional aliases.
The drawback of this configuration is that it will work poorly on
systems supporting lists in several different mail domains. While
Mailman handles virtual domains, it does not yet support having two
lists with the same name in different virtual domains, using the same
Mailman installation. This will eventually change. (But see below for
a variation on this scheme that should accomodate virtual domains
better.)
The configuration file excerpts below are for use in an already
functional Exim configuration, which accepts mail for the domain in
which the list resides. If this domain is separate from the others
handled by your Exim configuration, then you'll need to:
* add the list domain, "my.list.domain" to local_domains
* add a "domains=my.list.domain" option to each of the directors
(routers) for the list
* (optional) exclude that domain from the other directors (routers)
[Note: the instructions in this document should work with either Exim 3
or Exim 4. In Exim 3, you must have a 'local_domains' configuration
setting; in Exim 4, you most likely have a 'local_domains' domainlist.
If you don't, you probably know what you're doing and can adjust
accordingly. Similarly, in Exim 4 the concept of "directors" has
disappeared -- there are only routers now. So if you're using Exim 4,
whenever this document says "director", read "router".]
Whether you are using Exim 3 or Exim 4, you will need to add some macros
to the main section of your Exim config file. You will also need to
define one new transport. With Exim 3, you'll need to add two new
directors; with Exim 4, two new routers play the same role.
Finally, the configuration supplied here should allow co-habiting
Mailman 2.0 and 2.1 installations, with the proviso that you'll probably
want to use "mm21" in place of "mailman" -- e.g., MM21_HOME,
mm21_transport, etc.
Main configuration settings
~~~~~~~~~~~~~~~~~~~~~~~~~~~
First, you need to add some macros to the top of your Exim config file.
These just make the directors (routers) and transport below a bit
cleaner. Obviously, you'll need to edit these based on how you
configured and installed Mailman.
# Home dir for your Mailman installation -- aka Mailman's prefix
# directory.
MAILMAN_HOME=/usr/local/mailman
MAILMAN_WRAP=MAILMAN_HOME/mail/mailman
# User and group for Mailman, should match your --with-mail-gid
# switch to Mailman's configure script.
MAILMAN_UID=mailman
MAILMAN_GID=mailman
Transport for Exim 3
~~~~~~~~~~~~~~~~~~~~
Add this to the transports section of your Exim config file,
i.e. somewhere between the first and second "end" line:
mailman_transport:
driver = pipe
command = MAILMAN_WRAP \
'${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}} \
{post}}' \
$local_part
current_directory = MAILMAN_HOME
home_directory = MAILMAN_HOME
user = MAILMAN_UID
group = MAILMAN_GID
(XXX this is untested by me under Exim 3! Can someone using Exim 3
please let me know if it works?)
Directors for Exim 3
~~~~~~~~~~~~~~~~~~~~
If you're using Exim 3, you'll need to add the following two directors
to your config file (directors go between the second and third "end"
lines). Also, don't forget that order matters -- eg. you can make
Mailman lists take precedence over system aliases by putting these
directors in front of your aliasfile director, or vice-versa.
# Director to handle a list's main posting address. Should come first
# in case there's a list named, eg. "mailman-owner". Automatically
# detects list existence by looking for lists/$local_part/config.pck
# under MAILMAN_HOME.
mailman_main_director:
driver = smartuser
require_files = MAILMAN_HOME/lists/$local_part/config.pck
transport = mailman_transport
# Handle all the other addresses related to a list: -bounces, -admin,
# etc. Same trick works to detect list existence thanks to the
# 'suffix' option.
mailman_director:
driver = smartuser
require_files = MAILMAN_HOME/lists/$local_part/config.pck
suffix = -bounces : -bounces+* : \
-confirm+* : -join : -leave : \
-owner : -request : -admin
transport = mailman_transport
Routers for Exim 4
~~~~~~~~~~~~~~~~~~
In Exim 4, there's no such thing as directors -- you instead need to add
two routers. Also, the canonical order of the configuration file was
changed so routers come before transports, so the routers for Exim 4
come first here. Put these two routers somewhere after the "begin
routers" line of your config file, and remember that order matters.
mailman_main_router:
driver = accept
require_files = MAILMAN_HOME/lists/$local_part/config.pck
transport = mailman_transport
mailman_router:
driver = accept
require_files = MAILMAN_HOME/lists/$local_part/config.pck
local_part_suffix = -bounces : -bounces+* : \
-confirm+* : -join : -leave : \
-owner : -request : -admin
transport = mailman_transport
Transports for Exim 4
~~~~~~~~~~~~~~~~~~~~~
The transport for Exim 4 is the same as for Exim 3; just copy the
transport given above to somewhere under the "begin transports" line of
your Exim config file.
Notes
-----
Exim should be configured to allow reasonable volume -- e.g. don't set
max_recipients down to a silly value -- and with normal degrees of
security -- specifically, be sure to allow relaying from 127.0.0.1, but
pretty much nothing else. Parallel deliveries and other tweaks can also
be used if you like; experiment with your setup to see what works.
Delay warning messages should be switched off or configured to only
happen for non-list mail, unless you like receiving tons of mail when
some random host is down.
Problems
--------
* Mailman will send as many MAIL FROM/RCPT TO as it needs. It may result
in more than 10 or 100 messages sent in one connection, which will exceed
the default value of Exim's smtp_accept_queue_per_connection
This is bad because it will cause Exim to switch into queue mode and
severely delay delivery of your list messages.
The way to fix this is to set mailman's SMTP_MAX_SESSIONS_PER_CONNECTION
(in ~mailman/Mailman/mm_cfg.py) to a smaller value than Exim's
smtp_accept_queue_per_connection
* Mailman should ignore Exim delay warning messages, even though
Exim should never send this to list messages. Mailman 2.1's
general bounce detection and VERP support should greatly improve
the bounce detector's hit rates.
* List existence is determined by the existence of a config.pck file
for a list. If you delete lists by foul means, be aware of this.
* If you are getting Exim or Mailman complaining about user ids
when you send mail to a list, check that the MAILMAN_UID and
MAILMAN_GID match those of Mailman itself (i.e. what were used
in the configure script). Also make sure you do not have
aliases in the main alias file for the list.
Receiver Verification
---------------------
Exim's receiver verification feature is very useful -- it lets Exim
reject unrouteable addresses at SMTP time. However, this is most useful
for externally-originating mail that is addresses to mail in one of your
local domains. For Mailman list traffic, mail originates on your
server, and is addressed to random external domains that are not under
your control. Furthermore, each message is addressed to many recipients
-- up to 500 if you use Mailman's default configuration, and don't tweak
SMTP_MAX_RCPTS.
Doing receiver verification on Mailman list traffic is a recipe for
trouble. In particular, Exim will attempt to route every recipient
addresses in outgoing Mailman list posts. Even though this requires
nothing more than a few DNS lookups for each address, it can still
introduce significant delays. Therefore, you should disable recipient
verification for Mailman traffic.
Under Exim 3, put this in your main configuration section:
receiver_verify_hosts = !127.0.0.1
Under Exim 4, this is probably already taken care of for you by the
default recipient verification ACL statement (in the "RCPT TO" ACL):
accept domains = +local_domains
endpass
message = unknown user
verify = recipient
which only does recipient verification on addresses in your domain.
(That's not exactly the same as doing recipient verification only on
messages coming from non-127.0.0.1 hosts, but it should do the trick for
Mailman.)
SMTP Callback
-------------
Exim's SMTP callback feature is an even more powerful way to detect
bogus sender addresses than normal sender verification. Unfortunately,
lots of servers send bounce messages with a bogus address in the header,
and there are plenty that sound bounces with bogus envelope senders
(even though they're supposed to just use an emtpy envelope sender for
bounces).
In order to ensure that Mailman can disable/remove bouncing addresses,
you generally want to receive bounces for Mailman lists, even if those
bounces are themselves not bouncable. Thus, you might want to disable
SMTP callback on bounce messages.
If you do header and envelope callbacks, you can disable them for bounces to
mailman lists (it is quite common for internal hosts to bounce with a non
reachable internal address). The idea is that you typically don't want non
bounceable Email, but you'd better accept bounces to mailman lists so that
you can unsubscribe the people who are bouncing.
With Exim 4, you can accomplish this using something like the following
in your "RCPT TO" ACL:
# Accept bounces to lists even if callbacks or other checks would fail
warn message = X-WhitelistedRCPT-nohdrfromcallback: Yes
condition = \
${if and {{match{$local_part}{(.*)-bounces\+.*}}
{exists {MAILMAN_HOME/lists/$1/config.pck}}} \
{yes}{no}}
{yes}{no}}
accept condition = \
${if and {{match{$local_part}{(.*)-bounces\+.*}}
{exists {MAILMAN_HOME/lists/$1/config.pck}}} \
{yes}{no}}
{yes}{no}}
# Now, check sender address with SMTP callback.
deny !verify = sender/callout=90s/check_postmaster
If you also do SMTP callbacks on header addresses, you'll want something
like this in your "DATA" ACL:
deny !condition = $header_X-WhitelistedRCPT-nohdrfromcallback:
!verify = header_sender/callout=90s/check_postmaster
[XXX all this stuff is completely untested by me! -Greg]
Doing VERP with Exim and Mailman
--------------------------------
VERP will send one email, with a separate envelope sender (return path),
for each of your subscribers -- read the information in
~mailman/Mailman/Default.py for the options that start with VERP. In a
nutshell, all you need to do to enable VERP with Exim is to add these
lines to ~mailman/Mailman/mm_cfg.py:
VERP_PASSWORD_REMINDERS = 1
VERP_PERSONALIZED_DELIVERIES = 1
VERP_DELIVERY_INTERVAL = 1
VERP_CONFIRMATIONS = 1
(The directors/routers above are smart enough to deal with VERP
bounces.)
Virtual Domains
---------------
One approach to handling virtual domains is to use a separate Mailman
installation for each virtual domain. (Currently, this is the only way
to have lists with the same name in different virtual domains handled by
the same machine.)
In this case, the MAILMAN_HOME and MAILMAN_WRAP macros are useless --
you can remove them. Change your directors (routers) to something like
this:
require_files = /virtual/${domain}/mailman/lists/${lc:$local_part}/config.pck
and change your transport like this:
command = /virtual/${domain}/mailman/mail/mailman \
${if def:local_part_suffix \
{${sg{$local_part_suffix}{-(\\w+)(\\+.*)?}{\$1}}}
{post}} \
$local_part
current_directory = /virtual/${domain}/mailman
home_directory = /virtual/${domain}/mailman
List Verification
-----------------
This is how a set of address tests for the Exim lists look on a working
system. The list in question is quixote-users@???, and
these commands were run on the mems-exchange.org mail server ("% "
indicates the Unix shell prompt):
% exim -bt quixote-users
quixote-users@???
router = mailman_main_router, transport = mailman_transport
% exim -bt quixote-users-request
quixote-users-request@???
router = mailman_router, transport = mailman_transport
% exim -bt quixote-users-bounces
quixote-users-bounces@???
router = mailman_router, transport = mailman_transport
% exim -bt quixote-users-bounces+luser=example.com
quixote-users-bounces+luser=example.com@???
router = mailman_router, transport = mailman_transport
If your "exim -bt" output looks something like this, that's a start: at
least it means Exim will pass the right messages to the right Mailman
commands. It by no means guarantees that your Exim/Mailman installation
is functioning perfectly, though!
Document History
----------------
Originally written by Nigel Metheringham <postmaster@???>.
Updated by Marc Merlin <marc_soft@???> for Mailman 2.1, Exim 4.
Overhauled/reformatted/clarified/simplified by Greg Ward <gward@???>.
------------------------------------------------------------------------
--
Greg Ward <gward@???> http://www.gerg.ca/
Never put off till tomorrow what you can put off till the day after tomorrow.
