[exim] Re: Bug in Exim - duplicating messages

Top Page
Delete this message
Reply to this message
Author: Russell Stuart
Date:  
To: exim-users
Subject: [exim] Re: Bug in Exim - duplicating messages
On Fri, 2005-02-25 at 17:09, Russell Stuart wrote:
> Version of Exim:       4.43, + Exiscan patch
> Operating System:      Debian Sarge
> URL will full details: http://www.lubemobile.com.au/ras/exim

>
> I am getting duplicated messages with Exim. The about URL has all
> the details, like logs, cron scripts, -H files, and config files.


I didn't get far looking at the code. I soon as commenced the
problem appeared to go away - I could not reproduce it, even
by sending the original messages that caused the problem. To
I have spent the last 5 hours or so trying to reproduce it.

The conditions trigger the bug are:

1.  A new address is added with a redirect router with the unseen
    directive.


2. There is a retry record pending for new address.

3.  The new address is moved to the front of deliver.c's delivery
    queue by another re-direct router.


4.  A "control = queue_only" is in effect (perhaps - not sure about
    this one).


5. The original address can be successfully delivered to.

If all these conditions are met, then each time a queue running
comes along, it will create a duplicate email to original
address. This will happen until the retry record expires.

Here is a shell script that demonstrates the bug, by re-creating
these conditions. A copy is on the web site, file name "bug.sh".
It does not do any real deliveries.



#!/bin/bash
#
# To run this script:
#
#   a.  Stop all Exim queue runners, and prevent any new ones from
#       starting.
#
#   b.  Create a new, empty directory.  Ensure the directory is
#       world readable, searchable and writable.  Ensure its parents
#       are all world readable and searchable.
#
#   c.  cd to the new directory.  Copy this script into it.
#
#   d.  Run the script as root.
#
# What it does:
#
#   This script demonstrates a bug in exim.  Under these conditions:
#
#     1. A new address is added with a redirect router with the
#        unseen directive.
#     2. There is a retry record pending for new address.
#     3. The new address is moved to the front of deliver.c's
#        delivery queue by another re-direct router.
#     4. "control = queue_only" is in effect (perhaps - not sure
#        about this one).
#     5. The original address can be successfully delivered to.
#
#   Every new queue runner will generate a duplicate email to
#   the original address until the defer expires.
#
#   The script doesn't do any real deliveries.  Instead, the
#   delivery is handled by the the lmtp emulation below, which
#   just records all successful delivery attempts.
#
# Outputs:
#
#   - A list of deliveries made by exim is printed on stdout.
#     If exim worked there should only be one to each address.
#   - A number of exim -d+all logs are created in the current
#     directory.
#   - The configuration file used is left in the current
#     directory.
#



#
# Implement the LMTP protocol for Exim. This is used by
# bug-exim.conf below.
#
# If the mail contains a line starting with "421" no delivery
# is simulated. Instead a 421 error is returned to exim.
#
# Every successful delivery done by this transport is logged
# to bug-lmtp-deliveries.log.
#
if [[ x"$1" = x"--lmtp" ]]
then
dir=$(dirname $0)

echo 220 Hi!

  while :
  do
    while read line
    do
      case "$line" in D*) break;; esac
      case "$line" in Q*) echo "221 2.0.0 Goodbye Exim!"; exit;; esac
      case "$line" in R*) rcpt=${line#*:}; rcpts="$rcpts ${rcpt%?}";; esac
      echo "220 Thank you Exim!"
    done


    echo "354 Go Exim!"
    reply=
    while :
    do
      case "$line" in 421*) reply="421 Error!";; esac
      case "$line" in .*) break;; esac
      read line
    done


    if [[ -n "$reply" ]]
    then
      echo $reply
      exit
    else
      d=
      for f in $rcpts
      do
    [[ -z "$d" ]] || echo "250 2.0.0 $d OK" | tee --append $dir/bug-lmtp-deliveries.log
    d=$f
      done
      echo "250 2.0.0 $d OK" | tee --append $dir/bug-lmtp-deliveries.log
    fi
  done
fi


[[ $(id -u) = 0 ]] || {
    echo 1>&2 "$0: I must be run as root!"
    exit 1
  }


if ps -ef | egrep -w '[e]xim|[e]xim4'
then
echo 1>&2 "$0: Please stop exim and all queue runners!"
exit 1
fi

#
# We must be in the local directory.
#
[[ -s bug.sh ]] || {
echo 1>&2 "$0: I must be run as ./bug.sh!"
exit 1
}
chmod +x bug.sh

rm -f bug-*.{conf,log}

>bug-lmtp-deliveries.log

chmod a+rw bug-lmtp-deliveries.log

#
# Build the exim.conf
#
cat >bug-exim.conf \
<<======================================
acl_not_smtp = not_smtp
begin acl
not_smtp:
accept control = queue_only

begin routers

duplicate:
driver = redirect
local_parts = x
data = z@www.exim.org
unseen

rename_z:
driver = redirect
local_parts = z
data = y@www.exim.org

lmtp:
driver = accept
transport = lmtp

begin transports

lmtp:
driver = lmtp
batch_max = 1000
command = $PWD/bug.sh --lmtp
retry_use_local_part = true

begin retry
* * F,1d,1h
======================================

#
# First we must generate a retry record for
# y@www.exim.org.
#
( echo "To: y@www.exim.org"
echo
echo "421"
) | exim4 -C $PWD/bug-exim.conf -t -d+all >bug-delivery-y.log 2>&1

exim4 -C $PWD/bug-exim.conf -q            # A queue runner to the retry record
exim4 -Mrm $(exipick -r '.@www\.exim\.org' -i)    # Don't need that message any more


#
# Now create the message that causes the problem.
#
( echo "To: x@www.exim.org"
echo
echo "Test"
) | exim4 -C $PWD/bug-exim.conf -t -d+all >bug-delivery-x.log 2>&1

exim4 -C $PWD/bug-exim.conf -q -d+all >bug-delivery-1.log 2>&1
exim4 -C $PWD/bug-exim.conf -q -d+all >bug-delivery-2.log 2>&1
exim4 -C $PWD/bug-exim.conf -q -d+all >bug-delivery-3.log 2>&1

exim4 -Mrm $(exipick -r '.@www\.exim\.org' -i)    # Don't need that message any more


#
# Print the results.
#
echo "Deliveries done by lmtp transport:"
cat bug-lmtp-deliveries.log