Re: [exim] Message duplication if To/Cc contain alias and on…

2008/5/11 Oliver von Bueren <maillist@???>:
>>>> The result, from all my experience of harnessing DSPAM, is that as
>>>> soon as pipe transport is used to preprocess the message, Exim's logic
>>>> of eliminating duplicates cease to function. I had to use the
>>>> preprocessing at the stage of actual delivery, inserting
>>>> transport_filter
>>>>
>>> If the pipe injects the message again into exim it cannot know about
>>> duplicates already seen, because it's an entirely new message.
>>>
>>> You should decide at which stage you want to "spread" your message,
>>> before or after DSPAM. (Before: you can't apply "user policies", After:
>>> you're probably doing duplicated work.)
>>>
>>
>> I'm sorry, what do you mean by 'spread'? If the DSPAM is invoked via a
>> router, then the described situation will result in multiple copies of
>> the same messages - regardless of whether DSPAM router is located
>> before or after system_aliases one.
>>
>>
> What he said is, that exim can not control any duplication of message in
> the setup you have at the moment because the message gets feed to exim
> as separate messages. From your log file, you feed exim three (3)
> messages. To illustrate this, I've changed the order of the log lines into
> the order exim sees as a session each. (stripped time stamps and SMTP
> info for better readability)
>
> 1. That part is the delivery of the message from the net with authenticated esmtp and a delivery to two addresses, user1 and userlist, both delivered through the "method" dspam_spamcheck.
> For this session I assume the the following SMTP envelope sender (aka RCPT TO):
> user1@??? and userlist@???
> You see, no duplication, it's delivered to each address once. The alias expansion and duplicate control detect that user1 is already in and only one delivery to it is done.
>
> 1JtHF4-00017m-2L <= user1@??? P=esmtpsa S=649 id=48200340.6020408@???
> 1JtHF4-00017m-2L => user1 <user1@???> R=dspam_router T=dspam_spamcheck
> 1JtHF4-00017m-2L => userlist <userlist@???> R=dspam_router T=dspam_spamcheck
> 1JtHF4-00017m-2L Completed
>
> 2. An entirely independent SMTP session to exim with the message from user1 to user1.
> The RCPT TO here will be user1@???
>
> 1JtHF4-00017t-8g <= user1@??? U=exim P=local S=1921 id=48200340.6020408@???
> 1JtHF4-00017t-8g => user1 <user1@???> R=procmail T=procmail
> 1JtHF4-00017t-8g Completed
>
> 3. Yet another completely independent SMTP session to exim with a message with two end deliveries.
> The RCPT TO here will be userlist@???, which is expanded according to the alias with an additional recipient of user1@???
>
> 1JtHF4-000181-Gt <= user1@??? U=exim P=local S=1904 id=48200340.6020408@???
> 1JtHF4-000181-Gt => userlist <userlist@???> R=procmail T=procmail
> 1JtHF4-000181-Gt => user1 <userlist@???> R=procmail T=procmail
> 1JtHF4-000181-Gt Completed
>
> What you expect, which can't be done, is that exim takes care of
> duplicate elimination across session 2 and 3.
>
> What you have to do is find a way that your dspam-setup does not create
> an smtp session for each recipient to feed the messages back into exim,
> meaning session 2 and 3 should really be one with RCPT TO of
> user1@??? and userlist@???. In that case, exim would
> detect the condition and deliver the message once to user1 and once to
> userlist. Or, you have to make sure that the alias expansion does not
> take place for session 2 and 3.

That brings me back to the original question: is it really DSPAM at
fault in this case? It is called through pipe transport. All DSPAM
does is to read data from stdin and send modified output to stdout.
Any piece of software that behaves in the asme manner will beget
duplicates of exactly the same kind.

If the pipe transport wouldn't open a new Exim session for every
message, the problem could go away, couldn't it?

I had to incorporate DSPAM as transport filter for the transports
performing actual delivery.

Could someone tell me whether it is at all possible to set up 'message
preprocessing' via pipe transports without causing the message
duplicates problem?

Thanks.