On Fri, 23 Jul 2004, Derek Martin wrote:
> Well, I can't say that I'm surprised. I'm sure that this behavior of
> exim has alarmed more than a few people... And I think rightly so.
Well, if it is as bad as "alarm", then let's get this generally
discussed by the experts - for example on the ietf-822 list and/or any
other list where the Big Gurus live.
> > We are talking about RFC 2822 here. That is all to do with MUAs, which
> > are the components that handle message content. MTAs should not be
> > concerned with content, other than to add Received: headers. (Of course,
> > some of them do stray a bit. :-)
>
> I have to disagree rather strenuously here, on several counts. First,
> there are lots of valid reasons for MTAs to mess with headers.
... and none of them are mentioned in RFC 2821, where I see this:
As discussed in section 2.4.1, a relay SMTP has no need to inspect or
act upon the headers or body of the message data and MUST NOT do so
except to add its own "Received:" header (section 4.4) and,
optionally, to attempt to detect looping in the mail system (see
section 6.2).
OK, that's specifically a "relay" SMTP. I suppose this comes down to
"submission" vs "relay", and Exim behaves as "submission" only for -t.
(Note, incidentally, that there are MUAs that submit using SMTP via
127.0.0.1; Exim does not recognize that automatically as special.)
How can Exim tell when it is receiving a "submission"?
1. For -t it is obvious that it is a submission.
2. Without -t, it is not (I contend). Consider a message that came from
outside, was piped to Majordomo or MailMan, and is then being
re-injected for final delivery. This isn't a "submission", at least
not from the local host.
3. What about messages coming in via SMTP? Some come from other MTAs,
some come from MUAs running on PCs etc. How can Exim tell which is
which?
I chose only to recognize the obvious -t case because the others are
murky ratholes. (See also the Sendmail documentation quoted below.)
> For example, the MTA can re-write headers to ensure that return mail
> will be deliverable.
It can, but SHOULD or MUST it? RFC 821 says:
3.8.1 Header Fields in Gatewaying
Header fields MAY be rewritten when necessary as messages are
gatewayed across mail environment boundaries. This may involve
inspecting the message body or interpreting the local-part of the
destination address in spite of the prohibitions in section 2.4.1.
but it is clear that this is referring to gatewaying out of the Internet
environment.
> Secondly, I think your interpretation that RFC 2822 applies only to
> MUAs is directly contradicted by the following paragraph from the
> introduction:
>
> However, some message systems may use information from the contents
> to create the envelope. It is intended that this standard
> facilitate the acquisition of such information by programs.
>
> Here, I think it is clear that the term "message system" refers, at
> least in part, to the MTA. It also states that the intent of the RFC
> is to make it easy for the "message system", again the MTA, to create
> an envelope. This is the purpose of the -t option which you have
> stated that you don't like.
With respect, understanding "message system" to mean "MTA" (even in
part) is just one interpretation.
> What I will grant is that upon re-reading portions of the RFC, I can
> see that an interpretation that some of it seems to refer to the MUAs
> is possible. However, I will say that at best, the RFC is vague and
> ambiguous in a number of ways that are important; I think it is not a
> well-written RFC.
On that point we are in *complete* agreement! I was involved in the
DRUMS discussions that resulted in 2121 and 2822; after a while I got
too frustrated (and things were taking far too long as well) so I
dropped out again. They SHOULD have written entirely new documents
instead of patching up the old ones (IMHO).
> I will include the idea that it must ensure that at most only the
> recipient named in the Bcc line receives a copy of the messge with a
> Bcc line mentioning his address (and no others).
Not correct. The Bcc line is permitted to contain a list of ALL the Bcc
recipients:
"but the recipients on the "Bcc:" line get a separate copy of the
message containing a "Bcc:" line. (When there are multiple recipient
addresses in the "Bcc:" field, some implementations actually send a
separate copy of the message to each recipient with a "Bcc:"
containing only the address of that particular recipient.)"
> However, RFC 2822 DOES NOT ALLOW for the recipients named in the To:
> and the Cc: lines to receive copies of the Bcc: lines.
Sure. But it does not make it clear whose job it is to enforce this.
That's due to the obscurity of the RFC. I believe it should be enforced
as early as possible, by the MUA, because of the difficulty an MTA has
in deciding when it is handling a "submission". For "foreign" messages
that originate elsewhere an MTA has no right to make this kind of
change.
> The fact is, the vast majority of other MTAs DO strip the Bcc lines,
Under what circumstances? Always?
> The point is, Exim's behavior is a departure from what the rest of the
> world does and expects,
If that is true, I am prepared to consider changes, BUT: (a) I need to
be convinced that it is true and (b) there needs to be a PRECISE
definition of the circumstances under which this action is to be taken.
I do not consider "always remove any Bcc lines from any message" to be
good enough.
> Exim is only deficient in handling #2. The RFC states that this line
> may exist in the message, and it also states that it is not to be
> passed on to recipients other than those named in the Bcc line.
But your MUA has just passed it on to those recipients!
> Logically, if it a) exists in the message, as pased on by the MUA, and
> b) must not be passed on to recipients which are unnamed in the Bcc
> line, then the only place where it is possible for this situation to
> be remedied is within the scope of the processing done by the MTA.
... as long as the MTA knows it is receiving a message from a "local"
MUA in "submission" mode.
> The best case scenario is for the MTA to ensure that seperate copies
> of the message are delivered to each of the recipients named in the
> Bcc line, with a modified Bcc line mentioning only that individual
> recipient.
That is only one choice. It is also permissible to name all Bcc
recipients. The MTA does not know which is desired.
And what if the Bcc recipient turns out to be an alias of a mailing
list? (For example). And worse, if the mailing list contains one of the
non-Bcc recipients? (OK, perhaps I'm getting carried away here.)
> Ok, what you're suggesting here is that the MUA generate individual
> copies of the message for each Bcc recipient.
Or two copies, one for each set of recipients, if you want all the Bcc
recipients to be listed to all of them. Or only one copy if you don't
want any Bcc lines in the message at all. Your choice. The RFC allows
all 3 formats. Only you know which you want - the MTA doesn't.
> This certainly is possible, but is not practical. The MTA already
> does this as a normal part of processing outgoing mail.
Not unless it's qmail. If all recipients are routed to the same remote
MTA, Exim will send (by default) ONE copy, as suggested by RFC 2821.
One particular case of this is when all outgoing mail is being sent to a
smarthost.
[Aside: you certainly do NOT want to generate an individual copy for
each recipient on a mailing list delivery to thousands of recipients
(who will typically be "Bcc" recipients, though never listed in any
actual Bcc: header) as a regular event when delivering via a smarthost.]
> The MTA alread can do and already does this kind of processing.
Nope. Exim keeps only one copy of each message.
> Even in Exim's case, it can do and does this processing in some cases;
Nope. All it does in the -t case is to REMOVE the Bcc line when it
receives the message and generates the envelope.
> To ask the client to do it when the MTA can (and usually will anyway)
> is a needless waste of computing resources.
Sorry, this just isn't the case with the way Exim is implemented.
> If this argument doesn't convince you, then I expect nothing will, so
> I'll sign of here.
I'm not convinced. However, I have a record of being persuaded on other
issues if enough people have a consensus. I suggest:
(1) This issue is aired (again) on the exim-users mailing list. It's
been 6 years... [exim-dev is a new list, mainly concerned at present
with setting new development infrastructure].
(2) This issue is aired on the ietf-822 mailing list.
(3) This issue is aired on any other appropriate mailing lists.
(4) Somebody finds out what the other MTAs do, in detail. I have only a
rather old copy of a Sendmail manual, on a CD-ROM. It says this:
35.10.4 Bcc:
Blind carbon copy
(RFC822)
A blind carbon copy is a copy of the mail message that is sent to
one or more recipients without the knowledge of the primary
recipients. Primary recipients are listed in the To: and Cc: lines.
When there are multiple blind carbon copy recipients, knowledge of
each other is also hidden.
When run with a -t command-line switch (to gather recipients from
the headers), the sendmail program achieves this end by saving a
list of all the blind carbon copy recipients, deleting the Bcc:
header line, and then delivering to each blind carbon copy
recipient. (See the Apparently-To: header.)
The Bcc: header should never be declared in the configuration file.
The field for the Bcc: header must contain one or more properly
formed addresses. Where there is more than one, each should be
separated from the others by commas.
I do not know if later Sendmails behave differently, or indeed if
Sendmail does things with Bcc other than what is stated above.
(4) Somebody finds out what the other MUAs do, in detail. I use Pine,
and it strips Bcc lines.
(5) Somebody searches for other reference material. A very quick google
has not thrown up anything significant (there was one old email that, as
it happened, supports my view).
I don't have time to do any of this now. I'm off to the SANOG meeting in
Nepal in a few hours. Maybe in a few weeks when I get back, if you have
done no more, I might try to write the question and get it asked.
Regards,
Philip
--
Philip Hazel University of Cambridge Computing Service,
ph10@??? Cambridge, England. Phone: +44 1223 334714.
