On Thu, Apr 22, 2004 at 10:54:12AM -0500, Edgar Lovecraft wrote:
> Matthew Byng-Maddick wrote:
> > While looking at PIPELINING support within SAUCE last night, the
> > following issues were highlit by Ian Jackson. The RFC2920 (STD 60) spec
> > for SMTP command pipelining includes no precise definition of a "command
> > group",
> How so? the RFC does not define 'what exactly is in a group', it does
> however define quite clearly what commands ARE ONLY last command, no matter
> what is in the group, these commands ARE ALWAYS defined as last.
Imagine that I send (in one TCP command) all the commands in a group. It
doesn't *actually* say I need to wait for an answer.
> Lets look at Section 3.1
> <START OF RFC SNIPET>
> 3.1. Client use of pipelining
> Once the client SMTP has confirmed that support exists for the
> pipelining extension, the client SMTP may then elect to transmit
> groups of SMTP commands in batches without waiting for a response to
> each individual command. In particular, the commands RSET, MAIL FROM,
> SEND FROM, SOML FROM, SAML FROM, and RCPT TO can all appear anywhere
> in a pipelined command group. The EHLO, DATA, VRFY, EXPN, TURN,
> QUIT, and NOOP commands can only appear as the last command in a
> group since their success or failure produces a change of state which
> the client SMTP must accommodate. (NOOP is included in this group so
> it can be used as a synchronization point.)
>
> Additional commands added by other SMTP extensions may only appear as
> the last command in a group unless otherwise specified by the
> extensions that define the commands.
> </START OF RFC SNIPET>
> That seems pretty clear to me, as there are some commands that MUST and
> CAN ONLY be the last of a group (a group can be one command by the way).
CAN ONLY is not part of RFC2119, and the word MUST does not appear in that
text. It seems pretty clear until you actually look at the language, which
doesn't prohibit the scenario I've outlined above. The RFC is confused, and
full of layer violations (expecting the Layer 4 stuff to know about what's
going on at Layer 3).
> Lets continue with Section 3.1...
> <START OF RFC SNIPET>
> The actual transfer of message content is explicitly allowed to be
> the first "command" in a group. That is, a RSET/MAIL FROM sequence
> used to initiate a new message transaction can be placed in the same
> group as the final transfer of the headers and body of the previous
> message.
>
> Client SMTP implementations that employ pipelining MUST check ALL
> statuses associated with each command in a group. For example, if
> none of the RCPT TO recipient addresses were accepted the client must
> then check the response to the DATA command -- the client cannot
> assume that the DATA command will be rejected just because none of
> the RCPT TO commands worked. If the DATA command was properly
> rejected the client SMTP can just issue RSET, but if the DATA command
> was accepted the client SMTP should send a single dot.
>
> Command statuses MUST be coordinated with responses by counting each
> separate response and correlating that count with the number of
> commands known to have been issued. Multiline responses MUST be
> supported. Matching on the basis of either the error code value or
> associated text is expressly forbidden.
> </START OF RFC SNIPET>
> Here again, the 'what exactly is in a group' is not defined, but what
> starts or ends the group is defined, as are the rules that the client must
> follow in regard to response codes.
Yes, but at no point does it explicitly say that a command group terminator
MUST wait for a response. Please point out the part where it says this. It
only says that it must be the end of a TCP send operation.
> > and in particular, although it's implied that the "last command
> > of a command group" is one that should wait for replies, the server:
> Not just implied, the client MUST wait for all replies.
Quote, please.
> > | (9) MUST NOT flush or otherwise lose the contents of the TCP input
> > | buffer under any circumstances whatsoever.
> That is to ensure that the server has seen the entire 'command group' sent
> by the client, it has nothing to with the client not waiting for ALL
> responses to the 'command group' which the client MUST do. There are also
Quote, please.
It may be "to ensure...." however, if you think through what is happening,
because there's no restriction on the command group actually having to wait
for a reply (despite your assertions of a MUST, it doesn't exist), it also
means that you can't legally (as a server) throw away stuff that exim
currently considers to be a pipelining violation.
> things that the server must clearly NOT buffer, and the rules on when to
> send responses to the client is very clear also.
The fact that the server "doesn't buffer" is neither here nor there.
> Look at Section 3.2:
[irrelevant, snipped]
> > Now, this means that it's quite reasonable to suggest pipelining
> > violations in the situations where no pipelining has been advertised, but
> > if it has, then it appears that the client can do pretty much what it
> > wants (within the standard).
> Not it cannot. If the client sends an 'ONLY LAST COMMAND' (EHLO or DATA
> for example) then it MUST wait for all responses to the command group to be
> returned, as the client MUST check all of the return codes before sending
> the next command group.
Quote, please.
> > Other things about this standard appear to be totally braindead, (VRFY
> > causing a change of state - you what!?) so it's not clear that a client
> > can't believe that it's allowed to send other things, because of the
> > unclearness between what goes in one TCP send operation and the places
> > where it is pausing for a response.
> There is nothing unclear about it, the client MUST check ALL statuses for
> each command in the group.
Yes, but it doesn't say that it MUST do that before sending another group.
> > In particular, if one does several send operations one after the other,
> > but without waiting for a response (having switched off Nagle, of
> > course), then it would appear to be conformant to STD60 as specified,
> Nope, this is just not true, on or off the client MUST wait for all
> responses to commands given in a command group before continuing.
Quote, please.
> > but would cause a synchronization error in exim.
> As it should.
Your opinion, not what is said in the RFC. (It so happens that it is my
opinion too, but I'm curious about this RFC)
> Let us finish looking at Section 3.1...
> <START OF RFC SNIPET>
[blocking causes deadlock if you don't respect the windowsize]
> </START OF RFC SNIPET>
> I believe that the 'Failure to perform this check can lead to deadlock
> conditions' answers your question, if it does not wait, then the client
> would not be conformant to the standard, thus, causing a 'synchronization
> error'.
Rubbish.
> > So, is it a bug for exim to say "synchronization error" in those other
> > situations?
> I would say no, even if what exactly a 'group' is can be fuzzy, when a
> group starts, and when a group ends is not.
But it's fuzzy because it talks about it in terms of TCP send operations,
not as pausing to wait for responses.
> MUST END A GROUP:
> EHLO
> DATA
> VRFY
> EXPN
> TURN
> QUIT
> NOOP
> any command added by other SMTP extensions, i.e. AUTH amongst others
You will notice VRFY and EXPN in there... What change of state of the
server do you think these produce?
> With that definition of when a group ends (therefore anything else starts
> a group), it is quite clear that the client MUST wait for ALL responses to
> ALL commands before continuing with the next group.
*blink*
Because elephants are grey, the ocean must be blue.
Non sequitur.
Please feel free to quote the relevant parts of 2920 to back up your
assertions where I've put "Quote, please." Because those are *exactly* the
bits it doesn't make clear.
MBM
--
Matthew Byng-Maddick <mbm@???> http://colondot.net/
(Please use this address to reply)
