On Tue, 29 Jan 2008, Ian Eiloart wrote:
> > But I don't see anywhere where it says I can't use "User-Agent: Whatever
> > I want" in email headers.
>
> I didn't say that you can't use it. I said it's non-standard. However,
> although I'm unable to find the rfc that suggests it, it's quite common
> to use X-something for non standard headers.
It was spelled out in RFC 822, 4.1 "Syntax":
extension-field =
<Any field which is defined in a document
published as a formal extension to this
specification; none will have names beginning
with the string "X-">
and later:
4.7.4. EXTENSION-FIELD
A limited number of common fields have been defined in
this document. As network mail requirements dictate, addi-
tional fields may be standardized. To provide user-defined
fields with a measure of safety, in name selection, such
extension-fields will never have names that begin with the
string "X-".
Names of Extension-fields are registered with the Network
Information Center, SRI International, Menlo Park, California.
4.7.5. USER-DEFINED-FIELD
Individual users of network mail are free to define and
use additional header fields. Such fields must have names
which are not already used in the current specification or in
any definitions of extension-fields, and the overall syntax of
these user-defined-fields must conform to this specification's
rules for delimiting and folding fields. Due to the
extension-field publishing process, the name of a user-
defined-field may be pre-empted
Note: The prefatory string "X-" will never be used in the
names of Extension-fields. This provides user-defined
fields with a protected set of names.
However it appears little if any of this language appeared in the revision
RFC 2822.
The Appendix on differences mentions:
11. Extension header fields no longer specifically called out.
I guess the intent is that RFC 2822 merely defines the syntax of the
overall message, stating certain minimal requirements, but leaves it open
to other documents to define additional fields as required.
So it would appear now anything you like can appear as a header, as long
as it conforms to the syntax of RFC 2822, in which case it is covered by
that "standard", but unless the header name and content are actually
defined in another document, they are in some sense also "non-standard".
So maybe "registered" was the better word to use :)
User-Agent does of course come from the HTTP world, which is obviously
superficially structurally similar to an RFC 2822 Internet Message. I
guess it is no surprise it was re-purposed into the email world.
I note RFC 2076 specifies that "X-Mailer" is also "Non Internet standard".
Is there actually a defined 'standard' field to indicate the software used
as UA? Anyone?
> It's certainly unwise to use something that might later be registered
> for a different purpose.
The point of the X- prefix was to guarantee a namespace that would never
be used by any subsequent standard, and was thus safe for arbitrary use.
The suggestion seems to have gone away over time for whatever reason.
> >> I suspect that Microsoft are adding spam points to messages with
> >> non-standard headers, which would explain why some messages are
> >> acceptable when they don't contain a user-agent header.
> >
> > Email systems add all sorts of unusual headers to emails. I'd suggest
> > that it's less likely for a spam to contain a "non-standard" header
> > than a ham, not the other way round.
I think we can probably all agree that it is a somewhat bizarre heuristic
for hotmail to be using, given all the arbitrary application-specific
fields that appear in email (as commented by Chris, MS themselves are
guilty of this one).
But if that's what they are doing, it's no wonder the results appear odd
from the outside.
"SmartScreen(tm)", don't you know?
Jethro.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Jethro R Binks
Computing Officer, IT Services
University Of Strathclyde, Glasgow, UK
