--On Friday, December 04, 2015 18:52 +0000 Jeremy Harris
<jgh@???> wrote:
> On 04/12/15 18:41, John C Klensin wrote:
>> "Ready" is a question only you can answer -- this is a fairly
>> large complex of changes and the standards are (deliberately)
>> fairly clear that advertising SMTPUTF8 commits you to the
>> whole collection. I'd certainly prefer to have it wait for a
>> bit than to have it done in haste.
>
> That's why I asked "who's using it".
>
> People are answering "I'd like to see it" - but I'd really like
> to know whether it (and the others) have seen any production
> use.
For i18n, the answer is "yes" but "it is complicated".
There is an intermittent effort to maintain a list of
implementations and deployment that I can check on if you would
find it helpful.
Basically, the situation is this: If I create and advertise an
address with a non-ASCII local part, that address is accessible
only if
(1) My MUA supports the needed protocol bits
(2) If POP3 or IMAP involved, the MUA has been upgraded to use
i18n-enhanced versions of those protocols.
(3) The IMAP and POP3 servers have to be upgraded.
(4) The mailstore has to have whatever necessary done to it to
support non-ASCII message headers, etc.
(5) My delivery MTA has to support SMTPUTF8
(6) So does every single MTA that might end up in the path,
back to the submission server.
And that still doesn't get the message to me unless the sender's
MUA has been upgraded. I also need to maintain fully-ASCII
addresses or address aliases and make sure my likely
correspondents know about them in case the above chain doesn't
work (for those who don't know, we spend a lot of time talking
about on-the-fly "downgrading" from non-ASCII and ASCII
addresses and it just doesn't work -- this is not like an
upper-case/lower-case relationship).
(1)- (3) are not Exim problems and (4) probably is not either,
but one needs to start somewhere and the MTAs (and MUAs
including POP3/IMAP clients) are probably the place to start.
If we could do flag days on the Internet, the above would be a
great candidate. We can't. So, deployment tends to be within
language or script groups. The result, as of the most recent
reports that I've gotten, is that there are several million
production users in China (probably, by itself as much
deployment and use as the other things on your list put
together). There are active development and deployment efforts
in Russia, Ukraine, and some Arabic-speaking countries (I don't
know the current status of any of those efforts). There is a
major effort in Thailand (again, I don't know its status -- for
all I know, they have deployed and have things in use).
Also note that the current SMTPUTF8 specs had a set of
experimental predecessors that were implemented, deployed in
test environments (some fairly large), and tested. The current
specs are the result of that learning experience, so I consider
them quite a bit more mature than the typical Proposed Standard
coming out of the IETF these days.
The thing that all of the above efforts have in common is that,
from a user standpoint, the relevant script is very different
from Latin script and use of Latin script is hard, unfamiliar,
and/or politically incorrect. Users and MUA designers have a
good rule of thumb: when communicating with someone within
country whose non-ASCII address you know, use the non-ASCII
address. When communicating with anyone else, use the ASCII one
(and probably write in English).
For places in which the difference between an i18n address and
an all-ASCII one is a character or two out of the address,
fussing with i18n addresses will rarely be worth the trouble
unless there is much wider deployment, leading to a traditional
chicken-and-egg problem.
Recommendation: I would really like the see the whole SMTPUTF8
collection in exim if you think you are ready. Noting one of
Victor's comments, we shouldn't be surprised to find bugs even
if you think you mostly have it right but, if you know of places
where the implementation is likely to be weak or fragile, I'd
recommend waiting. It will also be hard to test this in
production and end up end until there are a lot more
fully-upgraded MUAs and POP/IMAP servers out there or a lot of
upgraded webmail interfaces.
However, if you decide that you are not ready to do the whole
SMTPUTF8 package this time, I'd recommend two incremental
improvements for your list -- things that will help with the
SMTPUTF8 upgrade effort but have some value in their own right.
(1) Get ready to handle non-ASCII domains in envelopes and maybe
headers. In theory, no one should send Exim one of those in
either a forward or backward-pointing address unless SMTPUTF8
is turned on, but it would allow running an Exim server on a
host with a primary domain name that contained IDN labels and
would permit sorting out much of the code that determines
whether labels in U-label or A-label form are used in headers
(notably trace fields) or other contexts. You will eventually
need a flag or two that are set differently for SMTPUTF8 and
"normal" cases, but it should be possible to develop and deploy
the code with SMTPUTF8 off. As Victor points out, it is also a
necessary step for working with TLS certs containing IDNs, etc.
It probably also means that you need to be able to validate
U-label <-> A-label conversions.
(2) Be sure that, at least for UTF-8-based encodings, you have
good mechanisms for converting between Encoded Word form and
strings directly encoded in UTF-8. You will need it, and,
again, it is modular relative to the rest of the pieces.
best,
john
p.s. For a variety of reasons, I'm not a big DANE fan and you
may want to discount the comment that follows on that basis.
However, my perception from several of the relevant IETF lists
is that there is still a lot of controversy about details of the
relationship among DANE, MTA-MTA TLS, and assorted other mail
system issues. Until those discussions stabilize (again, with
the understanding that Victor and others may think they are a
lot more stable than I do), it would perhaps be unwise to
include anything in a production release that you would be
uncomfortable making incompatible changes to later.
