Re: split the load?

Top Page
Delete this message
Reply to this message
Author: Greg A. Woods
Date:  
To: exim-users
Subject: Re: split the load?
[ On Thu, November 21, 1996 at 20:08:56 (+0000), Piete Brooks wrote: ]
> Subject: Re: split the load?
>
> > My guess is that's only a "SHOULD" too, though if it's a "MUST".....
>
> I can't see where you are going ...


If DNS RFCs say a server MUST do round-robin serving of equal (MX) RRs,
then any that don't are, by definition broken, and since the *best* fix
for this is to fix the DNS server, then that's definitely the approach
I'd take..

> I disagree.
> Not every MTA manager can dictate how all the NSs for their zone work.
> Also, simply having the primaries for a zone running a RoundRobin NS isn't
> very efficient at randomising at the fine level. DNS RRs are cached, so if
> the NS used by the remote MTA doesn't RoundRobin, then the MTA will keep
> using the same host first until the TTL expires. Forcing people to set a low
> TTL isn't acceptable ....


Oh, I realized that -- poor excuses, but unforuntately all to often real.

> ... that's what people have been plugging ...
> However, I am not convinced -- why should it *undo* it ?


Well, first of all "round-robin" DNS is far from random -- it may appear
random to the average entity doing repetitive queries, but it most
certianly doesn't appear as "random" to the hosts being hit by the
service requests resulting from such queries. I think. I'd re-read the
DNS RFCs to be sure, but my interpretation of "round-robin" is more in
line with a circular queue, not a random sprinkler. Interestingly in
quick successive queries for MX records for cl.cam.ac.uk shows this to
be true.

Unfortunately since the DNS server always returns all of the MX RRs
without regard to their preference value, we must rely on correct
handling in the MTA in order to ensure their "round-robin-ness" will be
preserved.

Also, in the simplest case of only two equal MX RRs pointing at separate
hosts, any randomization of the alternating first reply will sometimes
result in the second reply being selected, which may result in one
server getting more connections than the other, and in the worst case
(true random selection by multiple MTAs) one server always getting all
of the connections.

> If (4) is wrong, let's come out into the open, and say it's too expensive !


"Doing The Right Thing" for randomization is far from trivial, though
once you've done it right you can hopefully re-use the code.

Doing it right in this case may not be of much help -- stats at work!

Even "Doing The Right Thing" for MX handling without adding randomizaton
is not necessarily trivial. Ideally "stacks" (FIFO order) of MXs for
each unique preference value must be constructed, then the stacks must
be sorted by their preference value but without disturbing their
internal ordering, then finally selection from the lowest value stack
must begin until a connection succeeds. This is the only way I can
think of to preserve the original DNS ordering while still allowing the
mailer to start with the lowest preference RRs.

(Of course by now I could have looked up the DNS RFCs if I hadn't been
writing this message! ;-)

-- 
                            Greg A. Woods


+1 416 443-1734            VE3TCP            robohack!woods
Planix, Inc. <woods@???>; Secrets of the Weird <woods@???>


---------- FYI, the MX query round-robin cycle in practice ----------
# yes, this is really a sequential series of real query examples!
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
ttyp1:<woods@very> $ host -t mx cl.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw3.cam.ac.uk
cl.cam.ac.uk            MX      10 ppsw4.cam.ac.uk
cl.cam.ac.uk            MX      20 sun2.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      21 sun3.nsfnet-relay.ac.uk
cl.cam.ac.uk            MX      8 bescot.cl.cam.ac.uk
cl.cam.ac.uk            MX      4 heaton.cl.cam.ac.uk