On Sun, 6 Jul 2003 13:16:26 -0400 (EDT)
"Greg A. Woods" <woods@???> wrote:
> [ On Sunday, July 6, 2003 at 10:16:02 (-0500), Jerry Jorgenson wrote: ]
> > Subject: Re: [Exim] error response rate limiting vs. overloaded
> > systems
> >
> > Idle connections take up resources: 1 slot in the TCP stack per
> > connection. Most systems that I know about don't have an infinite TCP
> > stack. But if you know of one please post it.
>
> Of course -- but would you rather have one socket and one process
> sitting idle and twiddling its thumbs, or have to close and reopen
> sockets, and fork ne processes, multiple times a minute just to send the
> same error response to the same client?
But that's not the problem, except for some misconfigured mailers, which
aren't in the majority, or even that frequent.
>
> Holding a connection open and idle for some time before you send the
> last line of an error response will save you massive amounts of
> resources at only a tiny expense. Pay a little, save a lot.
And prevent your normal customers from getting their legitimate mail.
> I'm not expert enough at queuing and operational theory to explain the
> math (and it wouldn't be simple if it were explained), but I have more
> than enough direct experience to understand intuitively that this is
> true.
Not in a large shop, it isn't.
> I can also assure you that the bigger the highway and the more traffic
> there is on it, the bigger the savings gained by delaying that matter
> transport. Indeed if 75% of the cars are black but only 5% of those go
> through the matter transport to come back and cause you trouble then
> delaying the whole 75% is still less far expensive than not delaying
> any of them. Brute force engineering only works when everyone is well
> behaved and even then there's still always a finite limit to how far you
> can go.
And how would you differentiate between the black and white cars, when
some white cars (e.g. large ISPs) act like black cars, but are legitimate,
and you absolutely must let them through without any delay (but there are
way too many of them to put on an exception list).
> If you have a machine that can accept 6000 separate connections in the
Hey, that's just from one spammer. There would be many others trying it at
the same time.
> space of one minute then it doesn't matter if you hold them open for
> another minute when you you reject them all because if you actually had
> accepted an e-mail message from any significant number of those
> connections then your machine require far greater resources to store,
> process, and deliver those messages. Holding the connections open and
> idle in order to rate-limit the error responses is a tiny cost in
> comparison to actually delivering messages.
No, all that would do would be to block 6000 real users. I don't know
about your shop, but mine declares a national emergency if even one user
gets refused because the server has used up all of the available
connections.
> What does matter is that if you reject them and drop them immediately
> and then they all re-connect again immediately to try to re-deliver the
> message you rejected, well then you're hosed because they're likely to
> just keep doing that for several minutes, hours (or as I've seen in some
Not really, because the firewalls can detect that behavior and block them.
> cases, days). Meanwhile if you simply hold those connections open
> you'll have lots of resources left over to accept another few thousand
> connections from legitimate clients which make no errors and which
> simply deliver messages to you.
>
> I.e. holding errant connections open and idle is a tiny cost in
> comparison to going into a tight accept, reject, close loop.
Not when the goal is to have a good customer experience (no mail delayed
for more than five minutes, no connections refused due to maximum
connections).
> > Correct, you actually have to have the capacity to filter out these
> > before they hit the mail servers. It's not fun, of course if it was
> > fun, people would pay to do it :-)
>
> Well, you guys have fun with your silly firewalling tricks. I'll stick
> to things that work automatically and don't need my direct attention and
> management.
If the delay method worked (and it's been tried, but it failed to achieve
the results you postulate), I'd be using it rather than the system now in
use, which admittedly is way more complex than it should be). But the
delaymethod doesn't work and only brings management down on us because of
maximum connection errors (and I'd say that few ISPs have more hardware
running then the shop I work in does).
Jerry
--
Jerry Jorgenson
jerry@???
http://www.j3iss.com/
