Re: [Exim] Performance bottleneck scanning large spools.

Top Page
Delete this message
Reply to this message
Author: Nigel Metheringham
Date:  
To: exim-users
Subject: Re: [Exim] Performance bottleneck scanning large spools.
cgknipe@??? said:
> On queues like this - as mentioned in this thread - won't an dedicated
> smtp server with some beefed up software be an better solution anyways
> ?


Maybe... there are many ways of skinning this cat.

> Some ISP's in South Africa lately mirged, and with the mirge, the user
> base jumped sky high (currently, over 700,000 email accounts alone).
> Surely with this type of load on SMTP, shouldn't something like SUN
> handle this *ALLOT* better?


You are starting to believe Sun's marketing material. The Sun
filesystem (straight system V UFS) is pretty lousy. You may improve
things significantly by moving to VxFS or similar (at a cost of say 3K
pounds per processor), but it really needs the application end to be
designed for that sort of scaling.

My own prefered solution for huge mail systems (ISP class - Enterprise
might have rather different characteristics), is to split the functions
down:-

  1. Have a small number of local delivery boxes.  These deliver to 
local
     mailboxes, and never send outgoing mail.  In general they will have
     nothing at all in the queue since local delivery is near instant
     (exception is if local delivery has blocked).  These machines are
     mentioned as your incoming MXes.


  2. Have a reasonable number of outgoing relays.  These take outgoing 
messages
     from your customers, and relay it out.  These would tend to 
acquire large
     queues, except as an additional hack you configure them so all 
messages that
     cannot be delivered first time are passed off to your fallback 
relays.


  3. Have a small number of fallback relays which handle recalcitrant/ 
     long-undelivered mail.  These will fill up with crud.  Put other
     means in to handle stuff on these machines - ie don't run queue 
runners,
     as such but manually queue run on them (its a bit of perl), maybe
     consider special case routing and other hacks.


All of these boxes can be relatively small, relatively cheap, and you
have optomised your most common cases to work fast with a minor cost to
the uncommon cases (which would take a load of time to deliver anyhow).
The downside is that its less simple than the minimum, but possibly be
as simple as it can be with reasonable performance.

[There are other minor tweaks - outgoing mail that happens to be for a
local domain may be short circuited into local delivery on (2) without
ever being sent to the (1) servers - depends on your local spool.]

    Nigel.
-- 
[ - Opinions expressed are personal and may not be shared by VData - ]
[ Nigel Metheringham                  Nigel.Metheringham@??? ]
[ Phone: +44 1423 850000                         Fax +44 1423 858866 ]