Re: [Exim] Handling high volume, low availability domains

On Wed, 29 May 2002, Sheldon Hearn wrote:

> This works nicely, with one drawback: my generation time has slowed
> down by at least 3 times. I don't believe that this is a function of
> the network connection time to the specialized listener.
>
> I believe that it's because the exim -bS process that I push all the
> generated messages into is now creating msglog files for all non-hotmail
> messages, with contents like this:

can't you arrange your queue feed so that you

- have separate bsmtp files for hotmail and !hotmail addresses (i
suppose you already do)
- feed the !hotmail addresses to your regular exim
- feed the hotmail addresses to the 'other' exim (i'd say basically
the same, only spool_directory being different)

this spares the network connection time, which is not much, and still
leaves

> 2002-05-29 09:54:30 xxx@??? routing defer (-45): domain is in queue_remote_domains

here. for which all i can think of is removing this from exim.

> I think this extra IO is dragging me down, and I don't believe it's
> necessary.

if this is your concern, you could edit all the fsync() calls out of
exim's source. my measures showed *huge* io throughput increase with
fsyncs removed (like, delivering 300k mails from localhost to
localhost, exim queue/mail spool/logs/everything being on the same
disk, time dropped from where i couldn't wait to approximately 1.5
hours). you have to be careful, though, cause if your system crashes,
you'll be left in the middle of nowhere wrt what exactly happened to
your messages. this is relatively easy to do (iirc <10 fsync()s are
there)

also, (not knowing your limitations) there are a number
of ways to increase io throughput:

- use split_spool_directory so that you link the 'letternames' in
groups to different scsi disks
- put the logs onto a different disk as well
- arrange msglog to be on a different disk as well
- put the whole thing onto tmpfs :) (this is fast, although undeniably
this is the one that calls for the rope if your box crashes).
- use fast disks. if you're using scsi, but slower than u160, you're
better off with udma100 ide disks, but only one on each controller.


--
[-]