Re: [exim] Exim4 optimisation

Praveen Chinthala wrote:

*snip* (no longer mainstream details)
>
> Hi,
>
> Thanks Marc and Bill for your replies.
>
> The huge number of mails I am talking about are tabular data sent to
> external list of users (all external) from a distribution email address.
> However these emails are sent every hour to the distribution from a separate
> host in the local network to the smarthost. The smarthost (exim4 server)
> then sends the email. My point is, sending the emails every hour is not
> configured on the exim4 server but it's just a cron job on another server to
> flush the emails every hour to the smart host.

OK - but even with an 'all at once' chron job, said server's MTA (if you can get
some help) could probably be reconfigured to space those out a bit over time.

ELSE...

You can throttle the rate at which Exim accepts, and said corespondent *should8
adapt to that if it is smtp-compliant. (not known to me).

Unless each message user-specific, it would be better to receive ONE message for
a list of recipients, then use a Mailing List Manager (don't say em-ell-em, SA
goes BF) to spread out the load.

>
> Hence the exim4 server (smart host) gets huge number of emails from that
> particular host at exactly between 0 and 2 mins of everyhour. However the
> exim4 server is configured to empty its queue every 15 mins as set in
> /etc/default/exim4 file and its also running separate daemons.

The 15 minutes won't ordinarily matter so long as you are not running 'queue_only".

OTOH, you may need/wish to do just that - as it unloads the peak number of child
processes instantiated with the massive arrival.

If you DO run queuue-only, then 15-minute intervals are way too infrequent to
avoid creating yet-another (set of) peak situation(s). Try decreasing that.

> So, when it
> actually gets these many emails and when it tries to send, the server's load
> shoots up to 95 and comes back in less than 2 mins.

So long as it faileth not, there is nothing wrong with running a Unix server at
that load level for a few minutes of the hour.

You could, OTOH, invoke Exim with a reduced priority to insure you can always
(at least) get in with ssh, etc..

man nice

> I would like to know is
> there any way to reconfigure exim4 such that the load is controlled and all
> the messages sent without any delay as and when they are received.
>

That is a contradictory wish.

There is always *some* delay, if only because the system bus and network are not
infinite, let alone CPU and RAM.

Speaking of being bandwidth-bound, you could also implement 'dummynet'
throttling for the source IP of the initial broadcast and/or your incoming port
25. If the slowest link in the present chain is, for example, a BT or Colt E1 @
2048 Mbps then emulating a 256 Kbps or 512 kbps link with dummynet (or
equivalent), could spread your incoming burst out over more than 2 minutes.

> Hope I am making sense, and if not please be patient with me as I am not
> very much experienced in exim4.
>
> Thanks a lot.
>
> Praveen
>

Sense enough (now).

But if fast turnaround is really your goal, and is working for you as-is, I'd
say go for a faster server (dual-core, 4 GB of fast RAM, SCSI or SATA 'hardware'
RAID, separate sending and receiving NIC's) that can do the job with more
reserve capacity.

HTH,

Bill