Re: [exim] exim performance: how to send 10^6 letters daily?

Peter Volkov wrote:
> Hello. At my work we are trying to use exim4 as a mail gateway for our
> news mailing lists with currently about 1000000 subscribers, thus we
> need to send 10^6 mails daily or better twice faster since we want all
> mails to proceed before they became too old. Currently we are using exim
> 4.69-9 on Debian. After reading mailing archives we've enabled the
> following options in exim configuration file:
>
> split_spool_directory
> queue_run_max = 100
> remote_max_parallel = 200
> queue_only_load = 8
> deliver_queue_load_max = 12
> smtp_connect_backlog = 50
> smtp_accept_max = 500
> smtp_accept_queue_per_connection = 1000
>
> We are running exim process with -odqs -q1m options. We've put hints
> database (/var/spool/exim4/db) on tmpfs. And in result on Intel
> Corporation E7520 with 2 Intel(R) Xeon(TM) CPU 3.00GHz, with 4G RAM and
> IDE software raid1 exim is not enough fast (we've managed to send 15 000
> letters per 30 minutes) So the question is are there any hints how to
> improve exim performance?
>
> Currently it looks like our performance bottleneck is our discs -
> currently we use software raid to improve performance, but still we have
> only about 50-70Mb speed (as hdparm states). But if we put, say scsi
> disks does this makes us much faster? Or should we use some different
> mail server software (suggestions, please?)?
>
> I found that to improve performance it was suggested to send to most
> popular domains first but it's not very easy (not to say problematical)
> in our setup.
>
>
> Another problem is related to -odqs option. As I understand the manual
> this option tels exim to do only routing and *no* delivery. Then why I
> see the following messages in the log file:
>
> 2009-04-28 11:34:09 1LyhpP-0004TO-HX no immediate delivery: load average 9.40
>
> ? Probably this is our performance bottleneck too?
>
>
> In any case thank you for any hints,
> --
> Peter.
>
>

What I've had luck with is putting the queues in a ram disk. Then
setting it up with a fallback host so that if the message isn't
delivered on the first try it transfers to another exim server that
keeps the queues on a hard drive. I'm processing about 10 million
messages a day on such a server (quad core phenom 8 gigs ram) and it
works really well. Of course if someone yanks the plug out you lose the
ram disk queue.