Gavin and Philipp,
Here comes my 2+ cents.
It is nice you have good networking equipment, fully managable switches and
good storage solutions. If you are talking high traffic, it is absolutely
essential to completely separate the inbound and outbound systems.
I find that intel boxes are much better delivering mail that suns
(price/performance).
My company currently maintains 20+ linux servers that deliver as much as 3
million emails/day each using exim (all unique with a single recipient) [not
SPAM]. The servers are relatively inexpensive (less than $3k). They are all
dual processor -- THIS IS A MUST as exim is not event driven and requires
multiple processes to delivery messages, thus many many context switches.
They all have 1-2 GB of RAM and dual 9GB fast SCSI drives (one for OS and one
for /var/spool/exim).
I heavily tweaked the exim configuration (playing with different timeouts for
different domains, dedicated queues for delivery to problematic domains, and
much more). Nice thing though is that they have stock kernel parameters.
We only receive about .5 million messages/day, so that is nothing to brag
about. The inbound exim instances are only tweaked slightly (more open files,
more processes.)
We do use mysql for lookups (but not on outbound machines -- they are
dumbed-downb as possible). I argue that the overhead of SQL (which is small)
is well worth it when you consider managing a large cluster than needs to hit
a single database source (easy with mysql master-slave replication). Mysql
can go well over 10000 lookups/sec without breaking a sweat if indexed
properly and if you can do over 10000 deliveries a second on a single
commodity machine, I have some things to learn from you!
For DNS lookups you need a very fast DNS server. After much experimentation,
running a dedicated caching instance of BIND on each machine works best for
us.
Another thing is that logging can cost a lot, so logging to the network
instead of to a file can help tremendously. I have also edited the exim code
extensively to remove most fsync calls on my outbound mailers.
Gavin Sherry wrote:
> > to get back to topic:
> > I'm completly lost with the Hardware thingy... (ever talked to a salesman,
> > asking for some sort of advice from that company, and all he comes up with
> > is "no problem, Sun Server, Storage System, Giga-everything. ...hrm, lets
> > say 8.000.000 US$. when do you want to sign?") grrrr.....
>
> I administer a system which has done up to 1.2 million emails a day (not
> spam :-)). It is running on PC hardware with slackware Linux. I have
> played with the kernel some to allow exim a better running environment -
> 4000 simultaneous hard limit of processes, lots of open files - you know.
>
> There is some other black magic behind it:
>
> * Extensive tweaking of the configuration file.
>
> The Exim documentation is your friend. Read it... again and again and
> again! Play with a setup, then see if you can make it faster.
>
> * Hardware to suit mail dispatch.
>
> Exim is not particularly computationally expensive. It's a good idea to
> know your operating system though. Linux for example will build a large
> file system cache. Looking at the primary box now, it is around about
> 340MB in size. Nonetheless, you need to have fast disk. I am using U160
> SCSI. The mail spool has its own disk so as to take advantage of the SCSI
> configuration.
>
> It is a dual CPU machine - but this is overkill really.
>
> I have 1GB of memory.
>
> * Seperate services
>
> You need to run the pop server on a different machine. Make sure that the
> MTA is not writing to an NFS box - this is very slow, esp. under
> Linux. This can cause problems if you want to do clustering and failover -
> my example is used only for dispatch.
>
> * Lookups
>
> Use Berkeley DB, not mysql, not pgsql, not sql. SQL is slow. You can
> easily plug applications into berkeley DB using the API - available for C,
> perl etc.
>
> Make sure you have a fast DNS near by. Setting up a DNS cache on the same
> machine will not hurt it.
>
> * Batch delivery
>
> See the manual. Its just good.
>
> There is plenty more that I could add. Perhaps Exim needs a 'build a fat
> mail server' howto? It is a popular discussion topic from time to time.
>
> Thanks
>
> Gavin Sherry
> Alcove Systems Engineering
>
> --
> ## List details at http://www.exim.org/mailman/listinfo/exim-users Exim details at http://www.exim.org/ ##
--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
