[Exim] [Fwd: mail solution]

Hi,

I originally sent this email to the courier-users list, and Mr. Tabor
Wells very sensibly suggested I send it to the exim-users list, so here
it is :)

My apologies if you got it twice.

-------- Original Message --------
Subject: mail solution
Date: Mon, 12 Feb 2001 18:17:26 -0000
From: "Nathan Ollerenshaw" <chrome@???>
To: <courier-users@???>

Hi there,

I've been tasked with finding a large mail solution for between 500,000
and
1,500,000 users, and I've been looking at a bunch of commercial
products,
namely iPlanet Messaging Server, Intermail KX/MX and Sendmail Pro. I've
also
been looking at fully open solutions, and would like to have some
feedback
from people who have used this software day-to-day in large
environments.

The open solution I have foremost in my mind is namely:

Exim 3.20
Courier IMAP/POP3 server with Maildirs

This would be linked to a proprietary Oracle database which contains all
our
user data for authentication. The actual configuration I'm envisioning
is
something like this:

a) two Sun E3500 NFS Servers running Sun Cluster 2.2 or 3 for high
availability. Gigabit ethernet for the NFS network, FCAL attached
storage
with a crapload of spindles and running Veritas.

This would basically export /var/mail which would be a tree of Maildirs
for
each user mailbox that would be created by the MTA on the fly as needed.
The
format would probably be something along the lines of
/var/mail/c/h/r/chrome/Maildir/. There would also need to be quota
checks by
the MTA to limit to 20MB.

b) Two SunFire 280r's with about 4GB of ram each.

These would be running the Exim, and I'd have something like an Alteon
load
balancer in front to distribute incoming SMTP connections over the
machines,
as well as detect if a machine failed so as to push all the connections
to
the remaining machine if needed. The local disks would probably be
A1000s or
T3s for the local spool, striped and mirrored for performance over as
many
spindles as I can cram in.

Incoming mail would be checked to see if it is a local recipient, and if
so,
I'd like to make a call to a custom library which would connect to our
Oracle proxy and ask the proxy if the account exists. If it does, try to
deliver the mail into the /var/mail/whatever Maildir.

c) three SunFire 280r's with 4GB ram for the POP3 and optional IMAP
access.

Initially, we wouldn't offer IMAP access, as this is something we'd want
to
charge extra for (for good reason) but we'd use the Courier-IMAP
distribution's POP3 server for the POP3 access. I'd like to have Alteons
in
front of these machines as well, so I can do load-balancing and
fail-over.

The machine would mount /var/mail, obviously, from the NFS server.

The authentication would be done via a command written by us that would
do
the Oracle check, similar to the authcustom command. The only
reservation I
have with this is the fork & execute for every POP3 connection this
would
incur, so I'd like to have a custom authentication library function
(ideally
the same used by exim) that the Courier IMAP and POP3 daemons call.

Obviously one of the choke points here is going to be the
authentication,
but making that run fast is my problem :)

Hopefully, this gives me a fast mailserver solution where I don't need
to
worry about locking over NFS, and I don't have to worry about any
machine
failing. I should theoretically be able to keep adding machines until
the
point the NFS breaks, at which point I'd split the data over several NFS
mail stores and put all the /var/mail/a-m/* on one cluster and
/var/mail/n-z/* on the other cluster (or whatever was an even split).
I'd
then either get the pop boxes to mount both filesystems over different
gigabit ethernet interfaces.

Yes, at some point this is going to break, but I should be able to scale
this up massively until it does. At least, thats the theory.

Has anyone on this list done anything remotely similar to this, and can
tell
me what their mileage is?

Is there any major holes in what I'm trying to do here, anything that
I'm
missing because I don't know the Courier-imap stuff well enough or Exim
well
enough? Would I be better to bugger off and stop trying to save money?
:)

Currently we effectively have one single (very loaded) E3500 holding the
mail for all our users and running sendmail & qpopper, surely what I've
described here will do better than that :)

Anyway, any and all feedback will be greatly appreciated.

Regards,

Nathan.