On Sat, 22 Jul 2000, Sascha E. Pollok wrote:
SEP >Dear folks,
SEP >
SEP >unfortunately I am not very experienced with larger scaled mail-concepts
SEP >but I currently have an idea and now I am wondering if this is
SEP >completely bullshit or perhaps a really possiblity.
SEP >
SEP >Imagine there is a location with only one machine (Linux, PIII 650 with
SEP >256 MB Ram) handling some thousand users mailboxes with POP3. It is
SEP >used for POP3 and SMTP.
Hmm. Firstly, your machine is overkill. I use a 486 with 16 MB ram for
300 users, and it spends much of it's time idle. Mind you it is ONLY
running SMTP. Spool files are on another machine mounted using NFS.
I suspect that a single pentium configured as you say would be able to
handle hundreds of thousands of messages per day. I'd be interested to
see what the average number of emails sent per user is for various
situations.
SEP >In case the machine gets too slow we could get a bigger and faster
SEP >server but that's not the solution for scaling larger environments.
SEP >What about several machines (e.g. 5) handling POP3/SMTP for
SEP >users (internal usernumbers like 00000001, doesn't matter).
SEP >No user would ever talk to them directly.
This is what yellow pages are for.
SEP >Additionally there are some "proxy" POP3-Servers in front of them
SEP >that get the queries from the customer's mailclients. The POP3-Proxy
SEP >looks the given User-ID up in a database and gets the information
SEP >which internal POP3-Server and with what User-ID is in charge
SEP >of that user's account. The POP3-Proxy connects to the internal
SEP >machine, logs in with the internal ID and then we just forward
SEP >the commands through the "proxy".
Messy. Suppose you have 3 file servers. Server 1 has /home/a_g
server 2 has /home/h_p and server 3 has /home/q_z
Call them spool1 spool2 and spool3
User ajsmith has a home directory of /home/a_g/ajsmith
Configure smtp agent (exim) to deliver to $home/mail/inbox
Configure pop servers similarly.
Set single DNS record to point to a round robin DNS server
so that each SMTP server is taken in turn. If one server
goes down, remove that IP from the DNS record.
Thus MX for your.domain.com points to exim.domain.com.
But the IP for exim.domain.com points in round robin style to
exim1.domain.com exim2.domain.com exim3.domain.com which have
IP's of 31.xxx.yyy.zzz 32.xxx.yyy.zzz and 33.xxx.yyy.zzz
Similarly everyone has pop.domain.com as their pop server.
In turn pop.domain.com points to each of your pop servers in turn.
Each exim box mounts spool1:/home/h_p on /home/h_p
Similarly for spool2 and spool3.
The pop servers do this similarly.
You run NIS for your home directory lookups. Have a master
administration server for this, and make all the exim and pop
servers be slave machines. By doing this, you avoid the network
stack for home directory lookups. Once you get past a few hundred
names NIS is faster than flatfile even if you do go over the network.
Connect the mess together with a fast switch.
Don't put pop and smtp onthe same server. Pop servers have a rep
for being insecure, so they should be isolated from the rest as
much as possible. (In general: As you grow, make boxes more and
more specialized.)
Redundency:
You can go for very expensive hardware that is almost certain
to stay up, or you can cheat a bit, and still have a very small
message loss rate:
Each of the spool servers has a second disk that it exports,
but normally isn't mounted. Each server runs an rsync daemon
Spool1 rsyncs to spool2:/home/a_g. Machine spool2 rsyncs to spool3:
/home/h_p and spool3 rsyncs to spool1:/home/q_z.
All the NFS mounts are done using the automounter.
Now if one of the spool machines goes down, then after 3 minutes
inactivity (or whatever) the automounter looks for it's second
choice for that mount. You may have some minor glitches
with file locks, but overall there shouldn't be much of a problem.
Similarly, you could have the exim machines rsync to eachother.
This one is more problematical, as the spool files change
with great speed. May want to set it up that only a message
that has been defered gets synchronized to another machine.
This would require some manual intervention if an exim server went
down, in order to deliver the backlog from the other box.
Next Level of protection is to have machine set up with disk
images for all three functions. If one of your servers goes down,
you reconfigure that machine to replace it's function.
Pop servers would be the fastest to swap in. spool servers the
slowest, as you need to restore their spool files from the
machine that swapped in. Not sure how to do this completely
seamlessly. I think that current users of that machine would
get bumped. However, if you take the dead machine out of the
RR DNS, put the new machine into the DNS, AND set up the
user login so that on login of a user it checks for the existence
of $home/mail/inbox, and if it doesn't exist copies it in from
the backup spool disk... (Whew!)
Be an interesting job. Probably lots of gotchas that I've missed.
SEP >
SEP >For incoming mail there would be several inbound mailservers
SEP >that look in a database to see to which POP3-server and to which
SEP >User-ID the mail has to be redirected.
Don't need to do this. Use their login, and use NIS to branch
to an appropraite directory. The key is getting both exim and
pop to look in $home/mail/inbox. The Operating system has features
to do the lookups quickly.
SEP >What do you people that are more experienced in large scaled
SEP >mail-systems think of that? In case there are not too much good
SEP >arguments against this, I'd like to get vi and gcc ready to start
SEP >writing the Pop3-"Proxy" :-)
(Well, I'd use vile or xvile instead of vi... <grin> ...
I think that modifying popper to look in home directories would be
both easier, and would have net speed increases over your system.
Sherwood Botsford | sherwood@???
Sorcerers Apprentice | Math Dept, U of A, Edmonton, AB T6G 2G1
System Administrator | Tel: 780 492 5728
Trouble shooter | Fax: 780 492 6826
