Re: [Exim] Mail storage formats/ POP/IMAP/webmail daemon/etc…

On Sun, 1 Apr 2001, Peter Radcliffe wrote:

> "Dave C." <djc@???> probably said:
> > I dont want the weaknesses of either model.
>
> Get coding, then.
>
> > Ok, you have 50,000 users popping their mailboxes, how do you distribute
> > this to multiple servers?
>
> Firstly I'd avoid NFS like the plague.
>
>
> The last time I had to build something scalable to solve a similar
> problem but with more users I laid it out to be scalable from the
> start.
>
> Several back end machines with pop mailboxes on them, mirrored or
> RAIDed disk, etc. Each machine had a percentage of the users on them.
>
> Then, several front end machines running pop proxies, behind load
> sharing or simple multiple A records.

What software did you use for the pop proxies? DId you also support
IMAP?

>
> Links between the front and back end machines are private network.
> Initially one switched net, but later scales to multiple interfaces
> and multiple networks. Scale limitation is number of interfaces per
> front end machine and number of back end machines; but with switched
> gig ether and since the traffic is almost completely unicast this
> is a reasonble top end.
>
> Clients connect to pop.foo.com, hit the front end machines. The
> proxies have either a table or a DB lookup to see which of the back
> end servers the mailbox is stored on and proxies the POP request to
> that machine. At that point I used simple dbm lookups but these
> days I'd be tempted to have redundant LDAP databases or private
> DNS lookups.
>
> Support code to migrate users from one back end machine to another can
> (fairly crudely) balance load and when the overall load gets too high,
> you add more front end/back end machines and migrate users mailboxes
> to the new back end machines. Any migration only inconveniences a
> small proportion of your users and monitoring to produce statisitics
> on which users are producing how much load can make balancing the load
> much much easier.
>
> If a front end machine goes down, take it out of your load sharing.
> If a back end machine goes down you only inconvenience a proportion of
> your users. A hot spare can return this to production fairly quickly
> if you have on site people. If you need higher availability then you
> can use dual attached disks so two machines can access each chunk of
> disk. One machine going down no longer inconveniences any users and
> you can load balance more finely between the two machines at the POP
> proxies (the proxy code can connect to the lesser loaded machine which
> has access to the disk with the user's mailbox).
>
> SMTP mail gets delivered either to the pop front end machines, which
> direct it to the appropriate back end machine, or you can use a
> seperate set of SMTP servers on the private network scaling as
> appropriate.
>
> During a user migration you can queue mail for that user on the SMTP
> servers and poke them to deliver it all when the mailbox migration is
> done. The POP proxies can deal with the POP hits for the user during
> a migration.
>
>
> Not simple, nor easy to set up initially, but once it's done you
> can scale it at any point in the system as it is needed.
>
> P.
>
>

--