Re: [exim] The Future of Email is SQL

Etusivu
Poista viesti
Vastaa
Lähettäjä: Mike Richardson
Päiväys:  
Vastaanottaja: exim-users
Aihe: Re: [exim] The Future of Email is SQL
On Sun, Jun 11, 2006 at 01:42:11AM -0700, Steve Lamb wrote:
>
>     Quite frankly the whole shared calendar resource is better done in a web
> application with email notifications.  At least then it is the primary focus
> and one, if coded to standards, is accessible by all instead of it being
> shoehorned on the side of a sloppy email client which then must be shoehorned
> into everyone's PC which has Windows shoehorned onto it.... and still leaves
> out the non-PC people.


We've been using Oracle Collaboration Suite for a while now and it is
actually separate products joined by a web portal or via an Outlook plugin.
However there are separate desktop calendar client binaries for Windows,
Motif (linux), Solaris and Mac. As a Linux user I use the Linux client and,
as you can tell, Mutt for email. It is a good combination - the desktop
calendar client is actually more functional than either the Outlook
(proprietry calendar functionality shoehorned into MS's calendar model) or
the web (UIX based, slow and not very cross-browser compliant).

OCS has an email server which is several orders of magnitude larger in terms
of binary code than exim, and a couple of orders of magnitude less in terms
of functionality. It uses a database (Oracle, obviously) for the storage.

There are some benefits from this but it's not enough. For instance Single
Instance Message Storage (SIMS) is pretty easy with a database. Backup is
simplified: A traditional storage model uses a filesystem and somewhere
between lots (UW IMAP) and metric shedloads (Courier) of files. This can
cause interesting indexing issues. An Oracle database doesn't suffer from
this. Recovery is more of an issue though - recovering an email folder (UW)
or email (Courier) is 'easy' using a traditionally indexed filesystem. Much
less so with a database - you have to start working through
redo/undo/doggydo logs and regenerating the email(s). So which is worth more
to you - short backup times or short recovery times?

Separation of processing and storage can be achieved more easily with
databases but there are other, non-db, approaches. OCS has a middle
tier/application layer for processing SMTP/IMAP etc. and this generates SQL
queries which are passed to the storage layer. An Oracle RAC database gives
a single database with multiple instances and is expandable sideways with
little impact on the middle tiers. The middle tiers are also expandable
sideways without any reconfig of the storage and are stateless - all mail is
queued and stored on the storage layer. I know that there are ways of
achieving this type of replication with other mail storage products but this
approach is rather nice... on paper. It is a shame that OCS's email
application layer implementation sucks so badly really.

People think that having a database should make administration easier -
craft a clever SQL statement and hey presto! The reality is that you have
way too much data to mess with and much too high a chance of screwing
something up. Especially with a proprietory system with limited internal
documentation.

I guess that databases can give you some gains on large scale, high
performance systems but you then need to be able to tune your DB properly,
which requires a whole new skill set for most email administrators.

I'm not as dismissive of DBs for email storage as I used to be but I
wouldn't recommend them unless you really, REALLY know what you doing and
they solve a high priority problem.

Mike

--
Mike Richardson
Networks
IT Services, University of Manchester
Email: doctor@???
*Plain text only please - attachments stripped on arrival*