[exim] The Future of Email is SQL

After considerable experimenting and thinking things through I thought
I'd start a thread on the future of email to start planting the seeds of
where MTA development needs to go. I'm convinced that someday soon we
will all realize that MBOX and MAILDIR are obsolete technologies and
that the future is going to be SQL based storage.

First - before everyone starts screaming about speed comparisons, I'm
not going to go there. Every storage technology has it's advantages and
disadvantages but I'm just going to say that SQL based mail storage is
fast enough. The advantages of SQL has to do with power and not with
speed. Those who would choose it would do so because they want to do new
things that you can do with a database and can't do without one.

SQL has several advantages. You don't have t deal with the quirks of the
underlying file system or OS. It takes care of all the locking issues
and indexing and makes it so that multiple applications can seamlessly
access the data. With an SQL backend email can be stored from the MTA,
read from and IMAP client that accesses the same database, and the spam
filtering engine will have access to the stored email as well.

To give you some examples of what could be done .....

Suppose a spammer sends 1000 phishing spams to your users and then you
figure out that the 1000 spams already delivered is spam. With a
database you can do a query to retroactively delete spam that was
already delivered to the mailboxes. This could also be used to
retroactively delete viruses already delivered.

Spam filtering programs can lookup existing email in existing folders
and compare it with new email already deliverd to help determine more
accurately if a message is spam or not. For example, if the host server
has a reputation for 100% ham then it can deliver new email without
running it through Spam Assassin. If programs like Spamassassin can
access existing email in existing folders it can evaluate new email
using tricks no one has yet considered.

SQL databases allow for multiple masters and slaves and replication that
lets you create a cluster that never fails under any conditions. It
would be far easier to create a system that is always on and always
backed up.

An SQL backend allows you to use a wide variety of tools, programming
languages, operating systems in order for you to easily integrate more
easily than non database systems.

And - this is important - once you have a database then new things that
no one has yet thought of will be possible and new things we've never
heard of will be developed because the new power will lend to the
development of more tricks than you can do without database power.

My point here is - think outside the box. I'm going to be lobbying IMAP
server developers to include SQL backends. exim could pipe data into a
local delivery agent, or it can have features written to write directly
to the SQL backend.

Thoughts ..... ?