On Mon, Dec 04, 2000 at 09:04:59AM -0500, Theo E. Schlossnagle wrote:
> > On Mon, Dec 04, 2000 at 03:07:40PM +0200, Warren Baker wrote:
> > > Do you think it would be feasible to have the ability to let
> > > Exim log to a MySQL/Postgres/<whatever> db ?
>
> We do it. Not in real time, but it is neither impractical nor insane to do
> so. We devlier about 1500 messages per second at peak (which as I understand
> is more than most people.) This equates to 2-3 log inserts. 4500 inserts per
> second on a well tuned database is completely feasible. Will you need a beefy
> machine? Of course, but anyone who really needs to process logs this way must
> have relatively large pipe and can probably afford a big enough box to run the
> db.
How big is your database? Do you rotate it every day? What are you
running it on?
Can you give details as I am intrested how to do it.
> Technically, this should be a very simple coding effort within exim. The
> mysql facilities are already there and the logging is already. You just need
> to add a few exim.conf parameters and hook into the log code for certain log
> events -- all of the plumbing is already inside of exim.
Technical and practical are diffirent, but if you say you did it... good
stuff.
> No matter how you arrange your logs, you cannot do adhoc queries on them
> quickly or easily unless you have a formal structure around them. A database
> is a damn good solution.
Very true. Log tend to be badly orginised for any amount of work done on
them.
> > As for log searches, how many do you *really* do?
>
> We do a lot. Hundreds to thousands per hour. Maybe you don't run that large
> of a site, but we send at least 14 million emails each day. If someone
> doesn't get the email that they expect to get, we need to know why. So, we
> (actually they -- through a web interface) do a look up and see the entire
> transaction from Exim's perspective. It is great!
I do run the largest UK mail system -- or did till last month and a
promotion later.
There are some security aspect here, I assume that you only let the user
look at his own mail, but what about mailing lists and the like?
How are you dealing withn spam? We do freeze mail and that will show in
the logs, thus letting a spammer know how to avoid our measures.
It seems to me that the system is great, but of limitted use. I found
lusers cann't even read a bounce message much less actull understand
what log lines mean.
> It is feasible to do with perl, but we have more than 10 outbound mailserver
> and I don't want to go looking on each one. Besides, a MySQL database is
> exactly what I am looking for. Basically normal Exim logs with an index on a
> few of the fields and an SQL query engine.
I'm impressed. Nice setup... Show how one can be wrong, doesn't it?
--
Dr Yann Golanski Senior Developer
Please use PGP: http://www.kierun.org/pgp/key-planet
