Re: [exim] Message logging with SQL

Top Page
Delete this message
Reply to this message
Author: Ron White
Date:  
CC: exim-users
Subject: Re: [exim] Message logging with SQL
On Thu, 2010-05-06 at 15:05 +0200, Karl Fischer wrote:
> > ...
> >> How about a different approach? Have you looked at syslog-ng?
> >> Having exim write to syslog and having syslog-ng write to SQL DB might
> >> be it ...
> >>
> >> see: http://www.balabit.com/network-security/syslog-ng/
> >
> > or
> > http://wiki.rsyslog.com/index.php/EximAmalgamatedLog
> >
> > Negative point to all these approaches (exilog and similar ones
> > as well): Every single user needs to maintain the rules and
> > pattern and will never be sure to have the complete logging
> > until a detailed and complete walk through the exim logging-code
> > and the pattern provided by the log-project in question.
> > The code-walk is to be repeated with every new exim version.
>
> Well, syslog-ng is a syslog replacement and therefore logs
> everything it gets, so nothing gets lost ...
> ... however, to /interpret/ the content of the logging you
> need to define your own rules and patterns, that's right.
>
>
> > A centrally maintained/supported solution (interface/lib/module
> > or at least log format documentaion) would be preferred.
>
> I definitely wouldn't mind having that!
> Is it Christmas already?
>
> - Karl
>

Thanks to Karl & Frank for your kind replies.

I use syslog-ng with Postfix currently, but it is an utter nightmare
parsing out the logs for all the possibles combinations. That said if
anyone else has worked with Postfix you'll know that the logs are
convoluted into different processes each with a different bit of the
puzzle. So far my code to marry all of this up (and it's still buggy)
runs to a couple of thousand line of Perl.

On the other hand, Exim's logs seem much more straightforward to parse.
The Message ID stays constant for the message (it's not handing things
over to cleanup/queue manager and the like) so I'll probably look to
code a simple agent to do the work and cron it for every few minutes. I
know there are some options with syslog-ng for talking to SQL and I'll
look into those first.

It would be a neat feature to have with Exim - the jam and icing on a
cake - but how it would scale is a very good point, and how it would
fail with grace would be interesting.

Thank you kindly for your help. Always appreciated.