On Thu, Aug 20, 2009 at 01:20:41PM +0400, Andrey wrote:
>
> Can anyone help me to find a fastest method to search in exim logs and
> get relevant message sessions or errors for particular from, to or both
> fields. The problem is that our exim logs are huge (>300Mb) and exigrep
> does not fast in that case.
>
> As I understand, exigrep is constructing 2 hash tables based on log
> file. First contains relevant message session messages by id and second
> contains flags if it contains searching pattern. Then it prints out only
> hash records from first table if flag=1. But the problem is that exigrep
> reads exim log file line by line and unfortunately it is not good
> solution in case of huge logs.
>
> Is there faster algorithm to perform search in exim logs based on
> from,to or both fields. Also I need to print error messages that are not
> relevant on message id, for example too many recipients messages.
I regularly search through bzipped logs which in bzip-format is
combined probably 1.2Gb or bigger. The combination of bzip2 and
exigrep is working for me.
Another tool we are using is exilog which saves the logs from our
three mail servers remotely in a postgresql database. Using exilog
for basic queries is quite fast if you want to look for spesific
emails. However it does not record log entries of aborted connections
due to errors.
Regards
Johann
--
Johann Spies Telefoon: 021-808 4599
Informasietegnologie, Universiteit van Stellenbosch
"But I would not have you to be ignorant, brethren,
concerning them which are asleep, that ye sorrow not,
even as others which have no hope. For if we believe
that Jesus died and rose again, even so them also
which sleep in Jesus will God bring with him."
I Thessalonians 4:13,14
