On 2010-09-18 at 18:15 +0200, Axel Rau wrote:
>
> Am 18.09.2010 um 17:01 schrieb Dave Evans:
>
> > Better than patching deliver.c? Personally I reckon that parsing
> > the main log
> > would be the much better option, unless you have some requirement
> > for the
> > updates to be synchronous with respect to delivery.
> No, that's no requirement.
> My rationale was:
> - having one more process in the chain is one more point which may fail,
See, the question is not "is there an extra step?" but "what happens
when it fails?". Ie, what is the coupling involved in the extra step?
Sometimes, adding the extra step *increases* overall reliability by
loosening the coupling. If separate components can fail without taking
down the entire system, then you can build a more reliable system than
any one of the components. Tight coupling reduces system reliability to
that of the least reliable of the coupled components.
With Exim talking directly to MySQL for the logging:
(1) Can Exim continue to work when MySQL is down?
(2) If Exim can continue to work, does that mean data is lost when
MySQL is down?
By contrast, with Exim writing to logs, the dependency is on the
filesystem not being full (a dependency which exists anyway). Exim can
continue writing to logs irregardless of the state of the MySQL injector
tool or MySQL itself.
If either the injector tool or MySQL fails, the progression of data to
MySQL stalls. But the data is not lost and mail continues to be
delivered. Once the source of the problem is identified and remedial
action taken, you can start back up the injector and continue feeding
data in. You just need to make sure that you can process the logs fast
enough to not only be able to update MySQL in real-time, but to be able
to do so faster than real-time, so that you can catch up after such an
incident.
So, I advise working from the logs, benchmarking the injector tool and
making sure that you can process a day's worth of logs in under an hour,
so that after a week of problems you still catch up in less than a day.
You can separately have your monitoring system do a query which returns
the most recent Exim event timestamp from MySQL and alert if it falls
too far behind "now", on the assumption that you're always receiving
*some* mail, so no events is a sign of problems.
> - log format may change over time as this is no official API (you told
> me, this is not true)
> - logging my fail (network problem on the way to loghost etc.)
The same network problem would take out data on the way to the MySQL
host. You should have Exim logging to local disk and if you want those
logs on a central host, other than the MySQL one, be taking action to
synchronise the logs with ability to catch back up, much as stated above
for the MySQL case.
Regards,
-Phil
