Re: [exim-dev] Data retention with Exim

> Given that each member state of the EU can modify the Directive
> 2006/24/EC to fit their own ends (which in fact the UK govt has done)
> it's my belief that providing a fixed format which "fits" is not the job
> of the application, but the job of the sysadmin running the application.

Right now, I am not aware of the national differences, but suggested to
add a log format to select the country, in case there are any. From the
various answers, it sounds like that's indeed required.

> I personally do not believe that patching Exim to fulfil the directive
> fully is necessary; more pertinently I don't believe it's possible given
> the myriad different local interpretations and transpositions applied by
> EU member states. On top of that, putting code in like this may actually
> breach regulations in other parts of the world (IANAL though).

Not a myriad really, just as many as countries in the world that ask
for data retension. ;)

I don't believe the code would violate any regulation, as long as it is
optional to use. After all, Exim can already log lots of things.

> My reading of the Directive and several interpretation documents leads
> me to believe that there is no one format which is necessary, as long as
> it's possible to post-process the retained data to locate the
> information - which in our case, exigrep is very good at.

Several people on the list suggested post-processing the mainlog. To me,
several reasons forbid that:

I am no longer free which log selectors to use. Logging data must
be reasoned by a purpose (if you care about privacy, which I do), and
depending on the purpose and changes of it, I want to log less or more
than required for data retention. Independent logfiles for mainlog and
data retention solve that conflict nicely. Otherwise I had to strip
the mainlog after extracting the data retention information.

The latency requirements of data retention and mainlog may differ, too.
I am not yet aware of a maximum latency allowed by the German law,
but heard telephone switches use 15 minutes, but anything up two 24h
would probably be OK. Should less than 24h be required in the future,
post-processing becomes even more painful and I need to rotate the
mainlog before I really want to.

I don't trust exigrep. Does it handle quoting right? Can I be sure it
does by construction? The mainlog is friendly to humans, and I like that,
but data retention is required by machines and asks for 100% accuracy,
best achieved by simplicity, like setting a logfile option and selecting
the format for Germany.

I did not send a patch to the list yet, because the actual code only adds
the new logfile so far - a trivial patch, and I assumed there would be
opinions against comitting anything in that area.

Michael