Jason Meers wrote:
> Hi all,
>
> here are my thoughts on an "exim-archive howto" I propose writing. I
> intend this to be a starting point for people who need help setting this
> up, not the definitive example and reference.
>
> 1) System filters seem to be just as good as any other method for
> copying and archiving messages. They are easy to drop in and out of a
> config without having to modify or re-write existing sections.
I'm neutral, but see below.
>
> 2) Modifying existing routers and transports _could_ lead to problems.
> Adding this functionality _could_ mean re-writing existing sections.
>
I see neither greater nor lesser chance of 'problems' than in
writing any other router/transport set. Adding an 'unseen'
section for archiving is usually going to 'trigger' on the same
criteria (100% archived), optionally a subset, of the same
criteria as the 'parent' router, so making these into congruent
'airs' is straightforward.
Delivery, of course is to a different location, but that also is
adaptable from comparable 'other than archive' routing, whether
traffic is to be retained on-box in a special 'account',
smtp_remote routed to an off-box admin/management reviewer, or
pgp'ed into a tarball somewhere.
> 3) Shadow transports are not designed for "remote" transports which
> would prevent the archive or copies being stored on a remote machine, or
> might force us to keep messages in a mailbox on a public facing server.
>
The delivery characteristics of any transport are whatever they
need to be. Nothing inherently prevent delivering or storing
messages wherever is most appropriate - on or off 'box'.
'Decisions' as to which, when, and whither are up to the routers.
> 4) The howto should begin with some VERY strong warnings about the
> postmasters responsibility to investigate legal side of monitoring or
> archiving users e-mail messages
Agree that! But is has to be in the nature of a 'disclaimer'.
- National, provincial/state, and local/municipal laws vary by
country, can overlap and contradict. Courts stay busy and
lawyers in employment just sorting out what applies, where and
to whom. And they cannot agree from one case to the next.
- Most nations claim at least a measure of extraterritorial
jurisdiction if/as/when one or more correspondents are under
their jurisdiction, or are their 'nationals', and/or traffic has
transited their 'space', even if the server(s) and sysadmins are
not.
There is no way to keep up with all possible exposure, nor is it
an Exim 'technical issue'.
For starters, any form of 'fascist logging' or full-body
archiving is probably best restricted to dedicated 'corporate'
servers, where there is a clear company policy, and it is part
of the conditions of employment.
Archiving the personal traffic of private citizens, or logging
any more detail than that required for prper administration of
the service, is another matter entirely, and not one to be
advised without their consent in advance.
Here again, government regulations may require more than you
would a provider might have wished to retain.
We all like to promote a minimally regulated, free and open
environment, but please let me know if you run across a firm or
sysadmin who took on a sovereign government over this issue -
and won the day with their resources intact.
>
> ---
>
> I propose starting with a simple system filter like:
>
> unseen deliver archive-server@??? (from exim the documentation)
>
> Then adding:
> first_delivery check (suggested by various people)
> error handling - errors_to or noerrors (suggested by Phillip)
> not error_message check (suggested by Dean Brooks)
>
> Then make it filter:
> by a specified domain
> by a specified list of domains (read from a file)
> by existing domainlist (such as +relay_domains)
> by a specified address
> by a specified list of addresses (read from file)
> by existing accounts (such as local mailbox users)
>
> The configs:
> Deans example seems as good as any other I've seen for using as a basis
> for the examples:
>
>> if first_delivery and not error_message and
>> foranyaddress $sender_address,$h_from: ($thisaddress is
>> "usertomonitor@???")
>> then
>> unseen deliver getsacopy@??? errors_to nobody@???
>> finish
>> endif
>
>
> Last question, if people want to strip attachments from archived
> messages (as I do) is this best done at the "archiver" or on the
> "archive" itself (assuming they are different boxes that pass messages
> via smtp)
>
Best kept as a separate issue, IMO.
Plenty of examples around, and not likely to be germane.
Monitoring or archiving is nearly useless if attachments are
stripped, as nothing would be left but headers for a lot of the
traffic out there.
Better to let the 'reviewer' set his reader to only retrieve and
display headers, then bore-down if/as/when an item of interest
is found. Having nothing further when that time comes will
just get you sent off to recover what you no longer have.
> Please review and correct me if you think any of this is just plain
> wrong or could be done better.
Potential adopters should also be advised that full-archiving
can take up *way more* storage space than normal traffic does.
Most users of POP leave very little on the server, deleting as
they download to their MUA. Many IMAP users also 'clean house'
at least now and then. Archives, by definition, are retained.
Any 'selectively monitored' traffic will probably have 'some of
each' such characteristics.
> Code samples would certainly help. I'll
> start work on this on Tuesday night once I've had some feedback in.
>
> Thanks
>
> Jason Meers
>
Can't hurt to start. There will be comment enough from those
interested. Primary need' IMO, is corporate, followed by
government rules - which seem to be on the rise.
JFWIW / side-issue department: If you use Exim as a 'frontend'
to a DBMail storage scheme - which I have done - a simple flag
in the DB can both make all traffic 'available' to a reviewer,
and functionally 'undeletable' by the normal owner. They will
perceive it as deleted, but the flag they are concerned with
simply no longer displays it for them. A different flag decides
if it is permissable for a maintenance run to actually remove it
from the mailstore.
The advantage to that is that no initial duplicate storage is
needed for an archive, though there is still a higher measure of
retained traffic than the typical POP/IMAP users would have left
on-box.
Downside is fairly complex DB, MTA, and POP/IMAP daemon configs,
not to mention the odd-man-out mail storage scheme, which can
bite yeraz.
- Mentioned, tested with Courier and Exim, but not recommended
for 'prime time'. With current HDD sizes, the oddities are not
ordinarily worth the space saved.
JM2CW
Bill
