Re: [exim] Archiving Email

* on the Fri, Feb 23, 2007 at 05:32:01PM +0100, Sander Smeenk wrote:

>> With flat files, your friend would be grep. With maildir you would need
>> a mail client, which might be ok, unless we are talking hundreds of
>> thousands of messages.
> Currently i store 'old messages' in maildir format and tar those.
> Thing is, to get a specific message i have to know the exact maildir
> filename or i have to untar the complete archive and go from there...
> Which isn't all that bad if you are talking about your private mailsetup
> with a couple of mailinglists, but company wide this can be a real pain.
>
> With mbox format you can usually just zless or zgrep through the gzipped
> mbox. Even cooler, mutt is able to read gzipped mboxes!. Mbox can easily
> be searched through with numerous tools from the cli too...
>
> So i'd say... Go for mbox format :-)

Mail on my system gets delivered into Maildir's in the following
format: (given the email address username@???)

/var/spool/mail/imap/gr/ep/ul/grepular.com/us/er/na/username/Maildir/

I recently added an unseen router that transparently delivers a second
copy of all incoming mail into:

/var/spool/mail/imap_archive/gr/ep/ul/grepular.com/us/er/na/username/Maildir/

Exim can deliver directly into subfolders depending on a set of rules,
and that structure is preserved into the archive.

I have a courier-imap setup for checking my mail. If I set the username
as "username@???" it accesses it from the first path. If I set
the username to "username@???" it accesses the second
path. ACL's are used to set the archive structure to be read only. The
courier imap ACL file is built when exim delivers into an archive folder
for the first time with:

${run{/usr/bin/maildiracl -set MAILDIR_ARCHIVE_PATH INBOX owner alrsw
&>/dev/null}}

I'm aware of the problem of giving the user "Administer" access, but
courier wont let the owner live without that access for some reason. I
might look into restricting it to read only using the FS as well at some
point.

This means, if someone needs to access mail they've deleted, they can
just look in their archive IMAP store. It also means they don't need to
keep old mail in their main imap store just in case they need it. This
keeping primary maildir's small, with all the efficiencies that brings.
It also means, a sysadmin wont need to be bothered with retrieving mail
as often, as in most cases the user can do it themselves.

If disk becomes an issue, I can just change the archive policy to be "x
years only", and run a simple command to purge old email, from a cron
job.

While I'm going off on one, I may as well mention that you could store
primary mail and archive mail on different devices/filesystems, and have
differing backup policies for the two. And you can also set low disk
quotas on the primary to force people into using the archive as it's
intended. If there's a catastrophic failure and you need to restore from
backups you can get the primary store online quickly, and then worry
about restoring the archive structure afterwards.

Mike