Re: [exim] Planning MBOX to MAILDIR Migration

On Fri, 2007-03-02 at 10:17 -0800, Jeff Lasman wrote:
> I would have considered ReiserFS if not for Hans's personal problems,
> and Suse's consequent (?) decision to drop support for it in future
> releases. But I now worry about future development of ReiserFS, and I
> don't want to use it on anything new.

Suse dropped ReiserFS v3 as it's no longer being activelt developed -
maintained yes, but development is now in v4 which hasn't been as widely
adopted. As far as I remember the announcement followed the somewhat
unsavoury incident with Hans Reiser's wife's death and his subsequent
arrest, however the decision predated both incidents in the dev tree. I
think the timing of both the formal release of 10.2 and Reiser's arrest
was something of an unfortunate coincidence.

Anyway, that's by the by: consensus amongst Reiser-following lists (and
indeed some threads on opensuse.org) seems to be that with Hans Reiser
out of action, the filesystem's development is all but over and it's
likely to be time to use something else.

Marc: we've used ReiserFS at work for several years now and (in
conjuntion with Dell hardware) have experienced a number of
service-affecting incidents where the filesystem has all but blown up.
On one occasion the filesystem for 1500 or so users was lost completely
(yes, there were backups). We're now deploying a system using Sun NAS
equipment instead.

I'd stick with EXT3, if I were you, and learn some of the tuning tricks
that exist for it. Also:

1. Ensure that your mailstore filesystem is on a different hardware
device to that processing the spool, logs, antispam, antivirus and so
on.
2. Get the fastest disks you can buy.
3. Try not to use RAID5 (it's a compromise between speed, resilience and
disk space overhead, and can cause significant bottlenecks in very heavy
mixed disk IO situations). In a hypothetical 16-disk array, take 7 disks
each and put them in a RAID0+1 or RAID10 config, and keep 2 disks back
as spares just in case if your system lets you do it. Yes, you "lose"
50% of your disk space, but you gain much in the event of a disk
failure. Even better, try to get something like IBM's "EE" RAID systems
which can cope with loss of 2 disks in some situations.
4. Mount your mailstore "noatime".
5. Consider having an external journal but note that this *can* be less
robust than having an in-filesystem journal. Suck this one and see, and
note that having the journal on a separate device makes that device
"hot".
6. Use the "dir_index" filesystem option when creating the filesystem.

Above all, make sure you create the filesystem with an appropriately
large maximum number of inodes. There's not much worse than having X
terabytes of storage, but running out of inodes...

You'll have much more disk IO with maildir in a large system than with
mbox, but this is overcome by the fact that your server doesn't need to
keep writing large files back to the filesystem if someone deletes a
message in the "middle" of their mailbox.

Once you've created your filesystem, establish a baseline for its' IO
using something like Bonnie++. Then you'll *know* where its' performance
threshold is, and be able to tell when you need a new one (as long as
you monitor performance).

YMMV, others may disagree, etc etc :)

Graeme