On Wed, 2008-06-18 at 13:04 +0200, Gergely Nagy wrote:
> First of all, apologies in advance for the lack of information regarding
> the problem - it's pretty darn hard to test the problem as it occurs on
> a mission critical system which I can't freely play around with.
...which might make debugging tricky, if not impossible. But we'll give
it a go anyway :)
> The bulk of the issue is, that I have a configuration which Works(tm),
> it's fast, reliable and works like a charm. However, I would like to use
> maildirsize files, but whenever I turn maildir_use_size_file on in the
> appropriate transport, the load goes from the usual 10-20 to 600 and
> above within half a minute. I believe it would rise even further, but so
> far, I always turned the option off again before that.
The *usual* 10-20? Wow. That smacks of wedged disks already to me. How
are your disks laid out?
> Without much further ado, the relevant transport in my exim4.conf looks
> like this:
>
> local_delivery:
> driver = appendfile
> directory = /m/${l_1:$local_part}/${local_part}@${domain}
> quota_directory = /m/${l_1:$local_part}/${local_part}@${domain}
These might be useful shortly, combined with the answer you give to my
earlier question.
> The hardware in question is a dual quad-core Intel Xeon, with 4G ram and
> a couple of SAS disks appropriately set up to handle the load. It's
> running on Debian etch, with Exim 4.63 (+ whatever patches Debian applied).
"Appropriately set up" - how? Mirrored in hardware, software? Striped?
> The usual load is between 10 and 30 (during the busiest hours), handling
> 15-25 mails / second with maildir_use_size_file turned off.
Goodness me. That load already smells fairly bad to me - it indicates
that some processes are either on the CPU permanently or waiting for
disk IO.
>From your tests, and the info provided above, I'd suggest that two disks
simply is not enough for this platform. I'd hazard a guess that the
heads are seeking around at a massive rate which causes both reads and
writes to be delayed, to the point where (at say 10 deliveries/sec) you
end up with 600 processes waiting for the disks after 60 seconds, thus
pushing the load through the roof.
There are some small things you can do:
1. Mount /m with the "noatime" option - this can be done in normal
operation with only a tiny interruption:
mount -o remount,noatime,nodiratime /m
2. Ensure your filesystems are using the "dir_index" option (only
applicable to EXT3). If you need to change it, see the tune2fs man page
and be prepared for an outage to do the conversion. You can see if it's
switched on by doing:
tune2fs -l /dev/sda1 (or whatever the partition is)
Look for the features line, and see if it looks something like this:
Filesystem features: has_journal ext_attr resize_inode dir_index
filetype needs_recovery sparse_super large_file
(that's off a recently built F9 box).
3. Add more disks to your array, if you can. How that happens depends on
your hardware.
It almost certainly looks to me like your disks just aren't quick enough
to keep up.
Graeme
