[ On Friday, July 1, 2005 at 17:28:28 (+0200), V. T. Mueller wrote: ]
> Subject: Re: [exim] High Perf server
>
> I don't really see the point in smashing it with
> replies like "go and buy expensive hardware"
Actually my point was that the hardware I suggested is _NOT_ expensive
(in fact it's the least expensive in the business right now), and yet it
provides both the low-latency, high-bandwidth, throughput necessary to
meet the traffic needs of the site, as well as providing decent
reliaiblity (especially when also backed by a real UPS :-).
And, ultimately, disk I/O can only be deferred for so long (i.e. silly
tricks like pre-allocation in the application, or buffering within the
host OS are not scalable), but a permanent and scalable improvement can
easily be achieved by offloading disk I/O, e.g. by using decent and
inexpensive approaches to parallelization such as those embodied by a
RAID controller which supports reliable "write-back" caching.
Such simple and common hardware has been cheaper than programming labour
for a good many years now, and it just keeps getting more-so since of
course manufacturing work well for hardware, but not at all for software.
> "High Perf" maybe an issue for a minority of exim users, but why in
> God's name shouldn't there be an open discussion about it?
One must be sure one is discussing such performance issues in the right
forum and with an eye to current best practices and existing technology
regarding high capacity and fast throughput systems.
Ideed, what I've said must be taken in context too.
One should of course use decent high-performance OS software where it is
sufficient for the task. OS software improvements are much more cost
effective than making custom hacks to applications since they apply to a
multitude of applications and platforms. No decently portable
application should ever have to work around the OS, or the hardware, in
order to achieve obvious performance improvements which can be easily
achieved at the more common OS or hardware layers and without having to
use less portable APIs.
I'm getting the strong impression that the O.P. is not currently using a
filesystem that can safely defer and serialize meta-data operations,
e.g. as the BSD UFS soft-updates option does. Preallocating files on a
filesystem with soft-updates enabled is a total duplication of effort
and could probably even make things slower.
On the other hand the use of UFS soft-updates in conjunction with a
storage sub-system that has lots of reliable write-back caching of its
own is also a total waste and duplication of effort; and at the same
time also invites more risk of loss than is necessary in face of
catastrophic failures such as OS crashes, UPS failure, etc. With
sufficient reliable write-back cache the metadata writes are already
very low-latency and nearly invisible.
Use the right tool and you won't end up with black-and-blue fingers, or
far worse! :-)
--
Greg A. Woods
H:+1 416 218-0098 W:+1 416 489-5852 x122 VE3TCP RoboHack <woods@???>
Planix, Inc. <woods@???> Secrets of the Weird <woods@???>
