RE: [exim] Performance considerations

Top Page
Delete this message
Reply to this message
Author: Peter Savitch
Date:  
To: Maykel Moya
Subject: RE: [exim] Performance considerations
Hi,

I've been managed an Exim installation, 30k users, single machine (Dual
PIII @700, 1G), in 2002-2003. Each Exim instance (SSL, LDAP-enabled) ate
about 1.5-2Mb of RSS (physical) and 5-6Mb of VSZ (virtual). That is,
about 256 concurrent Exim instances (128 listeners, 64 remote queue
runners, 64 auxiliary processes - something like this).

The main advantage of Exim is it's supreme flexibility.

The main drawbacks of Exim:
- poor ability to handle large mail queues, not the memory;
- inability to share the queue between multiple boxes (questionable
scalability);

My queue sizes were the following:
200 .. 500 Green (OK)
500 .. 1000 Yellow (OK)
1000 .. 5000 Red (Error)
5000 .. 10000 Red (Critical)

I utilized a custom kernel + custom Exim lookup that allowed me to
control the number of simultaneous processes and memory usage for the
single (exim) user. When there were enough resources, every message
delivered immediately. When we were low on memory, then queue everything
(otherwise, system would be unstable).

If you queue everything unconditionally, the time-on-queue could reach a
couple of hours with the large queues. Your customer support services
should install heat sinks on their phones...

The scenario: an (authenticated) ISP user gateways everything via your
server, but makes an open relay with his misconfigured firewall (don't
laugh, it's true). Then you've got thousands of spam messages in the
queue, with approximately the same amount of bounces (also in the
queue). If you don't control queuing, you'll hit the memory limit and
PANIC|DIE soon. As you don't like to be unstable, then queue everything.
And you've got it: the queue grows to about 10k messages.

The thing: Exim does not have any intrinsic queue priority control, it's
queue is optimized per destination-host. All that you can do is to make
additional routers that accomplish this: cron additional queue runners
for local (known-to-be-fast-to-deliver) mail; reduce the queue TTL for
bulk mail (you should first decide what mail is considered bulk).
Remember that everything in the mail headers could be forged; you don't
want the users to boost their mail priority, don't you? :-)

Well, that is.

-----Original Message-----
From: exim-users-bounces@??? [mailto:exim-users-bounces@exim.org]
On Behalf Of Maykel Moya
Sent: Tuesday, September 07, 2004 4:58 AM
To: Eximers
Subject: [exim] Performance considerations