Re: [exim] Bulk Outbound Performance

On 2012-09-02 at 10:40 +0100, Ron White wrote:
> Can anyone on the list tell me if it's possible to performance tune Exim
> to a point where it could complete with this and possible strategies?

You don't mention the number of machines used.

It would require C language level changes to Exim, or some complex
configuration, to change from the current spool model to something which
manages destination queues and shuffles mail between them.

I could do it as a configuration-only thing, and with documentation it
would be maintainable, but I'd use it as an opportunity to study what
was needed and find ways to move it into the core to be more easily
configured as a stock option and reduce the technical debt of the
configuration maintenance.

Exim's not geared, as is, for large backlogs. With enough grunt, you
can overcome that, but it won't be as capable as a major email-pushing
engine.

Note that Exim's bias is towards stability; the
one-process-per-connection model (both inbound and outbound) means that
one weird remote site can't screw up other deliveries. If you have an
MTA which never does TLS, doesn't worry about inbound connections except
enough to handle callout verifications, and where the volumes are enough
that a few connections dying at once is statistical noise, you can swing
the bias to optimise for outbound throughput instead.

I think it's PowerMTA that I looked at back in January or so, and if
I'm remembering the right product, it's got support for web-bugs and
other tricks to try to get status feedback from HTML-enabled
mail-clients, to try to gather deliverability statistics, and it's all
tied into some nice dashboards. If a marketing department is looking at
this, you've got a bunch of work to do to compete. The MTA I looked at
was quite nice, at a technical level; it took to heart the C10k lessons
and, from the paper blurb, gave my cynical side pause enough to be
willing to take a serious look. That doesn't mean I ever want a job
_running_ a closed source MTA. *shudder* Being a postmaster has enough
problems already, without having a product you can't investigate and fix
yourself, when you're talking to so many different products at remote
sites. MTAs are one product that _seriously_ benefit from having the
source. But that's my bias. And hey, I ended up becoming an MTA
maintainer as a result.

Most open source folks don't send high volumes of identical mails, so
there hasn't been the push to develop an open competitor, so the
closed-source side has more options here.

-Phil