Re: [exim] Bulk Outbound Performance

On Tue, 2012-09-04 at 00:53 -0400, Phil Pennock wrote:
> On 2012-09-02 at 10:40 +0100, Ron White wrote:
> > Can anyone on the list tell me if it's possible to performance tune Exim
> > to a point where it could complete with this and possible strategies?
>
> You don't mention the number of machines used.
>
> It would require C language level changes to Exim, or some complex
> configuration, to change from the current spool model to something which
> manages destination queues and shuffles mail between them.
>
> I could do it as a configuration-only thing, and with documentation it
> would be maintainable, but I'd use it as an opportunity to study what
> was needed and find ways to move it into the core to be more easily
> configured as a stock option and reduce the technical debt of the
> configuration maintenance.
>
> Exim's not geared, as is, for large backlogs. With enough grunt, you
> can overcome that, but it won't be as capable as a major email-pushing
> engine.
>
> Note that Exim's bias is towards stability; the
> one-process-per-connection model (both inbound and outbound) means that
> one weird remote site can't screw up other deliveries. If you have an
> MTA which never does TLS, doesn't worry about inbound connections except
> enough to handle callout verifications, and where the volumes are enough
> that a few connections dying at once is statistical noise, you can swing
> the bias to optimise for outbound throughput instead.
>
> I think it's PowerMTA that I looked at back in January or so, and if
> I'm remembering the right product, it's got support for web-bugs and
> other tricks to try to get status feedback from HTML-enabled
> mail-clients, to try to gather deliverability statistics, and it's all
> tied into some nice dashboards. If a marketing department is looking at
> this, you've got a bunch of work to do to compete. The MTA I looked at
> was quite nice, at a technical level; it took to heart the C10k lessons
> and, from the paper blurb, gave my cynical side pause enough to be
> willing to take a serious look. That doesn't mean I ever want a job
> _running_ a closed source MTA. *shudder* Being a postmaster has enough
> problems already, without having a product you can't investigate and fix
> yourself, when you're talking to so many different products at remote
> sites. MTAs are one product that _seriously_ benefit from having the
> source. But that's my bias. And hey, I ended up becoming an MTA
> maintainer as a result.
>
> Most open source folks don't send high volumes of identical mails, so
> there hasn't been the push to develop an open competitor, so the
> closed-source side has more options here.
>
> -Phil
>
Thanks Phil,

I agree that Exim is not specifically designed to do the same job as
something like PowerMTA - and I'm very happy with my Exim(s) doing what
they do. Curiosity and general discussion with a large newsletter mailer
brought the question up for me, and I just thought I would ask it on a
trusted list.

It seems by doing this I've caused some kind of offence to some - which
totally perplexes me!

I don't think there is a huge OS need for an MTA that can maximize
delivery. Qmail appears to be configurable to offer similar performance
to PowerMTA, but it's not something I have a need to do. I just find the
subject interesting.

Thanks again for the great response. It's really appreciated.