Re: [exim] Problems mitigating joe job

Ryan Thompson wrote:
> Hi all,
>
> One of my email domains has recently been the (repeat) victim of a fairly
> large-scale joe job. I am seeing thousands of back-scatter bounces for
> addresses like fox1@???, fox2@???, etc. However, when this
> attacker sends out one of their batches, it is enough to run my lightly
> loaded 1GB server out of swap within 3-4 minutes. (At which point I need
> remote hands to do a hard boot, because ssh, login, etc. have been killed by
> the kernel).
>
> So, there are three problems:
>
> 1. Root problem -- the joe job -- Not much to be done about this.
>
> 2. Exim accepting bounces for nonexistent addresses--at the very least would
> like to drop or auto-respond to anything for fox*@???
>
> 3. Exim memory performance -- I have set the following in exim.conf to
> attempt to throttle the queue processing:
>
> queue_run_max = 5
> remote_max_parallel = 1
> queue_smtp_domains = 1
>
> Unfortunately, these do not seem to have had an effect.
>
> As a stop-gap, I made a cron job that runs once a minute and stops exim if
> the load average goes above 15, and then restarts it after the load drops.
> It's not pretty, but it keeps the server alive.
>
> What is the best way to handle this? General or specific answers gratefully
> accepted!
>
> Ryan

Start Exim (and others) 'niced' to a *much* lower priority than your
sshd and other critical system critters. NB - the numbers are sort of
'backwards'. man nice.

That will have zero effect on normal days, yet insure that you can ssh
in and get control to fix a problem.

Worst-case here was a little over 850 million arrivals in one day -
nearly all rejected. Slowed-down normal delivery, as it outran the
PostgreSQL connection count limit. Bigtime.

Otherwise, didn't come anywhere close to crashing anything - one runs
out of frontside bandwidth first (a mere 100 Mb). Nor did it need a
reboot - just a few ipfw rules added to smite the heathens.

At the time, OS was FreeBSD 6-pre on Intel Core-D 3 Ghz dual-core, 2 GB
RAM. Similarly robust with OpenBSD - even on VIA C3 and C7.

HTH,

Bill