Autor: Mark Morley Data: A: exim-users Assumpte: [Exim] Possible DOS for Exim
A while back we noticed Exim processes that were going nuts, consuming
all available RAM and swap space and eventually dying.
We narrowed it down to a filter issue, and we've been able to reliably
reproduce it now. This could be used as a form of DOS attack.
If you inject a message into the system that contains a HUGE number of
headers, the filter processing routines seem to go into some sort of
endless loop, continuously allocating more and more RAM.
We've seen this happen on a number of occasions now with spam containing
as many as 1,500 "Apparently-to:" headers.
The actual content of the filter doesn't seem to matter that much. We
have some 300+ filter rules for spam scoring, but turning off large
sections of them didn't change Exim's behavior. Turning off the filter
completely did, but that's not an acceptable solution for us. We couldn't
narrow it down to any particular filter rule.
The process dealing with the message rapidly consumes all available RAM
and the server begins to swap. This drives the load up considerably.
Eventually malloc() fails (you'll see malloc errors in the panic log)
and the process dies.
Problem is, the message is left on the queue. When a queue runner reaches
the message, the same thing happens, and it dies. If you don't catch it,
you'll eventully have a lot of messages in the queue that never get
processed because the problem message is in the way.
Also, as the load climbs, Exim stops processing messages immediately
and just queues them. If it goes high enough, it stops accepting mail
at all (depending on your settings of course).
The first time this happened we wound up with several thousand messages
in the queue (there's normally less than 100) in less than an hour and
the server load was hovering around 15 (FreeBSD) making it almost
impossible to log in and examine anything.
In the end we doubled the RAM, which merely buys you a little time.
We also have to monitor the panic log and have the system page us if
it sees any malloc errors in there.
I discussed it with Philip and he's going to put some sort of limit on
headers in the next version.
But the question is, does anyone have any ideas about how to prevent this
kind of thing now, with the current version of Exim?
Would an appropriate call to setrlimit() in the right place help? If
not to solve the problem, at least to limit the process size to avoid
swapping? Even limiting a process to 256 megs would work for us.