Re: Very large font size crashing X Font Server and Grounding Server to a Halt (was: remote DoS in Mozilla 1.0)

[rjh@world.std.com]

On 13 Jun, Federico Sevilla III wrote:
> Suggestions on how to work around this on multiple levels would definitely
> be appreciated. I'll be starting by removing the X font server from our
> file and authentication server onto some high-powered workstation, but I'm
> sure this won't be enough, and knowing that a user process like xfs-daemon
> can drag the Linux kernel down to knees is not very comforting. :(
> 

The protection that you need is provided by "ulimit" on most Unixes.
There are facilities to limit maximum real memory used, maximum virtual
memory, maximum number of processes, etc.  This specific bug in XFree is
one of a general case of inescapable user process bugs.  It resulted in
an almost infinite size malloc() request.  You can acheive the same
effect in any userspace program by just putting malloc() inside an
infinite loop.

If you allow users to run with unlimited memory permission, you are
vulnerable.  The XFree bug will hit more people than usual because it is
common to put the ulimit on regular user logins and forget to place a
limit on the automatically started processes.  The default configuration
from RedHat, SuSE, and others is to start XFree outside the login
system.  You can also place limits on these processes but you need to
examine the startup scripts to install the limits in the right places.

This would then result in a different DoS.  Whenever XFree hits the
memory limit, the malloc's will fail, and XFree will decide what to do
about it.  Depending on the circumstances, XFree may shut down, thus
killing all the X window dependent processes.

R Horn