Re: [PATCH 3/3] mlock: avoid dirtying pages and triggering writeback

Ted Ts'o
2010-11-19T23:23:18Z

On Fri, Nov 19, 2010 at 02:54:42PM -0800, Andrew Morton wrote:
>
> Dirtying all that memory at mlock() time is pretty obnoxious.
> ...
> So all that leaves me thinking that we merge your patches as-is. Then
> work out why users can fairly trivially use mlock to hang the kernel on
> ext2 and ext3 (and others?)

So at least on RHEL 4 and 5 systems, pam_limits was configured so that
unprivileged processes could only mlock() at most 16k. This was
deemed enough so that programs could protect crypto keys. The
thinking when we added the mlock() ulimit setting was that
unprivileged users could very easily make a nuisance of themselves,
and grab way too much system resources, by using mlock() in obnoxious
ways.

I was just checking to see if my memory was correct, and to my
surprise, I've just found that Ubuntu deliberately sets the memlock
ulimit to be unlimited. Which means that Ubuntu systems are
completely wide open for this particular DOS attack. So if you
administer an Ubuntu-based server, it might be a good idea to make a
tiny little change to /etc/security/limits.conf....

- Ted