Re: Bug in VM accounting code, probably exploitable

[Stas Sergeev <stsp@aknet.ru>]

Hi Marcelo.

Marcelo Tosatti wrote:
>>As you can see, the program caused many
>>other processes to be killed, before it
>>died itself.
> About v2.4, can you try v2.4.26 with CONFIG_OOM_KILLER=y ? 
OK, with CONFIG_OOM_KILLER=y (2.4.26) I get this:
---
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1514 
(overc_test).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1320 
(mozilla-bin).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1406 
(mozilla-bin).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1407 
(mozilla-bin).
May 22 16:14:56 lin kernel: Out of Memory: Killed process 1409 
(mozilla-bin).
---
Better than without OOM killer, although
still not perfect - for some reasons it also
decided to kill mozilla *after* killing the
test program. Strange.

> As for the overcommit, I think it has always been "broken"? (its always 
> possible to overcommit).
At least I received a couple of messages that
stated that my test program doesn't OOM the
RedHat Fedora 2.4 kernels - mprotect() fails
with ENOMEM. That resembles the correct
behaveour of the fixed 2.6 so I think at least
RedHat got rid of that problem somehow.
Hope that helps.