Re: [PATCH] Prevent OOM from killing init

[Helge Hafting <helgehaf@idb.hist.no>]

Alan Cox wrote:
> 
> > >How do you return an out of memory error to a C program that is out of memory
> > >due to a stack growth fault. There is actually not a language construct for it
> > SIGSEGV.
> > Stack overflow for a language like C using standard implementation techniques
> > is the same as a page fault while accessing a page for which there is no backing
> > store. SIGSEGV is the logical choice, and the one I'd expect on other Unices.
> 
> Guess again. You are expanding the stack because you have no room left on it.
> You take a fault. You want to report a SIGSEGV. Now where are you
> going to put the stack frame ?
> 
> SIGSEGV in combination with a preallocated alternate stack maybe, but then you
> still need to recover. C++ you can maybe do it with exception handling but
> C doesnt really have the structure and longjmp just doesnt cut it.

Seems to me a guard page would do the trick.  Make the last page of the
stack
non-overcommitable and marked not present.  Maybe non-swappable too in
case
nothing else can be swapped out for some reason.
(Yes, that wastes a page per process)
Whenever we hit the guard page, try expanding the stack.  
If it works - fine.  If not - make the guard page present _and_ deliver
the SIGSEGV using this last page of stack.  No complicated alternate
stack construct, just report OOM one page in advance.

OOM is still possible if the program don't handle SIGSEGV well.
But a smart program now have the option of doing emergency deallocations
and/or dump its precious intermediate results to file.

Helge Hafting