Re: User space out of memory approach

Thomas Gleixner
2005-01-25T21:41:13Z

On Tue, 2005-01-25 at 17:13 -0400, Mauricio Lin wrote:
> Hi Andrea,
>
> Your OOM Killer patch was tested and a strange behaviour was found.
> Basically as normal user we started some applications as openoffice,
> mozilla and emacs.
> And as a root (in another tty) we started a simple program that uses
> malloc in a forever loop as below:
>
> int main (void)
> {
> int * mem;
> for (;;)
> mem = (int *) malloc(sizeof(int));
> return 0;
> }
>
>
> Using the original OOM Killer, malloc is the first killed application
> and the sytem is restored in a useful state. After applying your patch
> and accomplish the same experiment, the OOM Killer it does not kill
> malloc program and it enters in a kind of forever loop as below:
>
> 1) out_of_memory is invoked;
> 2) select_bad_process is invoked;

Which process is selected ?

> 3) the following condition is fullfied;
> if ((unlikely(test_tsk_thread_flag(p, TIF_MEMDIE)) || (p->flags &
> PF_EXITING)) &&
> !(p->flags & PF_DEAD))
> return ERR_PTR(-1UL);

???

Can you please show the kernel messages ?

tglx