A couple of OOM killer races

A couple of OOM killer races

[Richard Hayden]

Hi all,

It appears there is no protection in badness() (called by 
out_of_memory() for each process) when it reads p->mm->total_vm. Another 
processor (or a kernel preemption) could presumably run do_exit and then 
exit_mm, freeing the process in question's reference to its mm just 
after the (!p->mm) check but before it reads p->mm->total_vm, making the 
latter reference a null pointer reference.

Also there appears to be no protection when we set p->time_slice in 
__oom_kill_task(). Am I right in thinking that this field should be 
protected by the appropriate runqueue lock, at least this is what 
scheduler_tick() seems to use?

Have I missed anything?

Best regards,

Richard Hayden.

		
___________________________________________________________ 
How much free photo storage do you get? Store your holiday 
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com

Re: A couple of OOM killer races

[Zhou Yingchao]

On 9/2/05, Richard Hayden <rahaydenuk@yahoo.co.uk> wrote:
> Hi all,
> 
> It appears there is no protection in badness() (called by
> out_of_memory() for each process) when it reads p->mm->total_vm. Another
> processor (or a kernel preemption) could presumably run do_exit and then
> exit_mm, freeing the process in question's reference to its mm just
> after the (!p->mm) check but before it reads p->mm->total_vm, making the
> latter reference a null pointer reference.

 In badness, the tasklist_lock has been hold. And when an exit signal
delived to it because other thread call do_group_exit, it need to hold
the tasklist_lock first, so we are protected.
  
-- 
Yingchao Zhou

Re: A couple of OOM killer races

[Coywolf Qi Hunt]

On 9/2/05, Richard Hayden <rahaydenuk@yahoo.co.uk> wrote:
> Hi all,
> 
> It appears there is no protection in badness() (called by
> out_of_memory() for each process) when it reads p->mm->total_vm. Another
> processor (or a kernel preemption) could presumably run do_exit and then
> exit_mm, freeing the process in question's reference to its mm just
> after the (!p->mm) check but before it reads p->mm->total_vm, making the
> latter reference a null pointer reference.

We have read_lock(&tasklist_lock); .

> 
> Also there appears to be no protection when we set p->time_slice in
> __oom_kill_task(). Am I right in thinking that this field should be
> protected by the appropriate runqueue lock, at least this is what
> scheduler_tick() seems to use?

ditto

-- 
Coywolf Qi Hunt
http://sosdg.org/~coywolf/