Re: OOM behavior in constrained memory situations

[David Gibson <david@gibson.dropbear.id.au>]

On Mon, Feb 06, 2006 at 04:03:53PM -0800, Christoph Lameter wrote:
> On Mon, 6 Feb 2006, Andrew Morton wrote:
> 
> > The oom-killer is invoked from the page allocator.  A hugetlb pagefault
> > won't use the page allocator.  So there shouldn't be an oom-killing on
> > hugepage exhaustion.
> 
> Right..... and the arch specific fault code (at least ia64) does not call 
> the OOM killer.
> 
> > I think this comment is just wrong:
> > 
> > 		/* Logically this is OOM, not a SIGBUS, but an OOM
> > 		 * could cause the kernel to go killing other
> > 		 * processes which won't help the hugepage situation
> > 		 * at all (?) */
> > 
> > A VM_FAULT_OOM from there won't cause the oom-killer to do anything.  We
> > should return VM_FAULT_OOM and let do_page_fault() commit suicide with
> > SIGKILL.
> 
> Drop my patch that adds the comments explaining the bus error and add this 
> fix instead. This will terminate an application with out of memory instead 
> of bus error and remove the comment that you mentioned.

Looks good, except I think a comment should go in there so the next
person to look at it doesn't make the same wrong assumption I did,
that returning VM_FAULT_OOM will trigger the OOM killer.

> Signed-off-by: Christoph Lameter <clameter@sgi.com>
> 
> Index: linux-2.6.16-rc2/mm/hugetlb.c
> ===================================================================
> --- linux-2.6.16-rc2.orig/mm/hugetlb.c	2006-02-02 22:03:08.000000000 -0800
> +++ linux-2.6.16-rc2/mm/hugetlb.c	2006-02-06 16:02:53.000000000 -0800
> @@ -391,12 +391,7 @@ static int hugetlb_cow(struct mm_struct 
>  
>  	if (!new_page) {
>  		page_cache_release(old_page);
> -
> -		/* Logically this is OOM, not a SIGBUS, but an OOM
> -		 * could cause the kernel to go killing other
> -		 * processes which won't help the hugepage situation
> -		 * at all (?) */
> -		return VM_FAULT_SIGBUS;
> +		return VM_FAULT_OOM;
>  	}
>  
>  	spin_unlock(&mm->page_table_lock);
> @@ -444,6 +439,7 @@ retry:
>  		page = alloc_huge_page(vma, address);
>  		if (!page) {
>  			hugetlb_put_quota(mapping);
> +			ret = VM_FAULT_OOM;
>  			goto out;
>  		}
>  
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson