Re: Deadlock on the mm->mmap_sem

David Howells
2001-09-19T09:51:58Z

Looking through the do_page_fault(), I noticed there's a race in expand stack
because expand_stack() expects the caller to have the mm-sem write-locked.

I've attached a patch that might fix it appropriately. Alternatively, it may
be worth applying Andrea's 00_silent-stack-overflow-10 patch which fixes this
and something else too.

David

diff -uNr linux-2.4.10-pre12/include/linux/mm.h linux-rwsem/include/linux/mm.h
--- linux-2.4.10-pre12/include/linux/mm.h Wed Sep 19 10:39:23 2001
+++ linux-rwsem/include/linux/mm.h Wed Sep 19 10:40:48 2001
@@ -586,11 +586,11 @@
* before relocating the vma range ourself.
*/
address &= PAGE_MASK;
+ spin_lock(&vma->vm_mm->page_table_lock);
grow = (vma->vm_start - address) >> PAGE_SHIFT;
if (vma->vm_end - address > current->rlim[RLIMIT_STACK].rlim_cur ||
((vma->vm_mm->total_vm + grow) << PAGE_SHIFT) > current->rlim[RLIMIT_AS].rlim_cur)
- return -ENOMEM;
- spin_lock(&vma->vm_mm->page_table_lock);
+ goto nomem;
vma->vm_start = address;
vma->vm_pgoff -= grow;
vma->vm_mm->total_vm += grow;
@@ -598,6 +598,9 @@
vma->vm_mm->locked_vm += grow;
spin_unlock(&vma->vm_mm->page_table_lock);
return 0;
+ nomem:
+ spin_unlock(&vma->vm_mm->page_table_lock);
+ return -ENOMEM;
}

/* Look up the first VMA which satisfies addr < vm_end, NULL if none. */