Re: a.out issue

[Chris Wright <chrisw@osdl.org>]

* Florian Heinz (heinz@cronon-ag.de) wrote:
> seems like find_vma_prepare does not what insert_vm_struct expects when
> the whole addresspace is occupied.

The setup_arg_pages() is inserting an overlapping region.  If nothing
else, this will fix that problem.   Perhaps there's a better solution.

thanks,
-chris

===== fs/exec.c 1.143 vs edited =====
--- 1.143/fs/exec.c	2004-10-28 00:40:03 -07:00
+++ edited/fs/exec.c	2004-11-11 19:24:54 -08:00
@@ -413,6 +413,7 @@
 
 	down_write(&mm->mmap_sem);
 	{
+		struct vm_area_struct *vma;
 		mpnt->vm_mm = mm;
 #ifdef CONFIG_STACK_GROWSUP
 		mpnt->vm_start = stack_base;
@@ -433,6 +434,12 @@
 			mpnt->vm_flags = VM_STACK_FLAGS;
 		mpnt->vm_flags |= mm->def_flags;
 		mpnt->vm_page_prot = protection_map[mpnt->vm_flags & 0x7];
+		vma = find_vma(mm, mpnt->vm_start);
+		if (vma) {
+			up_write(&mm->mmap_sem);
+			kmem_cache_free(vm_area_cachep, mpnt);
+			return -ENOMEM;
+		}
 		insert_vm_struct(mm, mpnt);
 		mm->stack_vm = mm->total_vm = vma_pages(mpnt);
 	}