Re: [11/11] x86_64: TASK_SIZE fixes for compatibility mode processes

["Siddha, Suresh B" <suresh.b.siddha@intel.com>]

On Fri, Jul 15, 2005 at 10:53:33AM -0500, Justin M. Forbes wrote:
> That said, I will be testing this patch a bit further

Thanks. Let me know if you see any issues.

> myself, and because it does address a real memory leak issue, we should
> consider it or another fix for stable 2.6.12.4.

Appended patch will just fix the memory leak issue. Atleast, we should
apply this.

thanks,
suresh
--

malicious 32bit app can have an elf section at 0xffffe000.  During
exec of this app, we will have a memory leak as insert_vm_struct() is
not checking for return value in syscall32_setup_pages() and thus not
freeing the vma allocated for the vsyscall page.

Check the return value and free the vma incase of failure.

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>


--- linux-2.6.12.2/arch/x86_64/ia32/syscall32.c.orig	2005-06-29 16:00:53.000000000 -0700
+++ linux-2.6.12.2/arch/x86_64/ia32/syscall32.c	2005-07-15 18:09:06.684409144 -0700
@@ -57,6 +57,7 @@
 	int npages = (VSYSCALL32_END - VSYSCALL32_BASE) >> PAGE_SHIFT;
 	struct vm_area_struct *vma;
 	struct mm_struct *mm = current->mm;
+	int ret;
 
 	vma = kmem_cache_alloc(vm_area_cachep, SLAB_KERNEL);
 	if (!vma)
@@ -78,7 +79,11 @@
 	vma->vm_mm = mm;
 
 	down_write(&mm->mmap_sem);
-	insert_vm_struct(mm, vma);
+	if ((ret = insert_vm_struct(mm, vma))) {
+		up_write(&mm->mmap_sem);
+		kmem_cache_free(vm_area_cachep, vma);
+		return ret;
+	}
 	mm->total_vm += npages;
 	up_write(&mm->mmap_sem);
 	return 0;