[PATCH] mmap PROT_NONE fix (was Re: serious performance regression

Daniel McNeil
2004-07-14T18:36:41Z

This patch fixes mmap PROT_NONE from elf binaries that do not
have the PT_GNU_STACK so that the do not have execute permission.

Before this fix, calling access() with a PROT_NONE page
was returning ENOENT instead of EFAULT.

BEFORE:
$ x.gcc322
p2=0x40013000
pid\x1982

access 0 ptr (nil) return code -1 errno 14
access result:: Bad address
access 1 ptr 0xffffffff return code -1 errno 14
access result:: Bad address
access 2 ptr 0x40013000 return code -1 errno 2
access result:: No such file or directory

AFTER:

$ ./x.gcc322
p2=0x40017000
pid$92

access 0 ptr (nil) return code -1 errno 14
access result:: Bad address
access 1 ptr 0xffffffff return code -1 errno 14
access result:: Bad address
access 2 ptr 0x40017000 return code -1 errno 14
access result:: Bad address

Program available at
http://developer.osdl.org/daniel/mmap.PROT_NONE.bug/

Here is the patch.

Signed-off-by: Daniel McNeil <daniel@osdl.org>

=== mm/mmap.c 1.133 vs edited ==--- 1.133/mm/mmap.c 2004-06-29 07:43:12 -07:00
+++ edited/mm/mmap.c 2004-07-14 09:56:37 -07:00
@@ -792,6 +792,12 @@
vm_flags = calc_vm_prot_bits(prot) | calc_vm_flag_bits(flags) |
mm->def_flags | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC;

+ /*
+ * mm->def_flags might have VM_EXEC set, which PROT_NONE does NOT want.
+ */
+ if (prot = PROT_NONE)
+ vm_flags &= ~VM_EXEC;
+
if (flags & MAP_LOCKED) {
if (!capable(CAP_IPC_LOCK))
return -EPERM;