Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2 (Was: in -mmX)

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2 (Was: in -mmX)

[Jean Delvare]

Quoting myself:

> I think I found a bug in 2.6.9-rc2-mm4. It doesn't seem to be able to
> mmap() files located on cdroms. Same problem with -mm3 and -mm1.
> 2.6.9-rc2 works fine. I reproduced it on two completely different
> systems, so I guess it isn't device dependant.

Looks like I should have done more testing before reporting. The problem
is not only in -mmX, it shows in the -bk series as well. The mmap()
problem I am experiencing seems to have been introduced between
2.6.9-rc2-bk1 and 2.6.9-rc2-bk2. This somewhat narrows the research
field.

I still don't know how to investigate the problem any further.
Suggestions welcome.

Thanks.

-- 
Jean Delvare
http://khali.linux-fr.org/

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2 (Was: in -mmX)

[Andrew Morton]

Jean Delvare <khali@linux-fr.org> wrote:
>
> Quoting myself:
> 
> > I think I found a bug in 2.6.9-rc2-mm4. It doesn't seem to be able to
> > mmap() files located on cdroms. Same problem with -mm3 and -mm1.
> > 2.6.9-rc2 works fine. I reproduced it on two completely different
> > systems, so I guess it isn't device dependant.
> 
> Looks like I should have done more testing before reporting. The problem
> is not only in -mmX, it shows in the -bk series as well. The mmap()
> problem I am experiencing seems to have been introduced between
> 2.6.9-rc2-bk1 and 2.6.9-rc2-bk2. This somewhat narrows the research
> field.

It works OK here.  Can you put together a simple test app?

Is it an iso9660 filesystem?

> I still don't know how to investigate the problem any further.
> Suggestions welcome.

Try the same thing on a read-only mounted ext3 filesystem, maybe?

Capture the strace output.

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Jean Delvare]

Hi Andrew,

> > > I think I found a bug in 2.6.9-rc2-mm4. It doesn't seem to be able
> > > to mmap() files located on cdroms. Same problem with -mm3 and
> > > -mm1. 2.6.9-rc2 works fine. I reproduced it on two completely
> > > different systems, so I guess it isn't device dependant.
> > 
> > Looks like I should have done more testing before reporting. The
> > problem is not only in -mmX, it shows in the -bk series as well. The
> > mmap() problem I am experiencing seems to have been introduced
> > between 2.6.9-rc2-bk1 and 2.6.9-rc2-bk2. This somewhat narrows the
> > research field.
> 
> It works OK here.  Can you put together a simple test app?

Quick-coded test source below.

> Is it an iso9660 filesystem?

Yes it is. I tested on two different CDs, one with Joliet ext, the other
with Rockridge ext. Both failed.

> Try the same thing on a read-only mounted ext3 filesystem, maybe?

Just tested, and the problem doesn't show. It gave me the idea to test
on other filesystems too, and results are interesting:
iso9660 ro: FAILED
ext3 rw: OK
ext3 ro: OK
ntfs ro: FAILED
vfat ro: FAILED
vfat rw: FAILED

> Capture the strace output.

Here it is (using 2.6.9-rc2-mm4):

execve("./test-mmap", ["./test-mmap", "/mnt/cdrom/COPYING"], [/* 35 vars */]) = 0
brk(0)                                  = 0x804a000
open("/etc/ld.so.preload", O_RDONLY)    = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=28201, ...}) = 0
old_mmap(NULL, 28201, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fe4000
close(3)                                = 0
open("/lib/libc.so.6", O_RDONLY)        = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300]\1"..., 1024) = 1024
fstat64(3, {st_mode=S_IFREG|0755, st_size=1458907, ...}) = 0
old_mmap(NULL, 1268836, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xb7eae000
mprotect(0xb7fdd000, 27748, PROT_NONE)  = 0
old_mmap(0xb7fdd000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x12f000) = 0xb7fdd000
old_mmap(0xb7fe2000, 7268, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fe2000
close(3)                                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ead000
munmap(0xb7fe4000, 28201)               = 0
open("/mnt/cdrom/COPYING", O_RDONLY)    = 3
old_mmap(NULL, 42, PROT_READ, MAP_SHARED, 3, 0) = -1 EPERM (Operation not permitted)
fstat64(1, {st_mode=S_IFCHR|0700, st_rdev=makedev(136, 0), ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fea000
write(1, "mmap size=42 flags=1: FAILED\n", 29) = 29
old_mmap(NULL, 42, PROT_READ, MAP_PRIVATE, 3, 0) = -1 EPERM (Operation not permitted)
write(1, "mmap size=42 flags=2: FAILED\n", 29) = 29
old_mmap(NULL, 4096, PROT_READ, MAP_SHARED, 3, 0) = -1 EPERM (Operation not permitted)
write(1, "mmap size=4096 flags=1: FAILED\n", 31) = 31
old_mmap(NULL, 4096, PROT_READ, MAP_PRIVATE, 3, 0) = -1 EPERM (Operation not permitted)
write(1, "mmap size=4096 flags=2: FAILED\n", 31) = 31
close(3)                                = 0
munmap(0xb7fea000, 4096)                = 0
exit_group(0)                           = ?

For comparison, same command using 2.6.9-rc2:

execve("./test-mmap", ["./test-mmap", "/mnt/cdrom/COPYING"], [/* 35 vars */]) = 0
brk(0)                                  = 0x804a000
open("/etc/ld.so.preload", O_RDONLY)    = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=28201, ...}) = 0
old_mmap(NULL, 28201, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fe4000
close(3)                                = 0
open("/lib/libc.so.6", O_RDONLY)        = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300]\1"..., 1024) = 1024
fstat64(3, {st_mode=S_IFREG|0755, st_size=1458907, ...}) = 0
old_mmap(NULL, 1268836, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xb7eae000
mprotect(0xb7fdd000, 27748, PROT_NONE)  = 0
old_mmap(0xb7fdd000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x12f000) = 0xb7fdd000
old_mmap(0xb7fe2000, 7268, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7fe2000
close(3)                                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ead000
munmap(0xb7fe4000, 28201)               = 0
open("/mnt/cdrom/COPYING", O_RDONLY)    = 3
old_mmap(NULL, 42, PROT_READ, MAP_SHARED, 3, 0) = 0xb7fea000
fstat64(1, {st_mode=S_IFCHR|0700, st_rdev=makedev(136, 0), ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fe9000
write(1, "mmap size=42 flags=1: OK\n", 25) = 25
munmap(0xb7fea000, 42)                  = 0
old_mmap(NULL, 42, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fea000
write(1, "mmap size=42 flags=2: OK\n", 25) = 25
munmap(0xb7fea000, 42)                  = 0
old_mmap(NULL, 4096, PROT_READ, MAP_SHARED, 3, 0) = 0xb7fea000
write(1, "mmap size=4096 flags=1: OK\n", 27) = 27
munmap(0xb7fea000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fea000
write(1, "mmap size=4096 flags=2: OK\n", 27) = 27
munmap(0xb7fea000, 4096)                = 0
close(3)                                = 0
munmap(0xb7fe9000, 4096)                = 0
exit_group(0)                           = ?

Nothing really helpful I guess, old_mmap() works there while failing
with EPERM in -mm4. No other difference as far as I can see.

Test code:

#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>

static void test_mmap(int fd, size_t size, int flags)
{
	void *mm;

	mm = mmap(0, size, PROT_READ, flags, fd, 0);
	printf("mmap size=%d flags=%d: %s\n", size, flags,
		mm == MAP_FAILED ? "FAILED" : "OK");
	if (mm != MAP_FAILED)
		munmap(mm, size);
}

int main(int argc, char **argv)
{
	int fd;

	if (argc < 2) {
		printf("Usage: %s file\n", argv[0]);
		return 1;
	}

	fd = open(argv[1], O_RDONLY);
	if (fd == -1) {
		printf("Error: open failed\n");
		return 1;
	}

	test_mmap(fd, 42, MAP_SHARED);
	test_mmap(fd, 42, MAP_PRIVATE);
	test_mmap(fd, 4096, MAP_SHARED);
	test_mmap(fd, 4096, MAP_PRIVATE);

	close(fd);

	return 0;
}

What else can I try/test now?

Thanks.

-- 
Jean Delvare
http://khali.linux-fr.org/

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Ingo Molnar]

* Jean Delvare <khali@linux-fr.org> wrote:

> > Capture the strace output.
> 
> Here it is (using 2.6.9-rc2-mm4):

> old_mmap(NULL, 42, PROT_READ, MAP_SHARED, 3, 0) = -1 EPERM (Operation not permitted)

could you try the patch below? mmap() done from !pt_gnu_stack binaries
on noexec mounted filesystems indeed could fail due to the extra
PROT_EXEC bit.

	Ingo

Signed-off-by: Ingo Molnar <mingo@elte.hu>

--- linux/mm/mmap.c.orig
+++ linux/mm/mmap.c
@@ -773,13 +773,6 @@ unsigned long do_mmap_pgoff(struct file 
 	int accountable = 1;
 	unsigned long charged = 0;
 
-	/*
-	 * Does the application expect PROT_READ to imply PROT_EXEC:
-	 */
-	if (unlikely((prot & PROT_READ) &&
-			(current->personality & READ_IMPLIES_EXEC)))
-		prot |= PROT_EXEC;
-
 	if (file) {
 		if (is_file_hugepages(file))
 			accountable = 0;
@@ -791,6 +784,15 @@ unsigned long do_mmap_pgoff(struct file 
 		    (file->f_vfsmnt->mnt_flags & MNT_NOEXEC))
 			return -EPERM;
 	}
+	/*
+	 * Does the application expect PROT_READ to imply PROT_EXEC?
+	 *
+	 * (the exception is when the underlying filesystem is noexec
+	 *  mounted, in which case we dont add PROT_EXEC.)
+	 */
+	if ((prot & PROT_READ) && (current->personality & READ_IMPLIES_EXEC))
+		if (!(file && (file->f_vfsmnt->mnt_flags & MNT_NOEXEC)))
+			prot |= PROT_EXEC;
 
 	if (!len)
 		return addr;

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Jean Delvare]

Hi Ingo,

> > old_mmap(NULL, 42, PROT_READ, MAP_SHARED, 3, 0) = -1 EPERM
> > (Operation not permitted)
> 
> could you try the patch below? mmap() done from !pt_gnu_stack binaries
> on noexec mounted filesystems indeed could fail due to the extra
> PROT_EXEC bit.

Indeed, all files on which mmap was failing were located on noexec'd
devices (although for the cdrom the noexec is not explicitely stated in
my /etc/fstab file). Your patch fixes my problem, mmap on these devices
is working again. Thanks!

Now I'm only curious as to why the problem only affected me. Since it
looks like noexec is implied on cdrom devices, the problem should have
affected everyone. Or are the "!pt_gnu_stack binaries" something rare? I
admit I have no idea what it refers to.

Thanks again.

-- 
Jean Delvare
http://khali.linux-fr.org/

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Ingo Molnar]

* Jean Delvare <khali@linux-fr.org> wrote:

> Indeed, all files on which mmap was failing were located on noexec'd
> devices (although for the cdrom the noexec is not explicitely stated
> in my /etc/fstab file). Your patch fixes my problem, mmap on these
> devices is working again. Thanks!
> 
> Now I'm only curious as to why the problem only affected me. Since it
> looks like noexec is implied on cdrom devices, the problem should have
> affected everyone. Or are the "!pt_gnu_stack binaries" something rare?
> I admit I have no idea what it refers to.

it means you are running an older distro which was built with a gcc that
doesnt generate PT_GNU_STACK signatures in binaries yet. On the biggest
distros the transition to PT_GNU_STACK binaries coincided with the
transition to a 2.6 kernel, so only people who run newer kernels on
older distros are affected, which is relatively rare since most 'older
distros' are not 2.6-ready in a number of system-support areas.
(although the kernel itself of course must be able to run old code too.)

	Ingo

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Jean Delvare]

> > Now I'm only curious as to why the problem only affected me. Since
> > it looks like noexec is implied on cdrom devices, the problem should
> > have affected everyone. Or are the "!pt_gnu_stack binaries"
> > something rare? I admit I have no idea what it refers to.
> 
> it means you are running an older distro which was built with a gcc
> that doesnt generate PT_GNU_STACK signatures in binaries yet. On the
> biggest distros the transition to PT_GNU_STACK binaries coincided with
> the transition to a 2.6 kernel, so only people who run newer kernels
> on older distros are affected, which is relatively rare since most
> 'older distros' are not 2.6-ready in a number of system-support areas.

OK, I understand now. Both affected systems are running Slackware 9.1,
which may be a bit old (October 2003), but 2.6-ready, so I am using
bleeding edge 2.6 kernels, with no problems (well until a few days ago
;))

> (although the kernel itself of course must be able to run old code
> too.)

Well, I didn't enable any special option as far as I can remember, and
it works just fine :)

Thanks.

-- 
Jean Delvare
http://khali.linux-fr.org/

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Ingo Molnar]

* Jean Delvare <khali@linux-fr.org> wrote:

> OK, I understand now. Both affected systems are running Slackware 9.1,
> which may be a bit old (October 2003), but 2.6-ready, so I am using
> bleeding edge 2.6 kernels, with no problems (well until a few days ago
> ;))

i didnt mean to imply anything negative about Slackware - it's a
borderline date of release when it could have been 2.6-ready (although
2.6 was not relased yet at that date ;) but not have new gcc yet. It is
a full and primary requirement of the kernel to run !pt_gnu_stack
binaries too - it just wasnt that well tested for a complete distro, for
the reasons i mentioned.

	Ingo

Re: mmap() on cdrom files fails since 2.6.9-rc2-bk2

[Bill Davidsen]

On Sun, 3 Oct 2004, Ingo Molnar wrote:


> it means you are running an older distro which was built with a gcc that
> doesnt generate PT_GNU_STACK signatures in binaries yet. On the biggest
> distros the transition to PT_GNU_STACK binaries coincided with the
> transition to a 2.6 kernel, so only people who run newer kernels on
> older distros are affected, which is relatively rare since most 'older
> distros' are not 2.6-ready in a number of system-support areas.
> (although the kernel itself of course must be able to run old code too.)

Perhaps the majority of people with "old distro" configurations are people
who followed the development kernels from machines which were nearly
current then. My primary test machine is (loosely) based on RH7.3, with
only the updates needed for 2.6 as it developed and of course any security
issuesI saw.

The lack of reported issues may be caused by people who survived the
development process being able to identify the majority of problems like
this as well as there not being a lot of old systems.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.