[Qemu-devel] qemu-arm fails on test-mmap

[Qemu-devel] qemu-arm fails on test-mmap

[Jan-Simon Möller]

Hi!

I've spotted an bug in mmap for the qemu-arm . It causes a segfault of qemu or at least the running process.

Its reproducible here with "test-mmap" .

legolas:/> qemu-arm ./test-mmap
pagesize=4096 pagemask=fff
check_aligned_anonymous_unfixed_mmaps passed
check_aligned_anonymous_unfixed_colliding_mmapsSegmentation fault (core dumped)


A lengthy trace with debugging on in mmap.c is at
http://filebin.ca/yxypzq/qemu_mmap_segfault.bz2

[... last mmap call ...]
munmap: start=0x5fffd000 len=0x00001000
mmap: start=0x00000000 len=0x00008000 prot=r-- flags=MAP_ANON MAP_PRIVATE fd=-1 offset=00000000
ret=0x5ffff000
start    end      size     prot
00008000-0000b000 00003000 r-x
00012000-00013000 00001000 r--
00013000-00037000 00024000 rw-
40000000-40080000 00080000 rw-
40080000-40081000 00001000 ---
40081000-4009f000 0001e000 r-x
4009f000-400a6000 00007000 ---
400a6000-400a7000 00001000 r--
400a7000-400a8000 00001000 rw-
400a8000-42081000 01fd9000 ---
42085000-421bf000 0013a000 r-x
421bf000-421c6000 00007000 ---
421c6000-421c8000 00002000 r--
421c8000-421ce000 00006000 rw-
5fffe000-60007000 00009000 r--

Segmentation fault

It seems to hit > 0x5fffffff -> segfault.

Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Laurent Desnogues]

2009/7/31 Jan-Simon Möller <dl9pf@gmx.de>:
> Hi!
>
> I've spotted an bug in mmap for the qemu-arm . It causes a segfault of qemu or at least the running process.
>
> Its reproducible here with "test-mmap" .
>
> legolas:/> qemu-arm ./test-mmap
> pagesize=4096 pagemask=fff
> check_aligned_anonymous_unfixed_mmaps passed
> check_aligned_anonymous_unfixed_colliding_mmapsSegmentation fault (core dumped)
>
>
> A lengthy trace with debugging on in mmap.c is at
> http://filebin.ca/yxypzq/qemu_mmap_segfault.bz2
>
> [... last mmap call ...]
> munmap: start=0x5fffd000 len=0x00001000
> mmap: start=0x00000000 len=0x00008000 prot=r-- flags=MAP_ANON MAP_PRIVATE fd=-1 offset=00000000
> ret=0x5ffff000
> start    end      size     prot
> 00008000-0000b000 00003000 r-x
> 00012000-00013000 00001000 r--
> 00013000-00037000 00024000 rw-
> 40000000-40080000 00080000 rw-
> 40080000-40081000 00001000 ---
> 40081000-4009f000 0001e000 r-x
> 4009f000-400a6000 00007000 ---
> 400a6000-400a7000 00001000 r--
> 400a7000-400a8000 00001000 rw-
> 400a8000-42081000 01fd9000 ---
> 42085000-421bf000 0013a000 r-x
> 421bf000-421c6000 00007000 ---
> 421c6000-421c8000 00002000 r--
> 421c8000-421ce000 00006000 rw-
> 5fffe000-60007000 00009000 r--
>
> Segmentation fault
>
> It seems to hit > 0x5fffffff -> segfault.

I tested qemu-i386 and qemu-arm on both i386 and x86_64 and
the four pass that test.  That's with git head, and CSL 2009q1 on
my Fedora 11 x86_64 machine.


Laurent

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Jan-Simon Möller]

Am Friday 31 July 2009 11:58:04 schrieb Laurent Desnogues:
> I tested qemu-i386 and qemu-arm on both i386 and x86_64 and
> the four pass that test.  That's with git head, and CSL 2009q1 on
> my Fedora 11 x86_64 machine.

SMP or single-core host ?
What's "cat /proc/sys/vm/mmap_min_addr" in your environment ?

Best
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Laurent Desnogues]

2009/7/31 Jan-Simon Möller <dl9pf@gmx.de>:
>
> SMP or single-core host ?

Core i7 with SMT enabled so 8 core.

> What's "cat /proc/sys/vm/mmap_min_addr" in your environment ?

32768


Laurent

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Jan-Simon Möller]

Hi !

The funny thing is - i can run the test with the same binaries without problem _outside_ the arm chroot, but not inside:

dl9pf@legolas:~> /var/tmp/build-root/usr/bin/qemu-arm /var/tmp/build-root/test-mmap
pagesize=4096 pagemask=fff
check_aligned_anonymous_unfixed_mmaps passed
check_aligned_anonymous_unfixed_colliding_mmaps passed
check_aligned_anonymous_fixed_mmaps addr=0x6a271000 passed
check_file_unfixed_mmaps passed
check_file_fixed_mmaps addr=0x6a2c9000 passed
check_file_fixed_eof_mmaps addr=0x6a369000 passed
check_file_unfixed_eof_mmaps passed

legolas:/home/dl9pf # chroot /var/tmp/build-root

legolas:/> ./test-mmap
pagesize=4096 pagemask=fff
check_aligned_anonymous_unfixed_mmaps passed
check_aligned_anonymous_unfixed_colliding_mmapsSpeicherzugriffsfehler

legolas:/> file test-mmap
test-mmap: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, for GNU/Linux 2.6.16, not stripped
legolas:/> file usr/bin/qemu-arm
usr/bin/qemu-arm: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.4, not stripped


*scratching head*

Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Jan-Simon Möller]

Am Friday 31 July 2009 15:03:31 schrieb Jan-Simon Möller:
> Hi !
> 
> The funny thing is - i can run the test with the same binaries without problem _outside_ the arm chroot, but not inside:
> 
> dl9pf@legolas:~> /var/tmp/build-root/usr/bin/qemu-arm /var/tmp/build-root/test-mmap
> pagesize=4096 pagemask=fff
> check_aligned_anonymous_unfixed_mmaps passed
> check_aligned_anonymous_unfixed_colliding_mmaps passed
> check_aligned_anonymous_fixed_mmaps addr=0x6a271000 passed
> check_file_unfixed_mmaps passed
> check_file_fixed_mmaps addr=0x6a2c9000 passed
> check_file_fixed_eof_mmaps addr=0x6a369000 passed
> check_file_unfixed_eof_mmaps passed
> 
> legolas:/home/dl9pf # chroot /var/tmp/build-root
> 
> legolas:/> ./test-mmap
> pagesize=4096 pagemask=fff
> check_aligned_anonymous_unfixed_mmaps passed
> check_aligned_anonymous_unfixed_colliding_mmapsSpeicherzugriffsfehler
> 
> legolas:/> file test-mmap
> test-mmap: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, for GNU/Linux 2.6.16, not stripped
> legolas:/> file usr/bin/qemu-arm
> usr/bin/qemu-arm: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.4, not stripped
> 
> 
> *scratching head*

Inside the arm-chroot it segfaults when it hits 0x60000000 . No problems with that outside.

Any idea on this boundary ?

Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Jan-Simon Möller]

Am Freitag 31 Juli 2009 11:58:04 schrieb Laurent Desnogues:
I prepared bundles for::

qemu-arm  built for i586  which  can  run the test on a i586 machine, but not 
when its run on an x86_64 box.

Link for this package :  

http://www.stud.uni-hannover.de/~jsm/qemu-mmap_testbundle_prebuilt-i586.tar.bz2

qemu-arm build for x86_64 which can't run the test _inside_ my target chroot, 
but it runs on some hosts when its called standalone. (min_addr = 0 , runs 
probably)

Link: 
http://www.stud.uni-hannover.de/~jsm/qemu-mmap_testbundle_prebuilt_amd64.tar.bz2


Lets sum it up:

* qemu-arm compiled for i586 can run the test in the chroot @32bit host

* qemu-arm compiled for i586 can _not_ run the test on a 64bit host at all

* qemu-arm compiled for x86_64 can run the test only outside the target-chroot 

* The mmap tests fail  if they reach values >0x5fffffff  - starting from
    0x40000000 thats a 0x20000000 margin


It seems to me an issue on 64bit hosts with 32bit target, but I've no idea atm 
what causes this - especially the fact that it runs outside the target chroot 
on x86_64 .

Lest see if someone can reproduce this on his machine.

Best,
Jan-Simon

[Qemu-devel] qemu-arm fails on test-mmap - take #2

[Jan-Simon Möller]

Hi !

This is a follow-up to my tests about test-mmap failing inside an arm chroot 
when using qemu-arm in user-mode.

Here are 2 snippets running   "qemu-arm ./test-mmap"  outside and inside the 
ARM chroot env.

I turned DEBUG_MMAP on in linux-user/mmap.c .

legolas:/var/tmp/build-root # ./usr/bin/qemu-arm test-mmap
------snip-------
mmap: start=0x00000000 len=0x00001000 prot=r-- flags=MAP_ANON MAP_PRIVATE 
fd=-1 offset=00000000
ret=0x40b77000
start    end      size     prot
00008000-00081000 00079000 r-x
00088000-0008a000 00002000 rw-
0008a000-0008c000 00002000 rwx
0008c000-000af000 00023000 rw-
40000000-40080000 00080000 rw-
40080000-40081000 00001000 ---
40081000-40082000 00001000 rw-
40339000-40347000 0000e000 ---
407d6000-407d7000 00001000 ---
4096b000-40974000 00009000 ---
40b76000-40b78000 00002000 r--
40e8a000-40e8c000 00002000 ---
41dc7000-455ca000 03803000 ---
60000000-60166000 00166000 ---
60266000-6231d000 020b7000 ---
------snip-------

Note the last 2 lines !!

Now same procedure inside the chroot ...
legolas:/> chroot /var/tmp/build-root
legolas:/> ./test-mmap

------snip-------
mmap: start=0x00000000 len=0x00001000 prot=r-- flags=MAP_ANON MAP_PRIVATE 
fd=-1 offset=00000000
ret=0x40b74000
start    end      size     prot
00008000-00081000 00079000 r-x
00088000-0008a000 00002000 rw-
0008a000-0008c000 00002000 rwx
0008c000-000af000 00023000 rw-
40000000-40080000 00080000 rw-
40080000-40081000 00001000 ---
40081000-40082000 00001000 rw-
40191000-40192000 00001000 ---
40b73000-40b75000 00002000 r--
40c6c000-40c75000 00009000 ---
41105000-41106000 00001000 ---
41283000-44a95000 03812000 ---
------snip-------


The lines with 60000000-60166000 are gone ... 
Thus as soon as those pages will get allocated  it will segfault.

This is reproducible on different machines:
1)
openSUSE 11.1 64bit on Core 2 Duo  with 2GB Ram, qemu git head
The ARM chroot uses gcc4.4 and glibc 2.10.1 .

2)
openSUSE 11.0 32bit  Athlon XP 2GB RAM, same qemu, same chroot


Now my question is: how is the data of the already blocked pages 
processed/aquired ? It seems to me that the pages get "lost" somewhere.


Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap - take #2

[Jan-Simon Möller]

A quick hack around is this:

diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index e05caa0..a04d6b1 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -316,8 +316,14 @@ abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size)
         if (addr == addr_start)
             return (abi_ulong)-1;
     }
-    if (start == 0)
-        mmap_next_start = addr + size;
+    if (start == 0) {
+        abi_ulong mynext = addr + size ;
+        if ( (mynext > 0x5f000000) && (mynext < 0x65000000) ) {
+          mmap_next_start = 0x65000000 ;
+        } else {
+          mmap_next_start = addr + size;
+        }
+    }
     return addr;
 }


But this is for sure not the real solution.

Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Martin Mohring]

Hi,

yesterday, we tracked down the problem to a kernel bug in this area:
http://bugzilla.kernel.org/attachment.cgi?id=17219 What do I want to say
with this: the user mode memory allocator does not work because it runs
into a kernel bug.

Kudos to Jan-Simon. Has anyone an idea how to handle such a case....
Patching the kernel is not the best idea. We will check which kernels
are affected.

Cheers, Martin

Jan-Simon Möller wrote:
> A quick hack around is this:
>
> diff --git a/linux-user/mmap.c b/linux-user/mmap.c
> index e05caa0..a04d6b1 100644
> --- a/linux-user/mmap.c
> +++ b/linux-user/mmap.c
> @@ -316,8 +316,14 @@ abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size)
>          if (addr == addr_start)
>              return (abi_ulong)-1;
>      }
> -    if (start == 0)
> -        mmap_next_start = addr + size;
> +    if (start == 0) {
> +        abi_ulong mynext = addr + size ;
> +        if ( (mynext > 0x5f000000) && (mynext < 0x65000000) ) {
> +          mmap_next_start = 0x65000000 ;
> +        } else {
> +          mmap_next_start = addr + size;
> +        }
> +    }
>      return addr;
>  }
>
>
> But this is for sure not the real solution.
>
> Best,
> Jan-Simon
>
>
>