[Qemu-devel] qemu-arm fails on test-mmap - take #2

[Qemu-devel] qemu-arm fails on test-mmap - take #2

[Jan-Simon Möller]

Hi !

This is a follow-up to my tests about test-mmap failing inside an arm chroot 
when using qemu-arm in user-mode.

Here are 2 snippets running   "qemu-arm ./test-mmap"  outside and inside the 
ARM chroot env.

I turned DEBUG_MMAP on in linux-user/mmap.c .

legolas:/var/tmp/build-root # ./usr/bin/qemu-arm test-mmap
------snip-------
mmap: start=0x00000000 len=0x00001000 prot=r-- flags=MAP_ANON MAP_PRIVATE 
fd=-1 offset=00000000
ret=0x40b77000
start    end      size     prot
00008000-00081000 00079000 r-x
00088000-0008a000 00002000 rw-
0008a000-0008c000 00002000 rwx
0008c000-000af000 00023000 rw-
40000000-40080000 00080000 rw-
40080000-40081000 00001000 ---
40081000-40082000 00001000 rw-
40339000-40347000 0000e000 ---
407d6000-407d7000 00001000 ---
4096b000-40974000 00009000 ---
40b76000-40b78000 00002000 r--
40e8a000-40e8c000 00002000 ---
41dc7000-455ca000 03803000 ---
60000000-60166000 00166000 ---
60266000-6231d000 020b7000 ---
------snip-------

Note the last 2 lines !!

Now same procedure inside the chroot ...
legolas:/> chroot /var/tmp/build-root
legolas:/> ./test-mmap

------snip-------
mmap: start=0x00000000 len=0x00001000 prot=r-- flags=MAP_ANON MAP_PRIVATE 
fd=-1 offset=00000000
ret=0x40b74000
start    end      size     prot
00008000-00081000 00079000 r-x
00088000-0008a000 00002000 rw-
0008a000-0008c000 00002000 rwx
0008c000-000af000 00023000 rw-
40000000-40080000 00080000 rw-
40080000-40081000 00001000 ---
40081000-40082000 00001000 rw-
40191000-40192000 00001000 ---
40b73000-40b75000 00002000 r--
40c6c000-40c75000 00009000 ---
41105000-41106000 00001000 ---
41283000-44a95000 03812000 ---
------snip-------


The lines with 60000000-60166000 are gone ... 
Thus as soon as those pages will get allocated  it will segfault.

This is reproducible on different machines:
1)
openSUSE 11.1 64bit on Core 2 Duo  with 2GB Ram, qemu git head
The ARM chroot uses gcc4.4 and glibc 2.10.1 .

2)
openSUSE 11.0 32bit  Athlon XP 2GB RAM, same qemu, same chroot


Now my question is: how is the data of the already blocked pages 
processed/aquired ? It seems to me that the pages get "lost" somewhere.


Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap - take #2

[Jan-Simon Möller]

A quick hack around is this:

diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index e05caa0..a04d6b1 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -316,8 +316,14 @@ abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size)
         if (addr == addr_start)
             return (abi_ulong)-1;
     }
-    if (start == 0)
-        mmap_next_start = addr + size;
+    if (start == 0) {
+        abi_ulong mynext = addr + size ;
+        if ( (mynext > 0x5f000000) && (mynext < 0x65000000) ) {
+          mmap_next_start = 0x65000000 ;
+        } else {
+          mmap_next_start = addr + size;
+        }
+    }
     return addr;
 }


But this is for sure not the real solution.

Best,
Jan-Simon

Re: [Qemu-devel] qemu-arm fails on test-mmap

[Martin Mohring]

Hi,

yesterday, we tracked down the problem to a kernel bug in this area:
http://bugzilla.kernel.org/attachment.cgi?id=17219 What do I want to say
with this: the user mode memory allocator does not work because it runs
into a kernel bug.

Kudos to Jan-Simon. Has anyone an idea how to handle such a case....
Patching the kernel is not the best idea. We will check which kernels
are affected.

Cheers, Martin

Jan-Simon Möller wrote:
> A quick hack around is this:
>
> diff --git a/linux-user/mmap.c b/linux-user/mmap.c
> index e05caa0..a04d6b1 100644
> --- a/linux-user/mmap.c
> +++ b/linux-user/mmap.c
> @@ -316,8 +316,14 @@ abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size)
>          if (addr == addr_start)
>              return (abi_ulong)-1;
>      }
> -    if (start == 0)
> -        mmap_next_start = addr + size;
> +    if (start == 0) {
> +        abi_ulong mynext = addr + size ;
> +        if ( (mynext > 0x5f000000) && (mynext < 0x65000000) ) {
> +          mmap_next_start = 0x65000000 ;
> +        } else {
> +          mmap_next_start = addr + size;
> +        }
> +    }
>      return addr;
>  }
>
>
> But this is for sure not the real solution.
>
> Best,
> Jan-Simon
>
>
>   

Re: [Qemu-devel] qemu-arm fails on test-mmap - take #2

[Jan-Simon Möller]

[-- Attachment #1: Type: text/plain, Size: 988 bytes --]

Am Montag 10 August 2009 08:33:38 schrieb Kirill A. Shutemov:
> > Now my question is: how is the data of the already blocked pages
> > processed/aquired ? It seems to me that the pages get "lost" somewhere.
>
> Current mmap() implementation requires /proc to obtain all mappings.
>
> I have alternative implementation of mmap_find_vma() which doesn't
> require /proc:
>
> http://www.archivum.info/qemu-devel@nongnu.org/2008-10/01118/%5BQemu-devel%
>5D_%5BPATCH,_v2%5D_Rewrite_mmap_find_vma%28%29_to_work_fine_on_64-bit_hosts_
>with_32-bit_targets
>
> It is probably out of date.

This does the trick !  I've attached a version against trunk.

@Riku: can we apply this to git ?


Acked-by:  Jan-Simon Möller <dl9pf@gmx.de>

Original post: 
http://www.archivum.info/qemu-devel%40nongnu.org/2008-10/01118/[Qemu-devel]_[PATCH,_v2]_Rewrite_mmap_find_vma()_to_work_fine_on_64-bit_hosts_with_32-bit_targets

Attachement:
Patch rebased to trunk.


Best,
Jan-Simon

[-- Attachment #2: qemu-0.11-git-user-linux-mmap_find_vma.patch --]
[-- Type: text/x-diff, Size: 3284 bytes --]

diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index 6f300a0..8cec230 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -275,52 +275,59 @@ abi_ulong mmap_next_start = 0x40000000;
 
 unsigned long last_brk;
 
-/* find a free memory area of size 'size'. The search starts at
-   'start'. If 'start' == 0, then a default start address is used.
-   Return -1 if error.
-*/
-/* page_init() marks pages used by the host as reserved to be sure not
-   to use them. */
-abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size)
+/*
+ * Find and reserve a free memory area of size 'size'. The search
+ * starts at 'start'.
+ * It must be called with mmap_lock() held.
+ * Return -1 if error.
+ */
+abi_ulong mmap_find_vma(abi_ulong start, abi_ulong size)
 {
-    abi_ulong addr, addr1, addr_start;
-    int prot;
-    unsigned long new_brk;
-
-    new_brk = (unsigned long)sbrk(0);
-    if (last_brk && last_brk < new_brk && last_brk == (target_ulong)last_brk) {
-        /* This is a hack to catch the host allocating memory with brk().
-           If it uses mmap then we loose.
-           FIXME: We really want to avoid the host allocating memory in
-           the first place, and maybe leave some slack to avoid switching
-           to mmap.  */
-        page_set_flags(last_brk & TARGET_PAGE_MASK,
-                       TARGET_PAGE_ALIGN(new_brk),
-                       PAGE_RESERVED); 
-    }
-    last_brk = new_brk;
+    void *ptr;
+    abi_ulong addr;
 
     size = HOST_PAGE_ALIGN(size);
-    start = start & qemu_host_page_mask;
+    start &= qemu_host_page_mask;
+
+    /* If 'start' == 0, then a default start address is used. */
+    if (start == 0)
+        start = mmap_next_start;
+
     addr = start;
-    if (addr == 0)
-        addr = mmap_next_start;
-    addr_start = addr;
+
     for(;;) {
-        prot = 0;
-        for(addr1 = addr; addr1 < (addr + size); addr1 += TARGET_PAGE_SIZE) {
-            prot |= page_get_flags(addr1);
-        }
-        if (prot == 0)
+        /*
+         * Reserve needed memory area to avoid a race.
+         * It should be discarded using:
+         *  - mmap() with MAP_FIXED flag
+         *  - mremap() with MREMAP_FIXED flag
+         *  - shmat() with SHM_REMAP flag
+         */
+        ptr = mmap((void *)(unsigned long)addr, size, PROT_NONE,
+                   MAP_ANONYMOUS|MAP_PRIVATE|MAP_NORESERVE, -1, 0);
+
+        /* ENOMEM, if host address space has no memory */
+        if (ptr == MAP_FAILED)
+            return (abi_ulong)-1;
+
+        /* If address fits target address space we've found what we need */
+        if ((unsigned long)ptr + size - 1 <= (abi_ulong)-1)
             break;
+
+        /* Unmap and try again with new page */
+        munmap(ptr, size);
         addr += qemu_host_page_size;
-        /* we found nothing */
-        if (addr == addr_start)
+
+        /* ENOMEM if we check whole of target address space */
+        if (addr == start)
             return (abi_ulong)-1;
     }
-    if (start == 0)
-        mmap_next_start = addr + size;
-    return addr;
+
+    /* Update default start address */
+    if (start == mmap_next_start)
+        mmap_next_start = (unsigned long)ptr + size;
+
+    return h2g(ptr);
 }
 
 /* NOTE: all the constants are the HOST ones */