[Qemu-devel] Re: [regression] configure: add opengl detection

[Benjamin Poirier <benjamin.poirier@gmail.com>]

On Mon, Apr 4, 2011 at 6:13 PM, Michael Walle <michael@walle.cc> wrote:
> Hi Benjamin,
>
>> Let me know if you need more info.
>
> what happens if you configure with
>
> ./configure --target-list=x86_64-softmmu --disable-opengl
>

Works as usual.
The problem I'm facing stems from linking to libGL and memory
protection issues. The particular system I ran this on has the binary
nvidia driver and its companion libGL.so.260.19.44. As such I'd take
no offense if we wave it off as a "problem in the unsupported binary
drivers" and I'll be satisfied configuring with no opengl on that
system.

Nevertheless, I did investigate about what's happening a little
further to clearly show that the problem is on nvidia's side.
1) as stated earlier, qemu segfaults when linked with the opengl libraries.

2) if I start qemu under gdb and configure it not to stop on SIGUSR2
(as I had omitted before; handle SIGUSR2 nostop noprint), qemu runs
ok. Same goes for strace.

3) if we enable /proc/sys/debug/exception-trace, the kernel printks:
    qemu-system-x86[15693]: segfault at 10c7820 ip 00000000010c7820 sp
00007fff71e334c8 error 15

10c7820 is the faulting address. Looking at the core file, we see that
10c7820 is the famous code_gen_prologue:

Program terminated with signal 11, Segmentation fault.
#0  0x00000000010c7820 in code_gen_prologue ()

(gdb) x /20i code_gen_prologue
=> 0x10c7820 <code_gen_prologue>:	push   %rbp
   0x10c7821 <code_gen_prologue+1>:	push   %rbx
   0x10c7822 <code_gen_prologue+2>:	push   %r12
   0x10c7824 <code_gen_prologue+4>:	push   %r13
[...]

By adding some debug code to map_exec() and adding a sigsegv handler
(that prints /proc/self/maps) I can see that code_gen_prologue is
adequately mprotect()'ed PROT_EXEC. Come time to jump into it from
cpu_exec(), that map is no longer there, the page is not executable,
and qemu crashes with a segfault.

Here is my debug output:
[...]
0091a000-01125000 rw-p 00000000 00:00 0
[...]
Will now map_exec 0x10c7820
Running mprotect 0x10c7000
Result: 0
[...]
0091a000-010c7000 rw-p 00000000 00:00 0
010c7000-010c8000 rwxp 00000000 00:00 0
010c8000-01125000 rw-p 00000000 00:00 0
[...]
Got SIGSEGV at address: 0x10c7820
[...]
0091a000-01125000 rw-p 00000000 00:00 0

I suspect that the nvidia libraries are messing with memory
protection. A look at objdump -R /usr/lib/libGL.so.1 indicates it does
need the symbol mprotect(). I tried to confirm this. Using a kernel
tracer (ftrace, perf or lttng), I can see that there are usually over
500 mprotect system calls before qemu crashes, including this
interesting combination (ftrace output):
 qemu-system-x86-21216 [002] 87794.633373: sys_mprotect(start:
10c7000, len: 1000, prot: 7)
 qemu-system-x86-21216 [000] 87794.806065: sys_mprotect(start: 400000,
len: 2f1000, prot: 7)
 qemu-system-x86-21216 [000] 87794.806079: sys_mprotect(start: 8f0000,
len: 835000, prot: 3)

With prot: 3 (read, write) it is essentially undoing what was done
100+ ms. earlier. In order to track down exactly where that call comes
from I tried using an LD_PRELOAD wrapper around glibc's mprotect() -
source for the wrapper here: https://gist.github.com/905600
When I do that, qemu doesn't crash anymore. ftrace reports the number
of mprotect calls is down to 123 and the odd combination is no longer
present. I can put the wrapper code within qemu itself and forgo
LD_PRELOAD, result is the same - no crash.

I would've like to show the weird mprotect call coming out of libGL or
libnvidia-whatever so we could point the finger to nvidia, but alas.
I'm at a loss as to why it doesn't crash under gdb, strace or with a
wrapper. If anyone has thoughts on that, I'm all ears.

Thanks,
-Ben

>
> --
>  Michael
>