* [Qemu-devel] AArch64 regression
@ 2017-06-09 9:48 Edgar E. Iglesias
2017-06-09 10:54 ` Alex Bennée
0 siblings, 1 reply; 2+ messages in thread
From: Edgar E. Iglesias @ 2017-06-09 9:48 UTC (permalink / raw)
To: qemu-arm, qemu-devel, peter.maydell
Cc: Emilio G. Cota, Richard Henderson, fred.konrad, alex.bennee
Hi,
I was running a bunch of tests in preparation for the MMIO/exec
series but am running into regressions on master.
I'm at commit 64175afc695c0672876fbbfc31b299c86d562cb4
On the xlnx-zcu102 I see lockups, the console goes unresponsive.
On one of my vexpress-a15 setups (with an a57), I see weird
segfaults in user-space.
I bisected this multiple times with each of the test cases
and all of the bisects bring me to this commit:
commit e75449a346bf558296966a44277bfd93412c6da6
Author: Emilio G. Cota <cota@braap.org>
Date: Fri Apr 28 14:59:23 2017 -0400
target/aarch64: optimize indirect branches
Does this ring any bells? Are there any known issues at the moment?
I've copied a GDB backtrace form the ZynqMP lockup at the end
of this email.
Thanks,
Edgar
GDB backtrace:
(gdb) thread apply all bt
Thread 6 (Thread 0x7fff431fc700 (LWP 28586)):
#0 0x00007ffff5e2826d in __lll_lock_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007ffff5e21dbd in pthread_mutex_lock ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#2 0x0000555555b5cc40 in qemu_mutex_lock (
mutex=mutex@entry=0x55555620be80 <qemu_global_mutex>)
at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:61
#3 0x00005555557bc9cc in qemu_mutex_lock_iothread ()
at /home/edgar/src/c/qemu/qemu/cpus.c:1581
#4 0x000055555578e865 in cpu_handle_interrupt (last_tb=<synthetic pointer>,
cpu=0x5555566c40b0) at /home/edgar/src/c/qemu/qemu/cpu-exec.c:491
#5 cpu_exec (cpu=cpu@entry=0x7fffc543f4f8)
at /home/edgar/src/c/qemu/qemu/cpu-exec.c:670
#6 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc543f4f8)
at /home/edgar/src/c/qemu/qemu/cpus.c:1270
#7 qemu_tcg_cpu_thread_fn (arg=0x7fffc543f4f8)
at /home/edgar/src/c/qemu/qemu/cpus.c:1473
#8 0x00007ffff5e1f6ba in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#9 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 5 (Thread 0x7fff439fd700 (LWP 28585)):
#0 0x00007ffff5e2826d in __lll_lock_wait ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x00007ffff5e21dbd in pthread_mutex_lock ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#2 0x0000555555b5cc40 in qemu_mutex_lock (
mutex=mutex@entry=0x55555620be80 <qemu_global_mutex>)
at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:61
#3 0x00005555557bc9cc in qemu_mutex_lock_iothread ()
at /home/edgar/src/c/qemu/qemu/cpus.c:1581
#4 0x000055555578e865 in cpu_handle_interrupt (last_tb=<synthetic pointer>,
cpu=0x5555566c40b0) at /home/edgar/src/c/qemu/qemu/cpu-exec.c:491
#5 cpu_exec (cpu=cpu@entry=0x7fffc54208d0)
at /home/edgar/src/c/qemu/qemu/cpu-exec.c:670
#6 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc54208d0)
at /home/edgar/src/c/qemu/qemu/cpus.c:1270
#7 qemu_tcg_cpu_thread_fn (arg=0x7fffc54208d0)
at /home/edgar/src/c/qemu/qemu/cpus.c:1473
#8 0x00007ffff5e1f6ba in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#9 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 4 (Thread 0x7fff441fe700 (LWP 28584)):
#0 0x00007ffff5e21dac in pthread_mutex_lock ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#1 0x0000555555b5cc40 in qemu_mutex_lock (
mutex=mutex@entry=0x55555620be80 <qemu_global_mutex>)
at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:61
#2 0x00005555557bc9cc in qemu_mutex_lock_iothread ()
at /home/edgar/src/c/qemu/qemu/cpus.c:1581
#3 0x000055555578e865 in cpu_handle_interrupt (last_tb=<synthetic pointer>,
cpu=0x5555566c40b0) at /home/edgar/src/c/qemu/qemu/cpu-exec.c:491
#4 cpu_exec (cpu=cpu@entry=0x7fffc5401ca8)
at /home/edgar/src/c/qemu/qemu/cpu-exec.c:670
#5 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc5401ca8)
at /home/edgar/src/c/qemu/qemu/cpus.c:1270
#6 qemu_tcg_cpu_thread_fn (arg=0x7fffc5401ca8)
at /home/edgar/src/c/qemu/qemu/cpus.c:1473
#7 0x00007ffff5e1f6ba in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#8 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 3 (Thread 0x7fff449ff700 (LWP 28583)):
#0 0x00005555558c4c89 in regime_el (mmu_idx=ARMMMUIdx_S1NSE1,
env=0x7fffc53eb310) at /home/edgar/src/c/qemu/qemu/target/arm/helper.c:7028
#1 regime_tcr (mmu_idx=<optimized out>, env=0x7fffc53eb310)
at /home/edgar/src/c/qemu/qemu/target/arm/helper.c:7117
#2 arm_regime_tbi0 (env=env@entry=0x7fffc53eb310, mmu_idx=<optimized out>,
mmu_idx@entry=ARMMMUIdx_S12NSE1)
at /home/edgar/src/c/qemu/qemu/target/arm/helper.c:7142
#3 0x00005555557ba346 in cpu_get_tb_cpu_state (flags=<synthetic pointer>,
cs_base=<synthetic pointer>, pc=<synthetic pointer>, env=0x7fffc53eb310)
at /home/edgar/src/c/qemu/qemu/target/arm/cpu.h:2571
#4 helper_lookup_tb_ptr (env=0x7fffc53eb310, addr=18446743524088820436)
at /home/edgar/src/c/qemu/qemu/tcg-runtime.c:156
#5 0x00007fffcf2f7e53 in code_gen_buffer ()
#6 0x000055555578de7c in cpu_tb_exec (itb=<optimized out>,
itb=<optimized out>, cpu=0x7fffc5727478)
at /home/edgar/src/c/qemu/qemu/cpu-exec.c:166
#7 cpu_loop_exec_tb (tb_exit=<synthetic pointer>,
last_tb=<synthetic pointer>, tb=<optimized out>, cpu=0x7fffc5727478)
at /home/edgar/src/c/qemu/qemu/cpu-exec.c:574
#8 cpu_exec (cpu=cpu@entry=0x7fffc53e3080)
at /home/edgar/src/c/qemu/qemu/cpu-exec.c:672
#9 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc53e3080)
at /home/edgar/src/c/qemu/qemu/cpus.c:1270
#10 qemu_tcg_cpu_thread_fn (arg=0x7fffc53e3080)
at /home/edgar/src/c/qemu/qemu/cpus.c:1473
#11 0x00007ffff5e1f6ba in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#12 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 2 (Thread 0x7fffef7b7700 (LWP 28582)):
#0 0x00007ffff5b4f8e9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000555555b5d385 in qemu_futex_wait (val=<optimized out>,
f=<optimized out>) at /home/edgar/src/c/qemu/qemu/include/qemu/futex.h:26
#2 qemu_event_wait (ev=ev@entry=0x55555661bae4 <rcu_call_ready_event>)
at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:415
#3 0x0000555555b6cc6e in call_rcu_thread (opaque=<optimized out>)
at /home/edgar/src/c/qemu/qemu/util/rcu.c:249
#4 0x00007ffff5e1f6ba in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#5 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Thread 1 (Thread 0x7ffff7f31c00 (LWP 28578)):
#0 0x00007ffff5b49c21 in ppoll () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x0000555555b58f39 in ppoll (__ss=0x0, __timeout=0x7fffffffd5f0,
__nfds=<optimized out>, __fds=<optimized out>)
at /usr/include/x86_64-linux-gnu/bits/poll2.h:77
#2 qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>,
timeout=timeout@entry=9223371867734611302)
at /home/edgar/src/c/qemu/qemu/util/qemu-timer.c:334
#3 0x0000555555b59d6a in os_host_main_loop_wait (timeout=9223371867734611302)
at /home/edgar/src/c/qemu/qemu/util/main-loop.c:255
#4 main_loop_wait (nonblocking=<optimized out>)
at /home/edgar/src/c/qemu/qemu/util/main-loop.c:517
#5 0x0000555555779057 in main_loop () at /home/edgar/src/c/qemu/qemu/vl.c:1918
#6 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>)
at /home/edgar/src/c/qemu/qemu/vl.c:4752
(gdb)
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [Qemu-devel] AArch64 regression
2017-06-09 9:48 [Qemu-devel] AArch64 regression Edgar E. Iglesias
@ 2017-06-09 10:54 ` Alex Bennée
0 siblings, 0 replies; 2+ messages in thread
From: Alex Bennée @ 2017-06-09 10:54 UTC (permalink / raw)
To: Edgar E. Iglesias
Cc: qemu-arm, qemu-devel, peter.maydell, Emilio G. Cota,
Richard Henderson, fred.konrad
Edgar E. Iglesias <edgar.iglesias@gmail.com> writes:
> Hi,
>
> I was running a bunch of tests in preparation for the MMIO/exec
> series but am running into regressions on master.
> I'm at commit 64175afc695c0672876fbbfc31b299c86d562cb4
>
> On the xlnx-zcu102 I see lockups, the console goes unresponsive.
> On one of my vexpress-a15 setups (with an a57), I see weird
> segfaults in user-space.
>
> I bisected this multiple times with each of the test cases
> and all of the bisects bring me to this commit:
> commit e75449a346bf558296966a44277bfd93412c6da6
> Author: Emilio G. Cota <cota@braap.org>
> Date: Fri Apr 28 14:59:23 2017 -0400
>
> target/aarch64: optimize indirect branches
>
>
> Does this ring any bells? Are there any known issues at the moment?
> I've copied a GDB backtrace form the ZynqMP lockup at the end
> of this email.
Yes - see the PULL req thread when this got merged:
Message-ID: <87h8zrdh02.fsf@linaro.org>
I'm currently trying to narrow it down. It's tricky as the problem goes
away when I add fprintfs but it affects both single and multi-thread
modes so I don't think its a race.
It seems to only affect TB's looked up in lookup_tb_ptr via the htable
which is odd.
>
> Thanks,
> Edgar
>
> GDB backtrace:
> (gdb) thread apply all bt
>
> Thread 6 (Thread 0x7fff431fc700 (LWP 28586)):
> #0 0x00007ffff5e2826d in __lll_lock_wait ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #1 0x00007ffff5e21dbd in pthread_mutex_lock ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #2 0x0000555555b5cc40 in qemu_mutex_lock (
> mutex=mutex@entry=0x55555620be80 <qemu_global_mutex>)
> at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:61
> #3 0x00005555557bc9cc in qemu_mutex_lock_iothread ()
> at /home/edgar/src/c/qemu/qemu/cpus.c:1581
> #4 0x000055555578e865 in cpu_handle_interrupt (last_tb=<synthetic pointer>,
> cpu=0x5555566c40b0) at /home/edgar/src/c/qemu/qemu/cpu-exec.c:491
> #5 cpu_exec (cpu=cpu@entry=0x7fffc543f4f8)
> at /home/edgar/src/c/qemu/qemu/cpu-exec.c:670
> #6 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc543f4f8)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1270
> #7 qemu_tcg_cpu_thread_fn (arg=0x7fffc543f4f8)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1473
> #8 0x00007ffff5e1f6ba in start_thread ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #9 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Thread 5 (Thread 0x7fff439fd700 (LWP 28585)):
> #0 0x00007ffff5e2826d in __lll_lock_wait ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #1 0x00007ffff5e21dbd in pthread_mutex_lock ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #2 0x0000555555b5cc40 in qemu_mutex_lock (
> mutex=mutex@entry=0x55555620be80 <qemu_global_mutex>)
> at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:61
> #3 0x00005555557bc9cc in qemu_mutex_lock_iothread ()
> at /home/edgar/src/c/qemu/qemu/cpus.c:1581
> #4 0x000055555578e865 in cpu_handle_interrupt (last_tb=<synthetic pointer>,
> cpu=0x5555566c40b0) at /home/edgar/src/c/qemu/qemu/cpu-exec.c:491
> #5 cpu_exec (cpu=cpu@entry=0x7fffc54208d0)
> at /home/edgar/src/c/qemu/qemu/cpu-exec.c:670
> #6 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc54208d0)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1270
> #7 qemu_tcg_cpu_thread_fn (arg=0x7fffc54208d0)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1473
> #8 0x00007ffff5e1f6ba in start_thread ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #9 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Thread 4 (Thread 0x7fff441fe700 (LWP 28584)):
> #0 0x00007ffff5e21dac in pthread_mutex_lock ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #1 0x0000555555b5cc40 in qemu_mutex_lock (
> mutex=mutex@entry=0x55555620be80 <qemu_global_mutex>)
> at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:61
> #2 0x00005555557bc9cc in qemu_mutex_lock_iothread ()
> at /home/edgar/src/c/qemu/qemu/cpus.c:1581
> #3 0x000055555578e865 in cpu_handle_interrupt (last_tb=<synthetic pointer>,
> cpu=0x5555566c40b0) at /home/edgar/src/c/qemu/qemu/cpu-exec.c:491
> #4 cpu_exec (cpu=cpu@entry=0x7fffc5401ca8)
> at /home/edgar/src/c/qemu/qemu/cpu-exec.c:670
> #5 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc5401ca8)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1270
> #6 qemu_tcg_cpu_thread_fn (arg=0x7fffc5401ca8)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1473
> #7 0x00007ffff5e1f6ba in start_thread ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #8 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Thread 3 (Thread 0x7fff449ff700 (LWP 28583)):
> #0 0x00005555558c4c89 in regime_el (mmu_idx=ARMMMUIdx_S1NSE1,
> env=0x7fffc53eb310) at /home/edgar/src/c/qemu/qemu/target/arm/helper.c:7028
> #1 regime_tcr (mmu_idx=<optimized out>, env=0x7fffc53eb310)
> at /home/edgar/src/c/qemu/qemu/target/arm/helper.c:7117
> #2 arm_regime_tbi0 (env=env@entry=0x7fffc53eb310, mmu_idx=<optimized out>,
> mmu_idx@entry=ARMMMUIdx_S12NSE1)
> at /home/edgar/src/c/qemu/qemu/target/arm/helper.c:7142
> #3 0x00005555557ba346 in cpu_get_tb_cpu_state (flags=<synthetic pointer>,
> cs_base=<synthetic pointer>, pc=<synthetic pointer>, env=0x7fffc53eb310)
> at /home/edgar/src/c/qemu/qemu/target/arm/cpu.h:2571
> #4 helper_lookup_tb_ptr (env=0x7fffc53eb310, addr=18446743524088820436)
> at /home/edgar/src/c/qemu/qemu/tcg-runtime.c:156
> #5 0x00007fffcf2f7e53 in code_gen_buffer ()
> #6 0x000055555578de7c in cpu_tb_exec (itb=<optimized out>,
> itb=<optimized out>, cpu=0x7fffc5727478)
> at /home/edgar/src/c/qemu/qemu/cpu-exec.c:166
> #7 cpu_loop_exec_tb (tb_exit=<synthetic pointer>,
> last_tb=<synthetic pointer>, tb=<optimized out>, cpu=0x7fffc5727478)
> at /home/edgar/src/c/qemu/qemu/cpu-exec.c:574
> #8 cpu_exec (cpu=cpu@entry=0x7fffc53e3080)
> at /home/edgar/src/c/qemu/qemu/cpu-exec.c:672
> #9 0x00005555557bcb9f in tcg_cpu_exec (cpu=0x7fffc53e3080)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1270
> #10 qemu_tcg_cpu_thread_fn (arg=0x7fffc53e3080)
> at /home/edgar/src/c/qemu/qemu/cpus.c:1473
> #11 0x00007ffff5e1f6ba in start_thread ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #12 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Thread 2 (Thread 0x7fffef7b7700 (LWP 28582)):
> #0 0x00007ffff5b4f8e9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x0000555555b5d385 in qemu_futex_wait (val=<optimized out>,
> f=<optimized out>) at /home/edgar/src/c/qemu/qemu/include/qemu/futex.h:26
> #2 qemu_event_wait (ev=ev@entry=0x55555661bae4 <rcu_call_ready_event>)
> at /home/edgar/src/c/qemu/qemu/util/qemu-thread-posix.c:415
> #3 0x0000555555b6cc6e in call_rcu_thread (opaque=<optimized out>)
> at /home/edgar/src/c/qemu/qemu/util/rcu.c:249
> #4 0x00007ffff5e1f6ba in start_thread ()
> from /lib/x86_64-linux-gnu/libpthread.so.0
> #5 0x00007ffff5b5582d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Thread 1 (Thread 0x7ffff7f31c00 (LWP 28578)):
> #0 0x00007ffff5b49c21 in ppoll () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x0000555555b58f39 in ppoll (__ss=0x0, __timeout=0x7fffffffd5f0,
> __nfds=<optimized out>, __fds=<optimized out>)
> at /usr/include/x86_64-linux-gnu/bits/poll2.h:77
> #2 qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>,
> timeout=timeout@entry=9223371867734611302)
> at /home/edgar/src/c/qemu/qemu/util/qemu-timer.c:334
> #3 0x0000555555b59d6a in os_host_main_loop_wait (timeout=9223371867734611302)
> at /home/edgar/src/c/qemu/qemu/util/main-loop.c:255
> #4 main_loop_wait (nonblocking=<optimized out>)
> at /home/edgar/src/c/qemu/qemu/util/main-loop.c:517
> #5 0x0000555555779057 in main_loop () at /home/edgar/src/c/qemu/qemu/vl.c:1918
> #6 main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>)
> at /home/edgar/src/c/qemu/qemu/vl.c:4752
> (gdb)
--
Alex Bennée
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2017-06-09 10:53 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-06-09 9:48 [Qemu-devel] AArch64 regression Edgar E. Iglesias
2017-06-09 10:54 ` Alex Bennée
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).