From: Christian Borntraeger <borntraeger@de.ibm.com>
To: Paolo Bonzini <pbonzini@redhat.com>, Fam Zheng <famz@redhat.com>,
qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
qemu-block@nongnu.org, "Michael S. Tsirkin" <mst@redhat.com>,
tubo@linux.vnet.ibm.com, Stefan Hajnoczi <stefanha@redhat.com>,
cornelia.huck@de.ibm.com
Subject: Re: [Qemu-devel] [PATCH 0/4] Tweaks around virtio-blk start/stop
Date: Wed, 16 Mar 2016 11:49:06 +0100 [thread overview]
Message-ID: <56E93A22.1080102@de.ibm.com> (raw)
In-Reply-To: <56E9355A.5070700@redhat.com>
On 03/16/2016 11:28 AM, Paolo Bonzini wrote:
>
>
> On 16/03/2016 11:10, Fam Zheng wrote:
>> These are some ideas originated from analyzing the Christian's crash report on
>> virtio-blk dataplane torture test:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02093.html
>>
>> The ideas are mostly inspired/suggested by Paolo. This doesn't fix the bug, but
>> the first and the last patches seem to make the crash less frequent. Also
>> thanks Cornelia Huck for reviewing the draft version posted in that thread.
>
> I see you have fixed the mutex and started check in patch 4, so perhaps
> this fixes the bug. :) Bo or Christian, could you try it out---and if
> it works try patches 2 to 4 only?
>
> Thanks,
>
> Paolo
>
Seems to lockup.
Thread 5 (Thread 0x3ff8b2ff910 (LWP 88956)):
#0 0x000003ff8c97f13e in syscall () at /lib64/libc.so.6
#1 0x00000000803d52fe in futex_wait (ev=0x80a4a104 <rcu_call_ready_event>, val=4294967295) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:292
#2 0x00000000803d558e in qemu_event_wait (ev=0x80a4a104 <rcu_call_ready_event>) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:399
#3 0x00000000803f2c34 in call_rcu_thread (opaque=0x0) at /home/cborntra/REPOS/qemu/util/rcu.c:250
#4 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#5 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 4 (Thread 0x3ff8aaff910 (LWP 88957)):
#0 0x000003ff8c9784d8 in ppoll () at /lib64/libc.so.6
#1 0x00000000802efdca in qemu_poll_ns (fds=0x3ff84002240, nfds=2, timeout=-1) at /home/cborntra/REPOS/qemu/qemu-timer.c:313
#2 0x00000000802f2528 in aio_poll (ctx=0xb9e94050, blocking=true) at /home/cborntra/REPOS/qemu/aio-posix.c:453
#3 0x000000008016392a in iothread_run (opaque=0xb9e93b10) at /home/cborntra/REPOS/qemu/iothread.c:46
#4 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#5 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 3 (Thread 0x3ff888dc910 (LWP 88958)):
#0 0x000003ff8ca90cd4 in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x000003ff8ca93e74 in __lll_lock_elision () at /lib64/libpthread.so.0
#2 0x00000000803d49ce in qemu_mutex_lock (mutex=0x8061f260 <qemu_global_mutex>) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:64
#3 0x0000000080060ef4 in qemu_mutex_lock_iothread () at /home/cborntra/REPOS/qemu/cpus.c:1226
#4 0x0000000080156af6 in kvm_arch_handle_exit (cs=0xba23b7f0, run=0x3ff8a200000) at /home/cborntra/REPOS/qemu/target-s390x/kvm.c:2024
#5 0x00000000800815de in kvm_cpu_exec (cpu=0xba23b7f0) at /home/cborntra/REPOS/qemu/kvm-all.c:1921
#6 0x000000008006074c in qemu_kvm_cpu_thread_fn (arg=0xba23b7f0) at /home/cborntra/REPOS/qemu/cpus.c:1050
#7 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#8 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 2 (Thread 0x3ff67fff910 (LWP 88959)):
#0 0x000003ff8ca90d04 in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x000003ff8ca93e74 in __lll_lock_elision () at /lib64/libpthread.so.0
#2 0x00000000803d49ce in qemu_mutex_lock (mutex=0x8061f260 <qemu_global_mutex>) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:64
#3 0x0000000080060ef4 in qemu_mutex_lock_iothread () at /home/cborntra/REPOS/qemu/cpus.c:1226
#4 0x0000000080156af6 in kvm_arch_handle_exit (cs=0xb9f2e970, run=0x3ff88080000) at /home/cborntra/REPOS/qemu/target-s390x/kvm.c:2024
#5 0x00000000800815de in kvm_cpu_exec (cpu=0xb9f2e970) at /home/cborntra/REPOS/qemu/kvm-all.c:1921
#6 0x000000008006074c in qemu_kvm_cpu_thread_fn (arg=0xb9f2e970) at /home/cborntra/REPOS/qemu/cpus.c:1050
#7 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#8 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 1 (Thread 0x3ff8e55bb90 (LWP 88953)):
#0 0x000003ff8ca90cd4 in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x000003ff8ca93e74 in __lll_lock_elision () at /lib64/libpthread.so.0
#2 0x00000000803d49ce in qemu_mutex_lock (mutex=0xba232df8) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:64
#3 0x00000000800b713e in virtio_blk_data_plane_start (s=0xba232d80) at /home/cborntra/REPOS/qemu/hw/block/dataplane/virtio-blk.c:224
#4 0x00000000800b4ea0 in virtio_blk_handle_output (vdev=0xb9eee7e8, vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/block/virtio-blk.c:590
#5 0x00000000800ef3dc in virtio_queue_notify_vq (vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1095
#6 0x00000000800f1c9c in virtio_queue_host_notifier_read (n=0xba3052c8) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1785
#7 0x00000000800f1e14 in virtio_queue_set_host_notifier_fd_handler (vq=0xba305270, assign=false, set_handler=false) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1817
#8 0x0000000080109c50 in virtio_ccw_set_guest2host_notifier (dev=0xb9eed6a0, n=0, assign=false, set_handler=false) at /home/cborntra/REPOS/qemu/hw/s390x/virtio-ccw.c:97
#9 0x0000000080109ef2 in virtio_ccw_stop_ioeventfd (dev=0xb9eed6a0) at /home/cborntra/REPOS/qemu/hw/s390x/virtio-ccw.c:154
#10 0x000000008010d5aa in virtio_ccw_set_host_notifier (d=0xb9eed6a0, n=0, assign=true) at /home/cborntra/REPOS/qemu/hw/s390x/virtio-ccw.c:1211
#11 0x00000000800b722c in virtio_blk_data_plane_start (s=0xba232d80) at /home/cborntra/REPOS/qemu/hw/block/dataplane/virtio-blk.c:242
#12 0x00000000800b4ea0 in virtio_blk_handle_output (vdev=0xb9eee7e8, vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/block/virtio-blk.c:590
#13 0x00000000800ef3dc in virtio_queue_notify_vq (vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1095
#14 0x00000000800f1c9c in virtio_queue_host_notifier_read (n=0xba3052c8) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1785
#15 0x00000000802f1cd4 in aio_dispatch (ctx=0xb9e81c70) at /home/cborntra/REPOS/qemu/aio-posix.c:327
#16 0x00000000802df31c in aio_ctx_dispatch (source=0xb9e81c70, callback=0x0, user_data=0x0) at /home/cborntra/REPOS/qemu/async.c:232
---Type <return> to continue, or q <return> to quit---
#17 0x000003ff8d2d1c0a in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#18 0x00000000802ee9a6 in glib_pollfds_poll () at /home/cborntra/REPOS/qemu/main-loop.c:212
#19 0x00000000802eeae2 in os_host_main_loop_wait (timeout=979000000) at /home/cborntra/REPOS/qemu/main-loop.c:257
#20 0x00000000802eebee in main_loop_wait (nonblocking=0) at /home/cborntra/REPOS/qemu/main-loop.c:505
#21 0x000000008017b4a4 in main_loop () at /home/cborntra/REPOS/qemu/vl.c:1933
#22 0x0000000080183992 in main (argc=72, argv=0x3ffebc7e908, envp=0x3ffebc7eb50) at /home/cborntra/REPOS/qemu/vl.c:4656
next prev parent reply other threads:[~2016-03-16 10:49 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-16 10:10 [Qemu-devel] [PATCH 0/4] Tweaks around virtio-blk start/stop Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 1/4] block: Use drained section in bdrv_set_aio_context Fam Zheng
2016-03-16 10:27 ` Paolo Bonzini
2016-03-16 10:51 ` Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 2/4] block-backend: Introduce blk_drained_begin/end Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 3/4] virtio-blk: Use blk_drained_begin/end around dataplane stop Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 4/4] virtio-blk: Clean up start/stop with mutex and BH Fam Zheng
2016-03-17 15:00 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2016-03-17 15:07 ` Paolo Bonzini
2016-03-22 12:52 ` Fam Zheng
2016-03-22 18:05 ` Paolo Bonzini
2016-03-23 8:10 ` Cornelia Huck
2016-03-23 9:08 ` Paolo Bonzini
2016-03-23 9:12 ` Christian Borntraeger
2016-03-24 8:19 ` tu bo
2016-03-24 8:32 ` Cornelia Huck
2016-03-24 8:47 ` Cornelia Huck
2016-03-24 9:31 ` Cornelia Huck
2016-03-16 10:28 ` [Qemu-devel] [PATCH 0/4] Tweaks around virtio-blk start/stop Paolo Bonzini
2016-03-16 10:49 ` Christian Borntraeger [this message]
2016-03-16 11:09 ` Paolo Bonzini
2016-03-16 11:24 ` Christian Borntraeger
2016-03-16 12:55 ` Paolo Bonzini
2016-03-16 13:38 ` Christian Borntraeger
2016-03-16 13:45 ` Paolo Bonzini
2016-03-17 0:39 ` Fam Zheng
2016-03-17 11:03 ` tu bo
2016-03-21 10:57 ` Fam Zheng
2016-03-21 11:15 ` Cornelia Huck
2016-03-21 12:45 ` Fam Zheng
2016-03-21 13:02 ` Cornelia Huck
2016-03-21 23:45 ` Fam Zheng
2016-03-22 8:06 ` Cornelia Huck
2016-03-22 7:10 ` tu bo
2016-03-22 7:18 ` Fam Zheng
2016-03-22 9:07 ` Cornelia Huck
2016-03-22 9:46 ` Paolo Bonzini
2016-03-22 11:59 ` Cornelia Huck
2016-03-22 12:11 ` Paolo Bonzini
2016-03-22 12:54 ` Cornelia Huck
2016-03-17 12:22 ` tu bo
2016-03-17 12:39 ` Christian Borntraeger
2016-03-17 13:02 ` Cornelia Huck
2016-03-17 15:02 ` Paolo Bonzini
2016-03-17 15:07 ` Christian Borntraeger
2016-03-17 15:15 ` Christian Borntraeger
2016-03-17 15:16 ` Christian Borntraeger
2016-03-17 16:08 ` Christian Borntraeger
2016-03-18 15:03 ` Paolo Bonzini
2016-03-21 9:42 ` Fam Zheng
2016-03-21 11:10 ` Christian Borntraeger
2016-03-21 12:17 ` Cornelia Huck
2016-03-21 13:47 ` TU BO
2016-03-21 13:54 ` Paolo Bonzini
2016-03-21 14:19 ` Cornelia Huck
2016-03-22 0:31 ` Fam Zheng
2016-03-16 11:32 ` Cornelia Huck
2016-03-16 11:48 ` Paolo Bonzini
2016-03-16 11:56 ` Cornelia Huck
2016-03-16 11:59 ` Paolo Bonzini
2016-03-16 12:22 ` Cornelia Huck
2016-03-16 12:32 ` Paolo Bonzini
2016-03-16 12:42 ` Cornelia Huck
2016-03-16 12:49 ` Paolo Bonzini
2016-03-16 13:04 ` Cornelia Huck
2016-03-16 13:10 ` Paolo Bonzini
2016-03-16 13:14 ` Cornelia Huck
2016-03-16 13:15 ` Paolo Bonzini
2016-03-16 11:52 ` Cornelia Huck
2016-03-16 11:54 ` Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56E93A22.1080102@de.ibm.com \
--to=borntraeger@de.ibm.com \
--cc=cornelia.huck@de.ibm.com \
--cc=famz@redhat.com \
--cc=kwolf@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
--cc=tubo@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).