From: Christian Borntraeger <borntraeger@de.ibm.com>
To: Paolo Bonzini <pbonzini@redhat.com>, Fam Zheng <famz@redhat.com>,
qemu-devel@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
qemu-block@nongnu.org, "Michael S. Tsirkin" <mst@redhat.com>,
tubo@linux.vnet.ibm.com, Stefan Hajnoczi <stefanha@redhat.com>,
cornelia.huck@de.ibm.com
Subject: Re: [Qemu-devel] [PATCH 0/4] Tweaks around virtio-blk start/stop
Date: Wed, 16 Mar 2016 11:49:06 +0100 [thread overview]
Message-ID: <56E93A22.1080102@de.ibm.com> (raw)
In-Reply-To: <56E9355A.5070700@redhat.com>
On 03/16/2016 11:28 AM, Paolo Bonzini wrote:
>
>
> On 16/03/2016 11:10, Fam Zheng wrote:
>> These are some ideas originated from analyzing the Christian's crash report on
>> virtio-blk dataplane torture test:
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02093.html
>>
>> The ideas are mostly inspired/suggested by Paolo. This doesn't fix the bug, but
>> the first and the last patches seem to make the crash less frequent. Also
>> thanks Cornelia Huck for reviewing the draft version posted in that thread.
>
> I see you have fixed the mutex and started check in patch 4, so perhaps
> this fixes the bug. :) Bo or Christian, could you try it out---and if
> it works try patches 2 to 4 only?
>
> Thanks,
>
> Paolo
>
Seems to lockup.
Thread 5 (Thread 0x3ff8b2ff910 (LWP 88956)):
#0 0x000003ff8c97f13e in syscall () at /lib64/libc.so.6
#1 0x00000000803d52fe in futex_wait (ev=0x80a4a104 <rcu_call_ready_event>, val=4294967295) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:292
#2 0x00000000803d558e in qemu_event_wait (ev=0x80a4a104 <rcu_call_ready_event>) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:399
#3 0x00000000803f2c34 in call_rcu_thread (opaque=0x0) at /home/cborntra/REPOS/qemu/util/rcu.c:250
#4 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#5 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 4 (Thread 0x3ff8aaff910 (LWP 88957)):
#0 0x000003ff8c9784d8 in ppoll () at /lib64/libc.so.6
#1 0x00000000802efdca in qemu_poll_ns (fds=0x3ff84002240, nfds=2, timeout=-1) at /home/cborntra/REPOS/qemu/qemu-timer.c:313
#2 0x00000000802f2528 in aio_poll (ctx=0xb9e94050, blocking=true) at /home/cborntra/REPOS/qemu/aio-posix.c:453
#3 0x000000008016392a in iothread_run (opaque=0xb9e93b10) at /home/cborntra/REPOS/qemu/iothread.c:46
#4 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#5 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 3 (Thread 0x3ff888dc910 (LWP 88958)):
#0 0x000003ff8ca90cd4 in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x000003ff8ca93e74 in __lll_lock_elision () at /lib64/libpthread.so.0
#2 0x00000000803d49ce in qemu_mutex_lock (mutex=0x8061f260 <qemu_global_mutex>) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:64
#3 0x0000000080060ef4 in qemu_mutex_lock_iothread () at /home/cborntra/REPOS/qemu/cpus.c:1226
#4 0x0000000080156af6 in kvm_arch_handle_exit (cs=0xba23b7f0, run=0x3ff8a200000) at /home/cborntra/REPOS/qemu/target-s390x/kvm.c:2024
#5 0x00000000800815de in kvm_cpu_exec (cpu=0xba23b7f0) at /home/cborntra/REPOS/qemu/kvm-all.c:1921
#6 0x000000008006074c in qemu_kvm_cpu_thread_fn (arg=0xba23b7f0) at /home/cborntra/REPOS/qemu/cpus.c:1050
#7 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#8 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 2 (Thread 0x3ff67fff910 (LWP 88959)):
#0 0x000003ff8ca90d04 in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x000003ff8ca93e74 in __lll_lock_elision () at /lib64/libpthread.so.0
#2 0x00000000803d49ce in qemu_mutex_lock (mutex=0x8061f260 <qemu_global_mutex>) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:64
#3 0x0000000080060ef4 in qemu_mutex_lock_iothread () at /home/cborntra/REPOS/qemu/cpus.c:1226
#4 0x0000000080156af6 in kvm_arch_handle_exit (cs=0xb9f2e970, run=0x3ff88080000) at /home/cborntra/REPOS/qemu/target-s390x/kvm.c:2024
#5 0x00000000800815de in kvm_cpu_exec (cpu=0xb9f2e970) at /home/cborntra/REPOS/qemu/kvm-all.c:1921
#6 0x000000008006074c in qemu_kvm_cpu_thread_fn (arg=0xb9f2e970) at /home/cborntra/REPOS/qemu/cpus.c:1050
#7 0x000003ff8ca87c2c in start_thread () at /lib64/libpthread.so.0
#8 0x000003ff8c984c7a in thread_start () at /lib64/libc.so.6
Thread 1 (Thread 0x3ff8e55bb90 (LWP 88953)):
#0 0x000003ff8ca90cd4 in __lll_lock_wait () at /lib64/libpthread.so.0
#1 0x000003ff8ca93e74 in __lll_lock_elision () at /lib64/libpthread.so.0
#2 0x00000000803d49ce in qemu_mutex_lock (mutex=0xba232df8) at /home/cborntra/REPOS/qemu/util/qemu-thread-posix.c:64
#3 0x00000000800b713e in virtio_blk_data_plane_start (s=0xba232d80) at /home/cborntra/REPOS/qemu/hw/block/dataplane/virtio-blk.c:224
#4 0x00000000800b4ea0 in virtio_blk_handle_output (vdev=0xb9eee7e8, vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/block/virtio-blk.c:590
#5 0x00000000800ef3dc in virtio_queue_notify_vq (vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1095
#6 0x00000000800f1c9c in virtio_queue_host_notifier_read (n=0xba3052c8) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1785
#7 0x00000000800f1e14 in virtio_queue_set_host_notifier_fd_handler (vq=0xba305270, assign=false, set_handler=false) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1817
#8 0x0000000080109c50 in virtio_ccw_set_guest2host_notifier (dev=0xb9eed6a0, n=0, assign=false, set_handler=false) at /home/cborntra/REPOS/qemu/hw/s390x/virtio-ccw.c:97
#9 0x0000000080109ef2 in virtio_ccw_stop_ioeventfd (dev=0xb9eed6a0) at /home/cborntra/REPOS/qemu/hw/s390x/virtio-ccw.c:154
#10 0x000000008010d5aa in virtio_ccw_set_host_notifier (d=0xb9eed6a0, n=0, assign=true) at /home/cborntra/REPOS/qemu/hw/s390x/virtio-ccw.c:1211
#11 0x00000000800b722c in virtio_blk_data_plane_start (s=0xba232d80) at /home/cborntra/REPOS/qemu/hw/block/dataplane/virtio-blk.c:242
#12 0x00000000800b4ea0 in virtio_blk_handle_output (vdev=0xb9eee7e8, vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/block/virtio-blk.c:590
#13 0x00000000800ef3dc in virtio_queue_notify_vq (vq=0xba305270) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1095
#14 0x00000000800f1c9c in virtio_queue_host_notifier_read (n=0xba3052c8) at /home/cborntra/REPOS/qemu/hw/virtio/virtio.c:1785
#15 0x00000000802f1cd4 in aio_dispatch (ctx=0xb9e81c70) at /home/cborntra/REPOS/qemu/aio-posix.c:327
#16 0x00000000802df31c in aio_ctx_dispatch (source=0xb9e81c70, callback=0x0, user_data=0x0) at /home/cborntra/REPOS/qemu/async.c:232
---Type <return> to continue, or q <return> to quit---
#17 0x000003ff8d2d1c0a in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#18 0x00000000802ee9a6 in glib_pollfds_poll () at /home/cborntra/REPOS/qemu/main-loop.c:212
#19 0x00000000802eeae2 in os_host_main_loop_wait (timeout=979000000) at /home/cborntra/REPOS/qemu/main-loop.c:257
#20 0x00000000802eebee in main_loop_wait (nonblocking=0) at /home/cborntra/REPOS/qemu/main-loop.c:505
#21 0x000000008017b4a4 in main_loop () at /home/cborntra/REPOS/qemu/vl.c:1933
#22 0x0000000080183992 in main (argc=72, argv=0x3ffebc7e908, envp=0x3ffebc7eb50) at /home/cborntra/REPOS/qemu/vl.c:4656
next prev parent reply other threads:[~2016-03-16 10:49 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-03-16 10:10 [Qemu-devel] [PATCH 0/4] Tweaks around virtio-blk start/stop Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 1/4] block: Use drained section in bdrv_set_aio_context Fam Zheng
2016-03-16 10:27 ` Paolo Bonzini
2016-03-16 10:51 ` Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 2/4] block-backend: Introduce blk_drained_begin/end Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 3/4] virtio-blk: Use blk_drained_begin/end around dataplane stop Fam Zheng
2016-03-16 10:10 ` [Qemu-devel] [PATCH 4/4] virtio-blk: Clean up start/stop with mutex and BH Fam Zheng
2016-03-17 15:00 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2016-03-17 15:07 ` Paolo Bonzini
2016-03-22 12:52 ` Fam Zheng
2016-03-22 18:05 ` Paolo Bonzini
2016-03-23 8:10 ` Cornelia Huck
2016-03-23 9:08 ` Paolo Bonzini
2016-03-23 9:12 ` Christian Borntraeger
2016-03-24 8:19 ` tu bo
2016-03-24 8:32 ` Cornelia Huck
2016-03-24 8:47 ` Cornelia Huck
2016-03-24 9:31 ` Cornelia Huck
2016-03-16 10:28 ` [Qemu-devel] [PATCH 0/4] Tweaks around virtio-blk start/stop Paolo Bonzini
2016-03-16 10:49 ` Christian Borntraeger [this message]
2016-03-16 11:09 ` Paolo Bonzini
2016-03-16 11:24 ` Christian Borntraeger
2016-03-16 12:55 ` Paolo Bonzini
2016-03-16 13:38 ` Christian Borntraeger
2016-03-16 13:45 ` Paolo Bonzini
2016-03-17 0:39 ` Fam Zheng
2016-03-17 11:03 ` tu bo
2016-03-21 10:57 ` Fam Zheng
2016-03-21 11:15 ` Cornelia Huck
2016-03-21 12:45 ` Fam Zheng
2016-03-21 13:02 ` Cornelia Huck
2016-03-21 23:45 ` Fam Zheng
2016-03-22 8:06 ` Cornelia Huck
2016-03-22 7:10 ` tu bo
2016-03-22 7:18 ` Fam Zheng
2016-03-22 9:07 ` Cornelia Huck
2016-03-22 9:46 ` Paolo Bonzini
2016-03-22 11:59 ` Cornelia Huck
2016-03-22 12:11 ` Paolo Bonzini
2016-03-22 12:54 ` Cornelia Huck
2016-03-17 12:22 ` tu bo
2016-03-17 12:39 ` Christian Borntraeger
2016-03-17 13:02 ` Cornelia Huck
2016-03-17 15:02 ` Paolo Bonzini
2016-03-17 15:07 ` Christian Borntraeger
2016-03-17 15:15 ` Christian Borntraeger
2016-03-17 15:16 ` Christian Borntraeger
2016-03-17 16:08 ` Christian Borntraeger
2016-03-18 15:03 ` Paolo Bonzini
2016-03-21 9:42 ` Fam Zheng
2016-03-21 11:10 ` Christian Borntraeger
2016-03-21 12:17 ` Cornelia Huck
2016-03-21 13:47 ` TU BO
2016-03-21 13:54 ` Paolo Bonzini
2016-03-21 14:19 ` Cornelia Huck
2016-03-22 0:31 ` Fam Zheng
2016-03-16 11:32 ` Cornelia Huck
2016-03-16 11:48 ` Paolo Bonzini
2016-03-16 11:56 ` Cornelia Huck
2016-03-16 11:59 ` Paolo Bonzini
2016-03-16 12:22 ` Cornelia Huck
2016-03-16 12:32 ` Paolo Bonzini
2016-03-16 12:42 ` Cornelia Huck
2016-03-16 12:49 ` Paolo Bonzini
2016-03-16 13:04 ` Cornelia Huck
2016-03-16 13:10 ` Paolo Bonzini
2016-03-16 13:14 ` Cornelia Huck
2016-03-16 13:15 ` Paolo Bonzini
2016-03-16 11:52 ` Cornelia Huck
2016-03-16 11:54 ` Paolo Bonzini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=56E93A22.1080102@de.ibm.com \
--to=borntraeger@de.ibm.com \
--cc=cornelia.huck@de.ibm.com \
--cc=famz@redhat.com \
--cc=kwolf@redhat.com \
--cc=mst@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
--cc=tubo@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.