From: Kevin Wolf <kwolf@redhat.com>
To: qemu-block@nongnu.org
Cc: kwolf@redhat.com, qemu-devel@nongnu.org
Subject: [Qemu-devel] [PULL 02/27] block: Fix AioContext switch for bs->drv == NULL
Date: Tue, 30 Apr 2019 17:42:19 +0200 [thread overview]
Message-ID: <20190430154244.30083-3-kwolf@redhat.com> (raw)
In-Reply-To: <20190430154244.30083-1-kwolf@redhat.com>
Even for block nodes with bs->drv == NULL, we can't just ignore a
bdrv_set_aio_context() call. Leaving the node in its old context can
mean that it's still in an iothread context in bdrv_close_all() during
shutdown, resulting in an attempted unlock of the AioContext lock which
we don't hold.
This is an example stack trace of a related crash:
#0 0x00007ffff59da57f in raise () at /lib64/libc.so.6
#1 0x00007ffff59c4895 in abort () at /lib64/libc.so.6
#2 0x0000555555b97b1e in error_exit (err=<optimized out>, msg=msg@entry=0x555555d386d0 <__func__.19059> "qemu_mutex_unlock_impl") at util/qemu-thread-posix.c:36
#3 0x0000555555b97f7f in qemu_mutex_unlock_impl (mutex=mutex@entry=0x5555568002f0, file=file@entry=0x555555d378df "util/async.c", line=line@entry=507) at util/qemu-thread-posix.c:97
#4 0x0000555555b92f55 in aio_context_release (ctx=ctx@entry=0x555556800290) at util/async.c:507
#5 0x0000555555b05cf8 in bdrv_prwv_co (child=child@entry=0x7fffc80012f0, offset=offset@entry=131072, qiov=qiov@entry=0x7fffffffd4f0, is_write=is_write@entry=true, flags=flags@entry=0)
at block/io.c:833
#6 0x0000555555b060a9 in bdrv_pwritev (qiov=0x7fffffffd4f0, offset=131072, child=0x7fffc80012f0) at block/io.c:990
#7 0x0000555555b060a9 in bdrv_pwrite (child=0x7fffc80012f0, offset=131072, buf=<optimized out>, bytes=<optimized out>) at block/io.c:990
#8 0x0000555555ae172b in qcow2_cache_entry_flush (bs=bs@entry=0x555556810680, c=c@entry=0x5555568cc740, i=i@entry=0) at block/qcow2-cache.c:51
#9 0x0000555555ae18dd in qcow2_cache_write (bs=bs@entry=0x555556810680, c=0x5555568cc740) at block/qcow2-cache.c:248
#10 0x0000555555ae15de in qcow2_cache_flush (bs=0x555556810680, c=<optimized out>) at block/qcow2-cache.c:259
#11 0x0000555555ae16b1 in qcow2_cache_flush_dependency (c=0x5555568a1700, c=0x5555568a1700, bs=0x555556810680) at block/qcow2-cache.c:194
#12 0x0000555555ae16b1 in qcow2_cache_entry_flush (bs=bs@entry=0x555556810680, c=c@entry=0x5555568a1700, i=i@entry=0) at block/qcow2-cache.c:194
#13 0x0000555555ae18dd in qcow2_cache_write (bs=bs@entry=0x555556810680, c=0x5555568a1700) at block/qcow2-cache.c:248
#14 0x0000555555ae15de in qcow2_cache_flush (bs=bs@entry=0x555556810680, c=<optimized out>) at block/qcow2-cache.c:259
#15 0x0000555555ad242c in qcow2_inactivate (bs=bs@entry=0x555556810680) at block/qcow2.c:2124
#16 0x0000555555ad2590 in qcow2_close (bs=0x555556810680) at block/qcow2.c:2153
#17 0x0000555555ab0c62 in bdrv_close (bs=0x555556810680) at block.c:3358
#18 0x0000555555ab0c62 in bdrv_delete (bs=0x555556810680) at block.c:3542
#19 0x0000555555ab0c62 in bdrv_unref (bs=0x555556810680) at block.c:4598
#20 0x0000555555af4d72 in blk_remove_bs (blk=blk@entry=0x5555568103d0) at block/block-backend.c:785
#21 0x0000555555af4dbb in blk_remove_all_bs () at block/block-backend.c:483
#22 0x0000555555aae02f in bdrv_close_all () at block.c:3412
#23 0x00005555557f9796 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4776
The reproducer I used is a qcow2 image on gluster volume, where the
virtual disk size (4 GB) is larger than the gluster volume size (64M),
so we can easily trigger an ENOSPC. This backend is assigned to a
virtio-blk device using an iothread, and then from the guest a
'dd if=/dev/zero of=/dev/vda bs=1G count=1' causes the VM to stop
because of an I/O error. qemu_gluster_co_flush_to_disk() sets
bs->drv = NULL on error, so when virtio-blk stops the dataplane, the
block nodes stay in the iothread AioContext. A 'quit' monitor command
issued from this paused state crashes the process.
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1631227
Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
---
block.c | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
diff --git a/block.c b/block.c
index 16615bc876..9ae5c0ed2f 100644
--- a/block.c
+++ b/block.c
@@ -5672,10 +5672,6 @@ void bdrv_detach_aio_context(BlockDriverState *bs)
BdrvAioNotifier *baf, *baf_tmp;
BdrvChild *child;
- if (!bs->drv) {
- return;
- }
-
assert(!bs->walking_aio_notifiers);
bs->walking_aio_notifiers = true;
QLIST_FOREACH_SAFE(baf, &bs->aio_notifiers, list, baf_tmp) {
@@ -5690,7 +5686,7 @@ void bdrv_detach_aio_context(BlockDriverState *bs)
*/
bs->walking_aio_notifiers = false;
- if (bs->drv->bdrv_detach_aio_context) {
+ if (bs->drv && bs->drv->bdrv_detach_aio_context) {
bs->drv->bdrv_detach_aio_context(bs);
}
QLIST_FOREACH(child, &bs->children, next) {
@@ -5709,10 +5705,6 @@ void bdrv_attach_aio_context(BlockDriverState *bs,
BdrvAioNotifier *ban, *ban_tmp;
BdrvChild *child;
- if (!bs->drv) {
- return;
- }
-
if (bs->quiesce_counter) {
aio_disable_external(new_context);
}
@@ -5722,7 +5714,7 @@ void bdrv_attach_aio_context(BlockDriverState *bs,
QLIST_FOREACH(child, &bs->children, next) {
bdrv_attach_aio_context(child->bs, new_context);
}
- if (bs->drv->bdrv_attach_aio_context) {
+ if (bs->drv && bs->drv->bdrv_attach_aio_context) {
bs->drv->bdrv_attach_aio_context(bs, new_context);
}
--
2.20.1
next prev parent reply other threads:[~2019-04-30 15:48 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-30 15:42 [Qemu-devel] [PULL 00/27] Block layer patches Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 01/27] tests/qemu-iotests: Fix output of qemu-io related tests Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf [this message]
2019-04-30 15:42 ` [Qemu-devel] [PULL 02/27] block: Fix AioContext switch for bs->drv == NULL Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 03/27] vpc: unlock Coroutine lock to make IO submit Concurrently Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 04/27] vmdk: Set vmdk parent backing_format to vmdk Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 05/27] block/vhdx: Remove redundant IEC binary prefixes definition Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 06/27] block/vhdx: Use IEC binary prefixes for size constants Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 07/27] cutils: Fix size_to_str() on 32-bit platforms Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 08/27] qemu-img: Saner printing of large file sizes Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 09/27] qcow2: Avoid COW during metadata preallocation Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 10/27] qcow2: Add errp to preallocate_co() Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 11/27] qcow2: Fix full preallocation with external data file Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 12/27] iotests: Perform the correct test in 082 Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 13/27] qemu-img: Make create hint at protocol options Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 14/27] qcow2: Fix qcow2_make_empty() with external data file Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 15/27] qcow2: Fix error handling in the compression code Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 16/27] block: introduce byte-based io helpers Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 17/27] block/qcow2: use buffer-based io Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 18/27] block/qcow: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 19/27] block/qed: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 20/27] block/parallels: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 21/27] block/backup: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 22/27] block/commit: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 23/27] block/stream: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 24/27] qemu-img: " Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 25/27] commit: Make base read-only if there is an early failure Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 26/27] iotests: Check that images are in read-only mode after block-commit Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 15:42 ` [Qemu-devel] [PULL 27/27] block/qed: add missed coroutine_fn markers Kevin Wolf
2019-04-30 15:42 ` Kevin Wolf
2019-04-30 16:57 ` [Qemu-devel] [PULL 00/27] Block layer patches Peter Maydell
2019-04-30 16:57 ` Peter Maydell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190430154244.30083-3-kwolf@redhat.com \
--to=kwolf@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).