From: Hanna Czenczek <hreitz@redhat.com>
To: Kevin Wolf <kwolf@redhat.com>, qemu-block@nongnu.org
Cc: stefanha@redhat.com, qemu-devel@nongnu.org
Subject: Re: [PULL 11/33] scsi: only access SCSIDevice->requests from one thread
Date: Tue, 23 Jan 2024 17:40:39 +0100 [thread overview]
Message-ID: <73e752b2-a037-4b10-a903-56fa6ad75c6e@redhat.com> (raw)
In-Reply-To: <20231221212339.164439-12-kwolf@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 3214 bytes --]
On 21.12.23 22:23, Kevin Wolf wrote:
> From: Stefan Hajnoczi<stefanha@redhat.com>
>
> Stop depending on the AioContext lock and instead access
> SCSIDevice->requests from only one thread at a time:
> - When the VM is running only the BlockBackend's AioContext may access
> the requests list.
> - When the VM is stopped only the main loop may access the requests
> list.
>
> These constraints protect the requests list without the need for locking
> in the I/O code path.
>
> Note that multiple IOThreads are not supported yet because the code
> assumes all SCSIRequests are executed from a single AioContext. Leave
> that as future work.
>
> Signed-off-by: Stefan Hajnoczi<stefanha@redhat.com>
> Reviewed-by: Eric Blake<eblake@redhat.com>
> Message-ID:<20231204164259.1515217-2-stefanha@redhat.com>
> Signed-off-by: Kevin Wolf<kwolf@redhat.com>
> ---
> include/hw/scsi/scsi.h | 7 +-
> hw/scsi/scsi-bus.c | 181 ++++++++++++++++++++++++++++-------------
> 2 files changed, 131 insertions(+), 57 deletions(-)
My reproducer for https://issues.redhat.com/browse/RHEL-3934 now breaks
more often because of this commit than because of the original bug, i.e.
when repeatedly hot-plugging and unplugging a virtio-scsi and a scsi-hd
device, this tends to happen when unplugging the scsi-hd:
{"execute":"device_del","arguments":{"id":"stg0"}}
{"return": {}}
qemu-system-x86_64: ../block/block-backend.c:2429: blk_get_aio_context:
Assertion `ctx == blk->ctx' failed.
(gdb) bt
#0 0x00007f32a668d83c in () at /usr/lib/libc.so.6
#1 0x00007f32a663d668 in raise () at /usr/lib/libc.so.6
#2 0x00007f32a66254b8 in abort () at /usr/lib/libc.so.6
#3 0x00007f32a66253dc in () at /usr/lib/libc.so.6
#4 0x00007f32a6635d26 in () at /usr/lib/libc.so.6
#5 0x0000556e6b4880a4 in blk_get_aio_context (blk=0x556e6e89ccf0) at
../block/block-backend.c:2429
#6 blk_get_aio_context (blk=0x556e6e89ccf0) at
../block/block-backend.c:2417
#7 0x0000556e6b112d87 in scsi_device_for_each_req_async_bh
(opaque=0x556e6e2c6d10) at ../hw/scsi/scsi-bus.c:128
#8 0x0000556e6b5d1966 in aio_bh_poll (ctx=ctx@entry=0x556e6d8aa290) at
../util/async.c:218
#9 0x0000556e6b5bb16a in aio_poll (ctx=0x556e6d8aa290,
blocking=blocking@entry=true) at ../util/aio-posix.c:722
#10 0x0000556e6b4564b6 in iothread_run
(opaque=opaque@entry=0x556e6d89d920) at ../iothread.c:63
#11 0x0000556e6b5bde58 in qemu_thread_start (args=0x556e6d8aa9b0) at
../util/qemu-thread-posix.c:541
#12 0x00007f32a668b9eb in () at /usr/lib/libc.so.6
#13 0x00007f32a670f7cc in () at /usr/lib/libc.so.6
I don’t know anything about the problem yet, but as usual, I like
speculation and discovering how wrong I was later on, so one thing I
came across that’s funny about virtio-scsi is that requests can happen
even while a disk is being attached or detached. That is, Linux seems
to probe all LUNs when a new virtio-scsi device is being attached, and
it won’t stop just because a disk is being attached or removed. So
maybe that’s part of the problem, that we get a request while the BB is
being detached, and temporarily in an inconsistent state (BDS context
differs from BB context).
I’ll look more into it.
Hanna
[-- Attachment #2: Type: text/html, Size: 4297 bytes --]
next prev parent reply other threads:[~2024-01-23 16:41 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-21 21:23 [PULL 00/33] Block layer patches Kevin Wolf
2023-12-21 21:23 ` [PULL 01/33] nbd/server: avoid per-NBDRequest nbd_client_get/put() Kevin Wolf
2023-12-21 21:23 ` [PULL 02/33] nbd/server: only traverse NBDExport->clients from main loop thread Kevin Wolf
2023-12-21 21:23 ` [PULL 03/33] nbd/server: introduce NBDClient->lock to protect fields Kevin Wolf
2023-12-21 21:23 ` [PULL 04/33] block/file-posix: set up Linux AIO and io_uring in the current thread Kevin Wolf
2023-12-21 21:23 ` [PULL 05/33] virtio-blk: add lock to protect s->rq Kevin Wolf
2023-12-21 21:23 ` [PULL 06/33] virtio-blk: don't lock AioContext in the completion code path Kevin Wolf
2023-12-21 21:23 ` [PULL 07/33] virtio-blk: don't lock AioContext in the submission " Kevin Wolf
2023-12-21 21:23 ` [PULL 08/33] block: Fix crash when loading snapshot on inactive node Kevin Wolf
2023-12-21 21:23 ` [PULL 09/33] vl: Improve error message for conflicting -incoming and -loadvm Kevin Wolf
2023-12-21 21:23 ` [PULL 10/33] iotests: Basic tests for internal snapshots Kevin Wolf
2023-12-21 21:23 ` [PULL 11/33] scsi: only access SCSIDevice->requests from one thread Kevin Wolf
2024-01-23 16:40 ` Hanna Czenczek [this message]
2024-01-23 17:10 ` Kevin Wolf
2024-01-23 17:23 ` Hanna Czenczek
2024-01-24 12:12 ` Hanna Czenczek
2024-01-24 21:53 ` Stefan Hajnoczi
2024-01-25 9:06 ` Hanna Czenczek
2024-01-25 13:25 ` Stefan Hajnoczi
2024-01-25 17:32 ` Hanna Czenczek
2024-01-26 13:18 ` Kevin Wolf
2024-01-26 15:24 ` Hanna Czenczek
2024-01-31 20:35 ` Stefan Hajnoczi
2024-02-01 14:10 ` Hanna Czenczek
2024-02-01 14:28 ` Stefan Hajnoczi
2024-02-01 15:25 ` Hanna Czenczek
2024-02-01 15:49 ` Hanna Czenczek
2024-02-02 12:32 ` Hanna Czenczek
2024-02-06 19:32 ` Stefan Hajnoczi
2024-01-29 16:30 ` Hanna Czenczek
2024-01-31 10:17 ` Kevin Wolf
2024-02-01 9:43 ` Hanna Czenczek
2024-02-01 10:21 ` Kevin Wolf
2024-02-01 10:35 ` Hanna Czenczek
2024-01-23 17:21 ` Hanna Czenczek
2023-12-21 21:23 ` [PULL 12/33] virtio-scsi: don't lock AioContext around virtio_queue_aio_attach_host_notifier() Kevin Wolf
2023-12-21 21:23 ` [PULL 13/33] scsi: don't lock AioContext in I/O code path Kevin Wolf
2023-12-21 21:23 ` [PULL 14/33] dma-helpers: don't lock AioContext in dma_blk_cb() Kevin Wolf
2023-12-21 21:23 ` [PULL 15/33] virtio-scsi: replace AioContext lock with tmf_bh_lock Kevin Wolf
2023-12-21 21:23 ` [PULL 16/33] scsi: assert that callbacks run in the correct AioContext Kevin Wolf
2023-12-21 21:23 ` [PULL 17/33] tests: remove aio_context_acquire() tests Kevin Wolf
2023-12-21 21:23 ` [PULL 18/33] aio: make aio_context_acquire()/aio_context_release() a no-op Kevin Wolf
2023-12-21 21:23 ` [PULL 19/33] graph-lock: remove AioContext locking Kevin Wolf
2023-12-21 21:23 ` [PULL 20/33] block: " Kevin Wolf
2023-12-21 21:23 ` [PULL 21/33] block: remove bdrv_co_lock() Kevin Wolf
2023-12-21 21:23 ` [PULL 22/33] scsi: remove AioContext locking Kevin Wolf
2023-12-21 21:23 ` [PULL 23/33] aio-wait: draw equivalence between AIO_WAIT_WHILE() and AIO_WAIT_WHILE_UNLOCKED() Kevin Wolf
2023-12-21 21:23 ` [PULL 24/33] aio: remove aio_context_acquire()/aio_context_release() API Kevin Wolf
2023-12-21 21:23 ` [PULL 25/33] docs: remove AioContext lock from IOThread docs Kevin Wolf
2023-12-21 21:23 ` [PULL 26/33] scsi: remove outdated AioContext lock comment Kevin Wolf
2023-12-21 21:23 ` [PULL 27/33] job: remove outdated AioContext locking comments Kevin Wolf
2023-12-21 21:23 ` [PULL 28/33] block: " Kevin Wolf
2023-12-21 21:23 ` [PULL 29/33] block-coroutine-wrapper: use qemu_get_current_aio_context() Kevin Wolf
2023-12-21 21:23 ` [PULL 30/33] string-output-visitor: show structs as "<omitted>" Kevin Wolf
2023-12-21 21:23 ` [PULL 31/33] qdev-properties: alias all object class properties Kevin Wolf
2023-12-21 21:23 ` [PULL 32/33] qdev: add IOThreadVirtQueueMappingList property type Kevin Wolf
2023-12-21 21:23 ` [PULL 33/33] virtio-blk: add iothread-vq-mapping parameter Kevin Wolf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=73e752b2-a037-4b10-a903-56fa6ad75c6e@redhat.com \
--to=hreitz@redhat.com \
--cc=kwolf@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=stefanha@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).