From: "Daniel P. Berrangé" <berrange@redhat.com>
To: Eric Blake <eblake@redhat.com>
Cc: qemu-devel@nongnu.org, qemu-block@nongnu.org, kwolf@redhat.com
Subject: Re: [PATCH v2 10/12] qio: Add QIONetListener API for using AioContext
Date: Tue, 11 Nov 2025 14:18:27 +0000 [thread overview]
Message-ID: <aRNFs0oVp9-ZItfU@redhat.com> (raw)
In-Reply-To: <20251108230525.3169174-24-eblake@redhat.com>
On Sat, Nov 08, 2025 at 04:59:31PM -0600, Eric Blake wrote:
> The user "John Doe" reported a deadlock when attempting to use
> qemu-storage-daemon to serve both a base file over NBD, and a qcow2
> file with that NBD export as its backing file, from the same process,
> even though it worked just fine when there were two q-s-d processes.
> The bulk of the NBD server code properly uses coroutines to make
> progress in an event-driven manner, but the code for spawning a new
> coroutine at the point when listen(2) detects a new client was
> hard-coded to use the global GMainContext; in other words, the
> callback that triggers nbd_client_new to let the server start the
> negotiation sequence with the client requires the main loop to be
> making progress. However, the code for bdrv_open of a qcow2 image
> with an NBD backing file uses an AIO_WAIT_WHILE nested event loop to
> ensure that the entire qcow2 backing chain is either fully loaded or
> rejected, without any side effects from the main loop causing unwanted
> changes to the disk being loaded (in short, an AioContext represents
> the set of actions that are known to be safe while handling block
> layer I/O, while excluding any other pending actions in the global
> main loop with potentially larger risk of unwanted side effects).
>
> This creates a classic case of deadlock: the server can't progress to
> the point of accept(2)ing the client to write to the NBD socket
> because the main loop is being starved until the AIO_WAIT_WHILE
> completes the bdrv_open, but the AIO_WAIT_WHILE can't progress because
> it is blocked on the client coroutine stuck in a read() of the
> expected magic number from the server side of the socket.
>
> This patch adds a new API to allow clients to opt in to listening via
> an AioContext rather than a GMainContext. This will allow NBD to fix
> the deadlock by performing all actions during bdrv_open in the main
> loop AioContext.
>
> An upcoming patch will then add a unit test (kept separate to make it
> easier to rearrange the series to demonstrate the deadlock without
> this patch).
>
> Fixes: https://gitlab.com/qemu-project/qemu/-/issues/3169
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v2: Retitle and add new API rather than changing semantics of
> existing qio_net_listener_set_client_func; use qio accessor rather
> than direct access to the sioc fd and a lower-level aio call
> ---
> include/io/net-listener.h | 16 ++++++++++++++++
> io/net-listener.c | 36 +++++++++++++++++++++++++++++++++---
> 2 files changed, 49 insertions(+), 3 deletions(-)
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
next prev parent reply other threads:[~2025-11-11 14:20 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-08 22:59 [PATCH v2 00/12] Fix deadlock with bdrv_open of self-served NBD Eric Blake
2025-11-08 22:59 ` [PATCH v2 01/12] iotests: Drop execute permissions on vvfat.out Eric Blake
2025-11-10 15:57 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 02/12] qio: Add trace points to net_listener Eric Blake
2025-11-10 15:58 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 03/12] qio: Unwatch before notify in QIONetListener Eric Blake
2025-11-10 16:00 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 04/12] qio: Remember context of qio_net_listener_set_client_func_full Eric Blake
2025-11-10 16:08 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 05/12] qio: Minor optimization when callback function is unchanged Eric Blake
2025-11-10 16:09 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 06/12] qio: Factor out helpers qio_net_listener_[un]watch Eric Blake
2025-11-10 16:14 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 07/12] qio: Hoist ref of listener outside loop Eric Blake
2025-11-11 14:43 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 08/12] qio: Provide accessor around QIONetListener->sioc Eric Blake
2025-11-10 18:31 ` Eric Blake
2025-11-11 14:15 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 09/12] qio: Prepare NetListener to use AioContext Eric Blake
2025-11-11 14:17 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 10/12] qio: Add QIONetListener API for using AioContext Eric Blake
2025-11-11 14:18 ` Daniel P. Berrangé [this message]
2025-11-08 22:59 ` [PATCH v2 11/12] nbd: Avoid deadlock in client connecting to same-process server Eric Blake
2025-11-11 14:20 ` Daniel P. Berrangé
2025-11-08 22:59 ` [PATCH v2 12/12] iotests: Add coverage of recent NBD qio deadlock fix Eric Blake
2025-11-10 16:19 ` Daniel P. Berrangé
2025-11-12 6:35 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aRNFs0oVp9-ZItfU@redhat.com \
--to=berrange@redhat.com \
--cc=eblake@redhat.com \
--cc=kwolf@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.