qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Eric Blake <eblake@redhat.com>
To: qemu-devel@nongnu.org
Cc: qemu-block@nongnu.org, berrange@redhat.com, kwolf@redhat.com
Subject: [PATCH 0/8] Fix deadlock with bdrv_open of self-served NBD
Date: Mon,  3 Nov 2025 14:10:51 -0600	[thread overview]
Message-ID: <20251103202849.3687643-10-eblake@redhat.com> (raw)

https://gitlab.com/qemu-project/qemu/-/issues/3169 was an excellent
bug report of a deadlock scenario when qemu opens both the server and
client end of an NBD intermediary in a qcow2 backing chain.  It took
me quite a bit of time to understand why the deadlock was even
happening, and then some mental gymnastics on how best to break the
deadlock.  Ideally - we'd get rid of all nested event loops, and have
the main loop do EVERYTHING event-driven.  But that's a much bigger
code change to the entire code base of QEMU, so I went with the next
best thing of keeping the AioContext nested loop but ensuring that
accepting NBD clients is now properly part of the AioContext rather
than gated by only the main loop progress.

I have not researched how long the deadlock has been present, to
determine if it has ever worked before earlier refactorings when we
started using AioContext more heavily, or if it has always been broken
until now to try and connect QEMU as a client to a self-served NBD
server.  But either way, I think this series should be part of the
10.2 release; I'm awfully close to soft freeze, but I think this
counts as a bug fix worth having even if it doesn't make it in before
-rc1.

Eric Blake (8):
  qio: Add trace points to net_listener
  qio: Minor optimization when callback function is unchanged
  qio: Remember context of qio_net_listener_set_client_func_full
  qio: Factor out helpers qio_net_listener_[un]watch
  qio: Let listening sockets remember their owning QIONetListener
  qio: Hoist ref of listener outside loop
  qio: Use AioContext for default-context QIONetListener
  iotests: Add coverage of recent NBD qio deadlock fix

 include/io/channel-socket.h                   |   1 +
 include/io/net-listener.h                     |   1 +
 io/channel-socket.c                           |   1 +
 io/net-listener.c                             | 136 +++++++++++-------
 io/trace-events                               |   5 +
 tests/qemu-iotests/tests/nbd-in-qcow2-chain   |  84 +++++++++++
 .../qemu-iotests/tests/nbd-in-qcow2-chain.out |  56 ++++++++
 tests/qemu-iotests/tests/vvfat.out            |   0
 8 files changed, 236 insertions(+), 48 deletions(-)
 create mode 100755 tests/qemu-iotests/tests/nbd-in-qcow2-chain
 create mode 100644 tests/qemu-iotests/tests/nbd-in-qcow2-chain.out
 mode change 100755 => 100644 tests/qemu-iotests/tests/vvfat.out

-- 
2.51.1



             reply	other threads:[~2025-11-03 20:30 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-03 20:10 Eric Blake [this message]
2025-11-03 20:10 ` [PATCH 1/8] qio: Add trace points to net_listener Eric Blake
2025-11-04 10:43   ` Daniel P. Berrangé
2025-11-04 11:08   ` Kevin Wolf
2025-11-05 17:18     ` Eric Blake
2025-11-06 12:20       ` Kevin Wolf
2025-11-03 20:10 ` [PATCH 2/8] qio: Minor optimization when callback function is unchanged Eric Blake
2025-11-04 10:44   ` Daniel P. Berrangé
2025-11-04 11:13   ` Kevin Wolf
2025-11-05 17:23     ` Eric Blake
2025-11-03 20:10 ` [PATCH 3/8] qio: Remember context of qio_net_listener_set_client_func_full Eric Blake
2025-11-04 10:50   ` Daniel P. Berrangé
2025-11-04 11:25   ` Kevin Wolf
2025-11-05 19:18     ` Eric Blake
2025-11-03 20:10 ` [PATCH 4/8] qio: Factor out helpers qio_net_listener_[un]watch Eric Blake
2025-11-04 11:03   ` Daniel P. Berrangé
2025-11-04 13:15     ` Kevin Wolf
2025-11-05 19:22       ` Eric Blake
2025-11-04 12:37   ` Kevin Wolf
2025-11-04 13:10     ` Daniel P. Berrangé
2025-11-05 19:32       ` Eric Blake
2025-11-03 20:10 ` [PATCH 5/8] qio: Let listening sockets remember their owning QIONetListener Eric Blake
2025-11-05 20:06   ` Eric Blake
2025-11-06 18:35     ` Eric Blake
2025-11-07  8:50       ` Daniel P. Berrangé
2025-11-07 13:47         ` Eric Blake
2025-11-03 20:10 ` [PATCH 6/8] qio: Hoist ref of listener outside loop Eric Blake
2025-11-04 11:13   ` Daniel P. Berrangé
2025-11-05 21:57     ` Eric Blake
2025-11-11 14:43       ` Daniel P. Berrangé
2025-11-11 15:48         ` Kevin Wolf
2025-11-11 16:07           ` Daniel P. Berrangé
2025-11-11 19:09         ` Eric Blake
2025-11-11 20:07           ` Eric Blake
2025-11-12 10:31             ` Daniel P. Berrangé
2025-11-12 10:20           ` Daniel P. Berrangé
2025-11-03 20:10 ` [PATCH 7/8] qio: Use AioContext for default-context QIONetListener Eric Blake
2025-11-04 11:37   ` Daniel P. Berrangé
2025-11-05 22:06     ` Eric Blake
2025-11-04 15:14   ` Kevin Wolf
2025-11-03 20:10 ` [PATCH 8/8] iotests: Add coverage of recent NBD qio deadlock fix Eric Blake
2025-11-04 11:38   ` Vladimir Sementsov-Ogievskiy
2025-11-05 22:10     ` Eric Blake
2025-11-06  8:20       ` Vladimir Sementsov-Ogievskiy
2025-11-06 12:26       ` Kevin Wolf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20251103202849.3687643-10-eblake@redhat.com \
    --to=eblake@redhat.com \
    --cc=berrange@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).