From: Yang Erkun <yangerkun@huawei.com>
To: axboe@kernel.dk, josef@toxicpanda.com, xiubli@redhat.com,
prasanna.kalever@redhat.com, ming.lei@redhat.com, yukuai@fygo.io,
nilay@linux.ibm.com
Cc: linux-block@vger.kernel.org, nbd@other.debian.org,
yangerkun@huawei.com, wangkefeng.wang@huawei.com,
leijitang@huawei.com, yi.zhang@huawei.com,
chengzhihao1@huawei.com, houtao1@huawei.com, wozizhi@huawei.com
Subject: [RFC PATCH 4/5] nbd: hold config_lock while updating nr_hw_queues
Date: Sat, 7 Nov 2026 14:46:28 +0800 [thread overview]
Message-ID: <20261107064629.4093489-5-yangerkun@huawei.com> (raw)
In-Reply-To: <20261107064629.4093489-1-yangerkun@huawei.com>
Link [1] reports a possible deadlock, and the discussion there shows that
it is impossible. In order to silence the warning, commit 8b428f42f3ed
("nbd: fix lockdep deadlock warning") released nbd->config_lock before
blk_mq_update_nr_hw_queues(), and then relocked it.
But releasing the lock has a real cost. While nbd_start_device() runs
without config_lock, a concurrent disconnect can shut down the sockets,
and setup continues and reports success with a config that is already
dead. This patch fixes it in another way, by using mutex_lock_nested().
Only nbd_genl_connect() and the NBD_DO_IT path in nbd_ioctl() reach
nbd_start_device(), so only these take the separate subclass; the other
ioctls keep the default one and stay fully checked by lockdep.
Fixes: 8b428f42f3ed ("nbd: fix lockdep deadlock warning")
Assisted-by: LLM
Link: https://lore.kernel.org/all/6855034f.a00a0220.137b3.0031.GAE@google.com/T/#u
Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
drivers/block/nbd.c | 44 +++++++++++++++++++++++++++++++-------------
1 file changed, 31 insertions(+), 13 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 8c2cb312f209..cf2a47fc572d 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1510,6 +1510,31 @@ static void nbd_config_put(struct nbd_device *nbd)
}
}
+/*
+ * nbd_start_device() calls blk_mq_update_nr_hw_queues() with config_lock
+ * held. del_gendisk() takes the tag set's update_nr_hwq_lock and then
+ * disk->open_mutex, while ->open()/->release() run with disk->open_mutex
+ * held and take config_lock (nbd_open(), and nbd_release() through
+ * nbd_disconnect_and_put() or nbd_config_put()), so lockdep reports the
+ * class-level cycle
+ *
+ * config_lock -> update_nr_hwq_lock -> open_mutex -> config_lock
+ *
+ * as a possible deadlock. It cannot happen: nbd_start_device() is called
+ * with an nbd reference held, and the open/release side takes config_lock
+ * while the open file still holds one, so nbd->refs cannot reach zero and
+ * nbd_dev_remove_work() -> del_gendisk() cannot run concurrently. In
+ * addition, nbd_config_put() drops config_lock before nbd_put(), so the
+ * teardown path never calls del_gendisk() with config_lock held.
+ *
+ * Only the acquisitions that can reach nbd_start_device() are annotated:
+ * nbd_genl_connect(), and nbd_ioctl() for NBD_DO_IT (which ends up in
+ * nbd_start_device_ioctl()). No other ioctl reaches it, so they keep the
+ * default subclass and stay fully checked. This only changes the lockdep
+ * annotation, not the locking itself.
+ */
+#define NBD_CONFIG_LOCK_SETUP 1
+
static int nbd_start_device(struct nbd_device *nbd)
{
struct nbd_config *config = nbd->config;
@@ -1526,17 +1551,7 @@ static int nbd_start_device(struct nbd_device *nbd)
return -EINVAL;
}
-retry:
- mutex_unlock(&nbd->config_lock);
- blk_mq_update_nr_hw_queues(&nbd->tag_set, num_connections);
- mutex_lock(&nbd->config_lock);
-
- /* if another code path updated nr_hw_queues, retry until succeed */
- if (num_connections != config->num_connections) {
- num_connections = config->num_connections;
- goto retry;
- }
-
+ blk_mq_update_nr_hw_queues(&nbd->tag_set, config->num_connections);
nbd->pid = task_pid_nr(current);
nbd_parse_flags(nbd);
@@ -1695,7 +1710,10 @@ static int nbd_ioctl(struct block_device *bdev, blk_mode_t mode,
if (_IOC_TYPE(cmd) != 0xab)
return -EINVAL;
- mutex_lock(&nbd->config_lock);
+ if (cmd == NBD_DO_IT)
+ mutex_lock_nested(&nbd->config_lock, NBD_CONFIG_LOCK_SETUP);
+ else
+ mutex_lock(&nbd->config_lock);
/* Don't allow ioctl operations on a nbd device that was created with
* netlink, unless it's DISCONNECT or CLEAR_SOCK, which are fine.
@@ -2233,7 +2251,7 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
}
}
- mutex_lock(&nbd->config_lock);
+ mutex_lock_nested(&nbd->config_lock, NBD_CONFIG_LOCK_SETUP);
if (refcount_read(&nbd->config_refs)) {
mutex_unlock(&nbd->config_lock);
nbd_put(nbd);
--
2.52.0
next prev parent reply other threads:[~2026-11-07 6:54 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-07 6:46 [RFC PATCH 0/5] nbd: allow parallel genetlink operations Yang Erkun
2026-11-07 6:46 ` [RFC PATCH 1/5] nbd: validate the backend identifier under config_lock Yang Erkun
2026-11-07 6:46 ` [RFC PATCH 2/5] nbd: decrement nbd_total_devices when a device is removed Yang Erkun
2026-11-07 6:46 ` [RFC PATCH 3/5] nbd: publish devices atomically Yang Erkun
2026-11-07 6:46 ` Yang Erkun [this message]
2026-11-07 6:46 ` [RFC PATCH 5/5] nbd: allow genetlink operations to run in parallel Yang Erkun
2026-11-17 4:36 ` [RFC PATCH 0/5] nbd: allow parallel genetlink operations yangerkun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261107064629.4093489-5-yangerkun@huawei.com \
--to=yangerkun@huawei.com \
--cc=axboe@kernel.dk \
--cc=chengzhihao1@huawei.com \
--cc=houtao1@huawei.com \
--cc=josef@toxicpanda.com \
--cc=leijitang@huawei.com \
--cc=linux-block@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=nbd@other.debian.org \
--cc=nilay@linux.ibm.com \
--cc=prasanna.kalever@redhat.com \
--cc=wangkefeng.wang@huawei.com \
--cc=wozizhi@huawei.com \
--cc=xiubli@redhat.com \
--cc=yi.zhang@huawei.com \
--cc=yukuai@fygo.io \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox