Linux block layer
 help / color / mirror / Atom feed
From: Yang Erkun <yangerkun@huawei.com>
To: axboe@kernel.dk, josef@toxicpanda.com, xiubli@redhat.com,
	prasanna.kalever@redhat.com, ming.lei@redhat.com, yukuai@fygo.io,
	nilay@linux.ibm.com
Cc: linux-block@vger.kernel.org, nbd@other.debian.org,
	yangerkun@huawei.com, wangkefeng.wang@huawei.com,
	leijitang@huawei.com, yi.zhang@huawei.com,
	chengzhihao1@huawei.com, houtao1@huawei.com, wozizhi@huawei.com
Subject: [RFC PATCH 4/5] nbd: hold config_lock while updating nr_hw_queues
Date: Sat,  7 Nov 2026 14:46:28 +0800	[thread overview]
Message-ID: <20261107064629.4093489-5-yangerkun@huawei.com> (raw)
In-Reply-To: <20261107064629.4093489-1-yangerkun@huawei.com>

Link [1] reports a possible deadlock, and the discussion there shows that
it is impossible.  In order to silence the warning, commit 8b428f42f3ed
("nbd: fix lockdep deadlock warning") released nbd->config_lock before
blk_mq_update_nr_hw_queues(), and then relocked it.

But releasing the lock has a real cost.  While nbd_start_device() runs
without config_lock, a concurrent disconnect can shut down the sockets,
and setup continues and reports success with a config that is already
dead.  This patch fixes it in another way, by using mutex_lock_nested().
Only nbd_genl_connect() and the NBD_DO_IT path in nbd_ioctl() reach
nbd_start_device(), so only these take the separate subclass; the other
ioctls keep the default one and stay fully checked by lockdep.

Fixes: 8b428f42f3ed ("nbd: fix lockdep deadlock warning")
Assisted-by: LLM
Link: https://lore.kernel.org/all/6855034f.a00a0220.137b3.0031.GAE@google.com/T/#u
Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 44 +++++++++++++++++++++++++++++++-------------
 1 file changed, 31 insertions(+), 13 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 8c2cb312f209..cf2a47fc572d 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1510,6 +1510,31 @@ static void nbd_config_put(struct nbd_device *nbd)
 	}
 }
 
+/*
+ * nbd_start_device() calls blk_mq_update_nr_hw_queues() with config_lock
+ * held.  del_gendisk() takes the tag set's update_nr_hwq_lock and then
+ * disk->open_mutex, while ->open()/->release() run with disk->open_mutex
+ * held and take config_lock (nbd_open(), and nbd_release() through
+ * nbd_disconnect_and_put() or nbd_config_put()), so lockdep reports the
+ * class-level cycle
+ *
+ *   config_lock -> update_nr_hwq_lock -> open_mutex -> config_lock
+ *
+ * as a possible deadlock.  It cannot happen: nbd_start_device() is called
+ * with an nbd reference held, and the open/release side takes config_lock
+ * while the open file still holds one, so nbd->refs cannot reach zero and
+ * nbd_dev_remove_work() -> del_gendisk() cannot run concurrently.  In
+ * addition, nbd_config_put() drops config_lock before nbd_put(), so the
+ * teardown path never calls del_gendisk() with config_lock held.
+ *
+ * Only the acquisitions that can reach nbd_start_device() are annotated:
+ * nbd_genl_connect(), and nbd_ioctl() for NBD_DO_IT (which ends up in
+ * nbd_start_device_ioctl()).  No other ioctl reaches it, so they keep the
+ * default subclass and stay fully checked.  This only changes the lockdep
+ * annotation, not the locking itself.
+ */
+#define NBD_CONFIG_LOCK_SETUP	1
+
 static int nbd_start_device(struct nbd_device *nbd)
 {
 	struct nbd_config *config = nbd->config;
@@ -1526,17 +1551,7 @@ static int nbd_start_device(struct nbd_device *nbd)
 		return -EINVAL;
 	}
 
-retry:
-	mutex_unlock(&nbd->config_lock);
-	blk_mq_update_nr_hw_queues(&nbd->tag_set, num_connections);
-	mutex_lock(&nbd->config_lock);
-
-	/* if another code path updated nr_hw_queues, retry until succeed */
-	if (num_connections != config->num_connections) {
-		num_connections = config->num_connections;
-		goto retry;
-	}
-
+	blk_mq_update_nr_hw_queues(&nbd->tag_set, config->num_connections);
 	nbd->pid = task_pid_nr(current);
 
 	nbd_parse_flags(nbd);
@@ -1695,7 +1710,10 @@ static int nbd_ioctl(struct block_device *bdev, blk_mode_t mode,
 	if (_IOC_TYPE(cmd) != 0xab)
 		return -EINVAL;
 
-	mutex_lock(&nbd->config_lock);
+	if (cmd == NBD_DO_IT)
+		mutex_lock_nested(&nbd->config_lock, NBD_CONFIG_LOCK_SETUP);
+	else
+		mutex_lock(&nbd->config_lock);
 
 	/* Don't allow ioctl operations on a nbd device that was created with
 	 * netlink, unless it's DISCONNECT or CLEAR_SOCK, which are fine.
@@ -2233,7 +2251,7 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
 		}
 	}
 
-	mutex_lock(&nbd->config_lock);
+	mutex_lock_nested(&nbd->config_lock, NBD_CONFIG_LOCK_SETUP);
 	if (refcount_read(&nbd->config_refs)) {
 		mutex_unlock(&nbd->config_lock);
 		nbd_put(nbd);
-- 
2.52.0


  parent reply	other threads:[~2026-11-07  6:54 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-07  6:46 [RFC PATCH 0/5] nbd: allow parallel genetlink operations Yang Erkun
2026-11-07  6:46 ` [RFC PATCH 1/5] nbd: validate the backend identifier under config_lock Yang Erkun
2026-11-07  6:46 ` [RFC PATCH 2/5] nbd: decrement nbd_total_devices when a device is removed Yang Erkun
2026-11-07  6:46 ` [RFC PATCH 3/5] nbd: publish devices atomically Yang Erkun
2026-11-07  6:46 ` Yang Erkun [this message]
2026-11-07  6:46 ` [RFC PATCH 5/5] nbd: allow genetlink operations to run in parallel Yang Erkun
2026-11-17  4:36 ` [RFC PATCH 0/5] nbd: allow parallel genetlink operations yangerkun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261107064629.4093489-5-yangerkun@huawei.com \
    --to=yangerkun@huawei.com \
    --cc=axboe@kernel.dk \
    --cc=chengzhihao1@huawei.com \
    --cc=houtao1@huawei.com \
    --cc=josef@toxicpanda.com \
    --cc=leijitang@huawei.com \
    --cc=linux-block@vger.kernel.org \
    --cc=ming.lei@redhat.com \
    --cc=nbd@other.debian.org \
    --cc=nilay@linux.ibm.com \
    --cc=prasanna.kalever@redhat.com \
    --cc=wangkefeng.wang@huawei.com \
    --cc=wozizhi@huawei.com \
    --cc=xiubli@redhat.com \
    --cc=yi.zhang@huawei.com \
    --cc=yukuai@fygo.io \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox