[PATCH v5 0/8] nbd: eliminate queue freeze/unfreeze overhead in connection setup

[PATCH v5 0/8] nbd: eliminate queue freeze/unfreeze overhead in connection setup

[Yang Erkun]

This series eliminates the blk_mq_freeze_queue()/unfreeze_queue()
overhead that currently occurs during nbd connection setup(nbd provide
ondemand and was using in AI sandbox e2b[1] to mount and attach rootfs).
Each freeze cycle will introduce a RCU grace which was significant
with multi-core system.

time nbd-client --name myexport --connections 96 127.0.0.1 1234

After this patchset:
real    0m0.090s
user    0m0.004s
sys     0m0.018s

Without this patchset:
real    0m2.195s
user    0m0.005s
sys     0m0.022s


v4->v5:
1. add patch 3 to reset write cache on disconnect
2. fix ("nbd: factor out a nbd_genl_foreach_sock")

v3->v4:
1. factor out nbd_genl_foreach_sock to cleanup duplicated code
2. rewrite some commit message to make what's going more clearly

v2->v3:
1. Stop using xarray, and protect it by rejecting late NBD_SET_SOCKS
2. Eliminate freeze in nbd_set_size when device setup
3. Add module parameter to eliminate freeze for pre-created device

v1->v2:
1. Rewrite cover letter
2. Eliminate freeze call from blk_mq_update_nr_hw_queues in
   nbd_start_device for netlink path


Long Li (1):
  nbd: simplify find_fallback() by removing redundant logic

Yang Erkun (7):
  nbd: disallow NBD_SET_SOCK on an active device
  nbd: reset write cache on disconnect
  nbd: remove queue freeze in nbd_add_socket
  nbd: skip queue freeze when setting size at device startup
  nbd: factor out a nbd_genl_foreach_sock
  nbd: remove queue freeze for newly created nbd from netlink path
  nbd: add nr_hw_queues module parameter for pre-created devices

 drivers/block/nbd.c | 235 ++++++++++++++++++++++++--------------------
 1 file changed, 128 insertions(+), 107 deletions(-)

-- 
2.52.0

[PATCH v5 1/8] nbd: simplify find_fallback() by removing redundant logic

[Yang Erkun]

From: Long Li <leo.lilong@huawei.com>

The second conditional checking nsock->fallback_index validity is the
logical inverse of the first, so drop it and let execution fall through
naturally. Consolidate the two identical dev_err_ratelimited() + return
paths into a single no_fallback label to reduce duplication.

Signed-off-by: Long Li <leo.lilong@huawei.com>
Reviewed-by: Yu Kuai <yukuai@fygo.io>
---
 drivers/block/nbd.c | 37 ++++++++++++++-----------------------
 1 file changed, 14 insertions(+), 23 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 8f10762e90ef..b1a5acd57426 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1061,40 +1061,31 @@ static int find_fallback(struct nbd_device *nbd, int index)
 	int new_index = -1;
 	struct nbd_sock *nsock = config->socks[index];
 	int fallback = nsock->fallback_index;
+	int i;
 
 	if (test_bit(NBD_RT_DISCONNECTED, &config->runtime_flags))
 		return new_index;
 
-	if (config->num_connections <= 1) {
-		dev_err_ratelimited(disk_to_dev(nbd->disk),
-				    "Dead connection, failed to find a fallback\n");
-		return new_index;
-	}
+	if (config->num_connections <= 1)
+		goto no_fallback;
 
 	if (fallback >= 0 && fallback < config->num_connections &&
 	    !config->socks[fallback]->dead)
 		return fallback;
 
-	if (nsock->fallback_index < 0 ||
-	    nsock->fallback_index >= config->num_connections ||
-	    config->socks[nsock->fallback_index]->dead) {
-		int i;
-		for (i = 0; i < config->num_connections; i++) {
-			if (i == index)
-				continue;
-			if (!config->socks[i]->dead) {
-				new_index = i;
-				break;
-			}
-		}
-		nsock->fallback_index = new_index;
-		if (new_index < 0) {
-			dev_err_ratelimited(disk_to_dev(nbd->disk),
-					    "Dead connection, failed to find a fallback\n");
-			return new_index;
+	for (i = 0; i < config->num_connections; i++) {
+		if (i != index && !config->socks[i]->dead) {
+			new_index = i;
+			break;
 		}
 	}
-	new_index = nsock->fallback_index;
+	nsock->fallback_index = new_index;
+	if (new_index >= 0)
+		return new_index;
+
+no_fallback:
+	dev_err_ratelimited(disk_to_dev(nbd->disk),
+			    "Dead connection, failed to find a fallback\n");
 	return new_index;
 }
 
-- 
2.52.0

[PATCH v5 2/8] nbd: disallow NBD_SET_SOCK on an active device

[Yang Erkun]

We cannot add a socket to an already running nbd device, the reconfigure
for netlink can only active an inactive socket. But for ioctl path, we can
call NBD_SET_SOCK after NBD_DO_IT, reject this using nbd->pid which has
been setted when NBD_DO_IT. Besides, it is the root cause for commit
b98e762e3d71 ("nbd: freeze the queue while we're adding connections").

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b1a5acd57426..7ec85f94f742 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1278,6 +1278,13 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg,
 	/* Arg will be cast to int, check it to avoid overflow */
 	if (arg > INT_MAX)
 		return -EINVAL;
+
+	if (nbd->pid) {
+		dev_err(disk_to_dev(nbd->disk),
+			"Cannot add socket to a running device\n");
+		return -EBUSY;
+	}
+
 	sock = nbd_get_socket(nbd, arg, &err);
 	if (!sock)
 		return err;
-- 
2.52.0

Re: [PATCH v5 2/8] nbd: disallow NBD_SET_SOCK on an active device

[yu kuai]

在 2026/7/30 16:20, Yang Erkun 写道:

> We cannot add a socket to an already running nbd device, the reconfigure
> for netlink can only active an inactive socket. But for ioctl path, we can
> call NBD_SET_SOCK after NBD_DO_IT, reject this using nbd->pid which has
> been setted when NBD_DO_IT. Besides, it is the root cause for commit
> b98e762e3d71 ("nbd: freeze the queue while we're adding connections").
>
> Signed-off-by: Yang Erkun<yangerkun@huawei.com>
> ---
>   drivers/block/nbd.c | 7 +++++++
>   1 file changed, 7 insertions(+)
Reviewed-by: Yu Kuai <yukuai@fygo.io>

-- 
Thanks,
Kuai

[PATCH v5 3/8] nbd: reset write cache on disconnect

[Yang Erkun]

An inactive nbd device may refuse any I/O operations. The nbd_config_put
function calls invalidate_disk, which sets the device capacity to zero
to reject all read and write I/O. For zero-sector flush I/O requests
from blkdev_issue_flush, if the write cache is disabled, the zero-sector
flush I/O immediately returns 0 in submit_bio_noacct. However, since
nbd_config_put does not clear the write cache state, an inactive nbd
device might still have the write cache enabled. In this situation,
zero-sector flush I/O will return -EIO because there is no active socket.
Although this does not cause any bugs, resetting the write cache state
would make the behavior consistent.

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 7ec85f94f742..da4b48ef3c79 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1469,8 +1469,13 @@ static void nbd_config_put(struct nbd_device *nbd)
 	if (refcount_dec_and_mutex_lock(&nbd->config_refs,
 					&nbd->config_lock)) {
 		struct nbd_config *config = nbd->config;
+		struct queue_limits lim;
 		nbd_dev_dbg_close(nbd);
 		invalidate_disk(nbd->disk);
+		/* reset write cache */
+		lim = queue_limits_start_update(nbd->disk->queue);
+		lim.features &= ~(BLK_FEAT_WRITE_CACHE | BLK_FEAT_FUA);
+		queue_limits_commit_update(nbd->disk->queue, &lim);
 		if (nbd->config->bytesize)
 			kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
 		if (test_and_clear_bit(NBD_RT_HAS_PID_FILE,
-- 
2.52.0

Re: [PATCH v5 3/8] nbd: reset write cache on disconnect

[yu kuai]

Hi,

在 2026/7/30 16:20, Yang Erkun 写道:
> An inactive nbd device may refuse any I/O operations. The nbd_config_put
> function calls invalidate_disk, which sets the device capacity to zero
> to reject all read and write I/O. For zero-sector flush I/O requests
> from blkdev_issue_flush, if the write cache is disabled, the zero-sector
> flush I/O immediately returns 0 in submit_bio_noacct. However, since
> nbd_config_put does not clear the write cache state, an inactive nbd
> device might still have the write cache enabled. In this situation,
> zero-sector flush I/O will return -EIO because there is no active socket.
> Although this does not cause any bugs, resetting the write cache state
> would make the behavior consistent.

I don't think this make sense, at this point nbd->config_refs is 0, which means
there is no opener for the device, so how can there still be outstanding read/write
or flush IO?

>
> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
> ---
>   drivers/block/nbd.c | 5 +++++
>   1 file changed, 5 insertions(+)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 7ec85f94f742..da4b48ef3c79 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -1469,8 +1469,13 @@ static void nbd_config_put(struct nbd_device *nbd)
>   	if (refcount_dec_and_mutex_lock(&nbd->config_refs,
>   					&nbd->config_lock)) {
>   		struct nbd_config *config = nbd->config;
> +		struct queue_limits lim;
>   		nbd_dev_dbg_close(nbd);
>   		invalidate_disk(nbd->disk);
> +		/* reset write cache */
> +		lim = queue_limits_start_update(nbd->disk->queue);
> +		lim.features &= ~(BLK_FEAT_WRITE_CACHE | BLK_FEAT_FUA);
> +		queue_limits_commit_update(nbd->disk->queue, &lim);
>   		if (nbd->config->bytesize)
>   			kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
>   		if (test_and_clear_bit(NBD_RT_HAS_PID_FILE,

-- 
Thanks,
Kuai

Re: [PATCH v5 3/8] nbd: reset write cache on disconnect

[yu kuai]

Hi,

在 2026/8/2 19:51, yu kuai 写道:
> Hi,
>
> 在 2026/7/30 16:20, Yang Erkun 写道:
>> An inactive nbd device may refuse any I/O operations. The nbd_config_put
>> function calls invalidate_disk, which sets the device capacity to zero
>> to reject all read and write I/O. For zero-sector flush I/O requests
>> from blkdev_issue_flush, if the write cache is disabled, the zero-sector
>> flush I/O immediately returns 0 in submit_bio_noacct. However, since
>> nbd_config_put does not clear the write cache state, an inactive nbd
>> device might still have the write cache enabled. In this situation,
>> zero-sector flush I/O will return -EIO because there is no active socket.
>> Although this does not cause any bugs, resetting the write cache state
>> would make the behavior consistent.
> I don't think this make sense, at this point nbd->config_refs is 0, which means
> there is no opener for the device, so how can there still be outstanding read/write
> or flush IO?

I see what you want to do after checking the next patch. However, since a disconnected
nbd device can be reused later, the stale BLK_FEAT flags from the last connection looks
like a real problem. So, I think it's better to reset queue_limits to default, the same
as add_disk().

>
>> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
>> ---
>>    drivers/block/nbd.c | 5 +++++
>>    1 file changed, 5 insertions(+)
>>
>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>> index 7ec85f94f742..da4b48ef3c79 100644
>> --- a/drivers/block/nbd.c
>> +++ b/drivers/block/nbd.c
>> @@ -1469,8 +1469,13 @@ static void nbd_config_put(struct nbd_device *nbd)
>>    	if (refcount_dec_and_mutex_lock(&nbd->config_refs,
>>    					&nbd->config_lock)) {
>>    		struct nbd_config *config = nbd->config;
>> +		struct queue_limits lim;
>>    		nbd_dev_dbg_close(nbd);
>>    		invalidate_disk(nbd->disk);
>> +		/* reset write cache */
>> +		lim = queue_limits_start_update(nbd->disk->queue);
>> +		lim.features &= ~(BLK_FEAT_WRITE_CACHE | BLK_FEAT_FUA);
>> +		queue_limits_commit_update(nbd->disk->queue, &lim);
>>    		if (nbd->config->bytesize)
>>    			kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
>>    		if (test_and_clear_bit(NBD_RT_HAS_PID_FILE,

-- 
Thanks,
Kuai

Re: [PATCH v5 3/8] nbd: reset write cache on disconnect

[yangerkun]

在 2026/8/2 20:02, yu kuai 写道:
> Hi,
> 
> 在 2026/8/2 19:51, yu kuai 写道:
>> Hi,
>>
>> 在 2026/7/30 16:20, Yang Erkun 写道:
>>> An inactive nbd device may refuse any I/O operations. The nbd_config_put
>>> function calls invalidate_disk, which sets the device capacity to zero
>>> to reject all read and write I/O. For zero-sector flush I/O requests
>>> from blkdev_issue_flush, if the write cache is disabled, the zero-sector
>>> flush I/O immediately returns 0 in submit_bio_noacct. However, since
>>> nbd_config_put does not clear the write cache state, an inactive nbd
>>> device might still have the write cache enabled. In this situation,
>>> zero-sector flush I/O will return -EIO because there is no active socket.
>>> Although this does not cause any bugs, resetting the write cache state
>>> would make the behavior consistent.
>> I don't think this make sense, at this point nbd->config_refs is 0, which means
>> there is no opener for the device, so how can there still be outstanding read/write
>> or flush IO?
> 
> I see what you want to do after checking the next patch. However, since a disconnected
> nbd device can be reused later, the stale BLK_FEAT flags from the last connection looks
> like a real problem. So, I think it's better to reset queue_limits to default, the same
> as add_disk().

Yeah, I prefer this way! Thanks for your advise.

> 
>>
>>> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
>>> ---
>>>     drivers/block/nbd.c | 5 +++++
>>>     1 file changed, 5 insertions(+)
>>>
>>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>>> index 7ec85f94f742..da4b48ef3c79 100644
>>> --- a/drivers/block/nbd.c
>>> +++ b/drivers/block/nbd.c
>>> @@ -1469,8 +1469,13 @@ static void nbd_config_put(struct nbd_device *nbd)
>>>     	if (refcount_dec_and_mutex_lock(&nbd->config_refs,
>>>     					&nbd->config_lock)) {
>>>     		struct nbd_config *config = nbd->config;
>>> +		struct queue_limits lim;
>>>     		nbd_dev_dbg_close(nbd);
>>>     		invalidate_disk(nbd->disk);
>>> +		/* reset write cache */
>>> +		lim = queue_limits_start_update(nbd->disk->queue);
>>> +		lim.features &= ~(BLK_FEAT_WRITE_CACHE | BLK_FEAT_FUA);
>>> +		queue_limits_commit_update(nbd->disk->queue, &lim);
>>>     		if (nbd->config->bytesize)
>>>     			kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
>>>     		if (test_and_clear_bit(NBD_RT_HAS_PID_FILE,
> 

[PATCH v5 4/8] nbd: remove queue freeze in nbd_add_socket

[Yang Erkun]

nbd_add_socket() kreallocs config->socks, which a concurrent reader in
nbd_handle_cmd() could UAF; commit b98e762e3d71 ("nbd: freeze the queue
while we're adding connections") froze the queue to block that.  But
the freeze costs an RCU grace period on every socket added, and setup
adds them one by one.

After the previous patch, nbd_add_socket() is rejected once nbd->pid is
set, so it only runs during setup.  There the capacity is 0 and the
write cache is off (cleared on disconnect by the preceding patch, and
re-enabled only later in nbd_set_size), so submit_bio_noacct() rejects
every bio before it reaches the driver -- non-zero-sector ones via
bio_check_eod(), and flush-only ones via the !bdev_write_cache() branch.
No I/O is in flight, so the freeze is unnecessary.

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 9 ---------
 1 file changed, 9 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index da4b48ef3c79..ce25c254e5fb 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1272,7 +1272,6 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg,
 	struct socket *sock;
 	struct nbd_sock **socks;
 	struct nbd_sock *nsock;
-	unsigned int memflags;
 	int err;
 
 	/* Arg will be cast to int, check it to avoid overflow */
@@ -1290,12 +1289,6 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg,
 		return err;
 	nbd_reclassify_socket(sock);
 
-	/*
-	 * We need to make sure we don't get any errant requests while we're
-	 * reallocating the ->socks array.
-	 */
-	memflags = blk_mq_freeze_queue(nbd->disk->queue);
-
 	if (!netlink && !nbd->task_setup &&
 	    !test_bit(NBD_RT_BOUND, &config->runtime_flags))
 		nbd->task_setup = current;
@@ -1335,12 +1328,10 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg,
 	INIT_WORK(&nsock->work, nbd_pending_cmd_work);
 	socks[config->num_connections++] = nsock;
 	atomic_inc(&config->live_connections);
-	blk_mq_unfreeze_queue(nbd->disk->queue, memflags);
 
 	return 0;
 
 put_socket:
-	blk_mq_unfreeze_queue(nbd->disk->queue, memflags);
 	sockfd_put(sock);
 	return err;
 }
-- 
2.52.0

[PATCH v5 5/8] nbd: skip queue freeze when setting size at device startup

[Yang Erkun]

Commit 242a49e5c878 ("nbd: freeze the queue for queue limits updates")
added the freeze to keep in-flight commands from seeing torn
queue_limits.  But at startup the capacity is still 0
(invalidate_disk cleared it) and the write cache is off (the previous
patch cleared it on disconnect, and nbd_set_size sets it back only after
the commit), so submit_bio_noacct() rejects any bio before it reaches
the driver and no I/O is in flight.  Drop the freeze there.

Split nbd_set_size() with a bool: nbd_start_device() passes false, the
runtime resize/reconfigure paths (ioctls and nbd_genl_size_set) keep
passing true.  nbd->pid is still 0 when nbd_genl_connect calls
nbd_genl_size_set, so the early return in nbd_set_size keeps that path
from applying limits before startup; the true covers the reconnect path
on a live device.

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index ce25c254e5fb..3b7363b11d0b 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -331,7 +331,8 @@ static void nbd_mark_nsock_dead(struct nbd_device *nbd, struct nbd_sock *nsock,
 	nsock->sent = 0;
 }
 
-static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize)
+static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize,
+			bool freeze)
 {
 	struct queue_limits lim;
 	int error;
@@ -371,7 +372,13 @@ static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize)
 
 	lim.logical_block_size = blksize;
 	lim.physical_block_size = blksize;
-	error = queue_limits_commit_update_frozen(nbd->disk->queue, &lim);
+
+	if (freeze)
+		error = queue_limits_commit_update_frozen(nbd->disk->queue,
+				&lim);
+	else
+		error = queue_limits_commit_update(nbd->disk->queue, &lim);
+
 	if (error)
 		return error;
 
@@ -1568,7 +1575,7 @@ static int nbd_start_device(struct nbd_device *nbd)
 		args->index = i;
 		queue_work(nbd->recv_workq, &args->work);
 	}
-	return nbd_set_size(nbd, config->bytesize, nbd_blksize(config));
+	return nbd_set_size(nbd, config->bytesize, nbd_blksize(config), false);
 }
 
 static int nbd_start_device_ioctl(struct nbd_device *nbd)
@@ -1636,13 +1643,13 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
 	case NBD_SET_SOCK:
 		return nbd_add_socket(nbd, arg, false);
 	case NBD_SET_BLKSIZE:
-		return nbd_set_size(nbd, config->bytesize, arg);
+		return nbd_set_size(nbd, config->bytesize, arg, true);
 	case NBD_SET_SIZE:
-		return nbd_set_size(nbd, arg, nbd_blksize(config));
+		return nbd_set_size(nbd, arg, nbd_blksize(config), true);
 	case NBD_SET_SIZE_BLOCKS:
 		if (check_shl_overflow(arg, config->blksize_bits, &bytesize))
 			return -EINVAL;
-		return nbd_set_size(nbd, bytesize, nbd_blksize(config));
+		return nbd_set_size(nbd, bytesize, nbd_blksize(config), true);
 	case NBD_SET_TIMEOUT:
 		nbd_set_cmd_timeout(nbd, arg);
 		return 0;
@@ -2097,7 +2104,7 @@ static int nbd_genl_size_set(struct genl_info *info, struct nbd_device *nbd)
 		bsize = nla_get_u64(info->attrs[NBD_ATTR_BLOCK_SIZE_BYTES]);
 
 	if (bytes != config->bytesize || bsize != nbd_blksize(config))
-		return nbd_set_size(nbd, bytes, bsize);
+		return nbd_set_size(nbd, bytes, bsize, true);
 	return 0;
 }
 
-- 
2.52.0

Re: [PATCH v5 5/8] nbd: skip queue freeze when setting size at device startup

[yu kuai]

Hi,

在 2026/7/30 16:20, Yang Erkun 写道:
> Commit 242a49e5c878 ("nbd: freeze the queue for queue limits updates")
> added the freeze to keep in-flight commands from seeing torn
> queue_limits.  But at startup the capacity is still 0
> (invalidate_disk cleared it) and the write cache is off (the previous
> patch cleared it on disconnect, and nbd_set_size sets it back only after
> the commit), so submit_bio_noacct() rejects any bio before it reaches
> the driver and no I/O is in flight.  Drop the freeze there.
>
> Split nbd_set_size() with a bool: nbd_start_device() passes false, the
> runtime resize/reconfigure paths (ioctls and nbd_genl_size_set) keep
> passing true.  nbd->pid is still 0 when nbd_genl_connect calls
> nbd_genl_size_set, so the early return in nbd_set_size keeps that path
> from applying limits before startup; the true covers the reconnect path
> on a live device.
>
> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
> ---
>   drivers/block/nbd.c | 21 ++++++++++++++-------
>   1 file changed, 14 insertions(+), 7 deletions(-)

Can you just checking if size is 0 and write cache is disabled first, and just
skip freeze in this case. I think this is simpler and straightforward, as you
don't have to modify all the callers.

>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index ce25c254e5fb..3b7363b11d0b 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -331,7 +331,8 @@ static void nbd_mark_nsock_dead(struct nbd_device *nbd, struct nbd_sock *nsock,
>   	nsock->sent = 0;
>   }
>   
> -static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize)
> +static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize,
> +			bool freeze)
>   {
>   	struct queue_limits lim;
>   	int error;
> @@ -371,7 +372,13 @@ static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize)
>   
>   	lim.logical_block_size = blksize;
>   	lim.physical_block_size = blksize;
> -	error = queue_limits_commit_update_frozen(nbd->disk->queue, &lim);
> +
> +	if (freeze)
> +		error = queue_limits_commit_update_frozen(nbd->disk->queue,
> +				&lim);
> +	else
> +		error = queue_limits_commit_update(nbd->disk->queue, &lim);
> +
>   	if (error)
>   		return error;
>   
> @@ -1568,7 +1575,7 @@ static int nbd_start_device(struct nbd_device *nbd)
>   		args->index = i;
>   		queue_work(nbd->recv_workq, &args->work);
>   	}
> -	return nbd_set_size(nbd, config->bytesize, nbd_blksize(config));
> +	return nbd_set_size(nbd, config->bytesize, nbd_blksize(config), false);
>   }
>   
>   static int nbd_start_device_ioctl(struct nbd_device *nbd)
> @@ -1636,13 +1643,13 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>   	case NBD_SET_SOCK:
>   		return nbd_add_socket(nbd, arg, false);
>   	case NBD_SET_BLKSIZE:
> -		return nbd_set_size(nbd, config->bytesize, arg);
> +		return nbd_set_size(nbd, config->bytesize, arg, true);
>   	case NBD_SET_SIZE:
> -		return nbd_set_size(nbd, arg, nbd_blksize(config));
> +		return nbd_set_size(nbd, arg, nbd_blksize(config), true);
>   	case NBD_SET_SIZE_BLOCKS:
>   		if (check_shl_overflow(arg, config->blksize_bits, &bytesize))
>   			return -EINVAL;
> -		return nbd_set_size(nbd, bytesize, nbd_blksize(config));
> +		return nbd_set_size(nbd, bytesize, nbd_blksize(config), true);
>   	case NBD_SET_TIMEOUT:
>   		nbd_set_cmd_timeout(nbd, arg);
>   		return 0;
> @@ -2097,7 +2104,7 @@ static int nbd_genl_size_set(struct genl_info *info, struct nbd_device *nbd)
>   		bsize = nla_get_u64(info->attrs[NBD_ATTR_BLOCK_SIZE_BYTES]);
>   
>   	if (bytes != config->bytesize || bsize != nbd_blksize(config))
> -		return nbd_set_size(nbd, bytes, bsize);
> +		return nbd_set_size(nbd, bytes, bsize, true);
>   	return 0;
>   }
>   

-- 
Thanks,
Kuai

Re: [PATCH v5 5/8] nbd: skip queue freeze when setting size at device startup

[yangerkun]

在 2026/8/2 19:45, yu kuai 写道:
> Hi,
> 
> 在 2026/7/30 16:20, Yang Erkun 写道:
>> Commit 242a49e5c878 ("nbd: freeze the queue for queue limits updates")
>> added the freeze to keep in-flight commands from seeing torn
>> queue_limits.  But at startup the capacity is still 0
>> (invalidate_disk cleared it) and the write cache is off (the previous
>> patch cleared it on disconnect, and nbd_set_size sets it back only after
>> the commit), so submit_bio_noacct() rejects any bio before it reaches
>> the driver and no I/O is in flight.  Drop the freeze there.
>>
>> Split nbd_set_size() with a bool: nbd_start_device() passes false, the
>> runtime resize/reconfigure paths (ioctls and nbd_genl_size_set) keep
>> passing true.  nbd->pid is still 0 when nbd_genl_connect calls
>> nbd_genl_size_set, so the early return in nbd_set_size keeps that path
>> from applying limits before startup; the true covers the reconnect path
>> on a live device.
>>
>> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
>> ---
>>    drivers/block/nbd.c | 21 ++++++++++++++-------
>>    1 file changed, 14 insertions(+), 7 deletions(-)
> 
> Can you just checking if size is 0 and write cache is disabled first, and just
> skip freeze in this case. I think this is simpler and straightforward, as you
> don't have to modify all the callers.

OK, it seems simple and clear, I will try it next version!

> 
>>
>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>> index ce25c254e5fb..3b7363b11d0b 100644
>> --- a/drivers/block/nbd.c
>> +++ b/drivers/block/nbd.c
>> @@ -331,7 +331,8 @@ static void nbd_mark_nsock_dead(struct nbd_device *nbd, struct nbd_sock *nsock,
>>    	nsock->sent = 0;
>>    }
>>    
>> -static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize)
>> +static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize,
>> +			bool freeze)
>>    {
>>    	struct queue_limits lim;
>>    	int error;
>> @@ -371,7 +372,13 @@ static int nbd_set_size(struct nbd_device *nbd, loff_t bytesize, loff_t blksize)
>>    
>>    	lim.logical_block_size = blksize;
>>    	lim.physical_block_size = blksize;
>> -	error = queue_limits_commit_update_frozen(nbd->disk->queue, &lim);
>> +
>> +	if (freeze)
>> +		error = queue_limits_commit_update_frozen(nbd->disk->queue,
>> +				&lim);
>> +	else
>> +		error = queue_limits_commit_update(nbd->disk->queue, &lim);
>> +
>>    	if (error)
>>    		return error;
>>    
>> @@ -1568,7 +1575,7 @@ static int nbd_start_device(struct nbd_device *nbd)
>>    		args->index = i;
>>    		queue_work(nbd->recv_workq, &args->work);
>>    	}
>> -	return nbd_set_size(nbd, config->bytesize, nbd_blksize(config));
>> +	return nbd_set_size(nbd, config->bytesize, nbd_blksize(config), false);
>>    }
>>    
>>    static int nbd_start_device_ioctl(struct nbd_device *nbd)
>> @@ -1636,13 +1643,13 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>>    	case NBD_SET_SOCK:
>>    		return nbd_add_socket(nbd, arg, false);
>>    	case NBD_SET_BLKSIZE:
>> -		return nbd_set_size(nbd, config->bytesize, arg);
>> +		return nbd_set_size(nbd, config->bytesize, arg, true);
>>    	case NBD_SET_SIZE:
>> -		return nbd_set_size(nbd, arg, nbd_blksize(config));
>> +		return nbd_set_size(nbd, arg, nbd_blksize(config), true);
>>    	case NBD_SET_SIZE_BLOCKS:
>>    		if (check_shl_overflow(arg, config->blksize_bits, &bytesize))
>>    			return -EINVAL;
>> -		return nbd_set_size(nbd, bytesize, nbd_blksize(config));
>> +		return nbd_set_size(nbd, bytesize, nbd_blksize(config), true);
>>    	case NBD_SET_TIMEOUT:
>>    		nbd_set_cmd_timeout(nbd, arg);
>>    		return 0;
>> @@ -2097,7 +2104,7 @@ static int nbd_genl_size_set(struct genl_info *info, struct nbd_device *nbd)
>>    		bsize = nla_get_u64(info->attrs[NBD_ATTR_BLOCK_SIZE_BYTES]);
>>    
>>    	if (bytes != config->bytesize || bsize != nbd_blksize(config))
>> -		return nbd_set_size(nbd, bytes, bsize);
>> +		return nbd_set_size(nbd, bytes, bsize, true);
>>    	return 0;
>>    }
>>    
> 

[PATCH v5 6/8] nbd: factor out a nbd_genl_foreach_sock

[Yang Erkun]

The NBD_ATTR_SOCKETS walk is duplicated in nbd_genl_connect (add sockets)
and nbd_genl_reconfigure (reconnect).  Factor out a single helper that
walks the list and calls a callback per fd; with a NULL callback it is a
pure counter, used by a later patch to learn nr_hw_queues before the
device exists.  Returns the number of fds walked (>= 0) or a negative
errno; a callback >0 stops early as success (reconnect's -ENOSPC).

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 137 +++++++++++++++++++++++---------------------
 1 file changed, 73 insertions(+), 64 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 3b7363b11d0b..34b84fc1c61f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -2108,6 +2108,58 @@ static int nbd_genl_size_set(struct genl_info *info, struct nbd_device *nbd)
 	return 0;
 }
 
+/*
+ * Walk the NBD_ATTR_SOCKETS nested list can call @cb for each socket fd.
+ *
+ * Return the number of fds walked, or a negative errno.
+ */
+static int nbd_genl_foreach_sock(struct genl_info *info,
+		int (*cb)(struct nbd_device *nbd, int fd),
+		struct nbd_device *nbd)
+{
+	struct nlattr *attr;
+	int rem, count = 0;
+
+	if (!info->attrs[NBD_ATTR_SOCKETS])
+		return 0;
+
+	nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS], rem) {
+		struct nlattr *socks[NBD_SOCK_MAX + 1];
+		int ret;
+
+		if (nla_type(attr) != NBD_SOCK_ITEM) {
+			pr_err("socks must be embedded in a SOCK_ITEM attr\n");
+			return -EINVAL;
+		}
+
+		if (nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
+						attr,
+						nbd_sock_policy,
+						info->extack)) {
+			pr_err("error processing sock list\n");
+			return -EINVAL;
+		}
+
+		if (!socks[NBD_SOCK_FD])
+			continue;
+
+		count++;
+		if (cb) {
+			ret = cb(nbd, (int)nla_get_u32(socks[NBD_SOCK_FD]));
+			if (ret > 0)
+				return count;
+			if (ret < 0)
+				return ret;
+		}
+	}
+	return count;
+}
+
+static int nbd_genl_connect_sock_cb(struct nbd_device *nbd, int fd)
+{
+	return nbd_add_socket(nbd, fd, true);
+}
+
 static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
 {
 	struct nbd_device *nbd;
@@ -2227,36 +2279,9 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
 		}
 	}
 
-	if (info->attrs[NBD_ATTR_SOCKETS]) {
-		struct nlattr *attr;
-		int rem, fd;
-
-		nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS],
-				    rem) {
-			struct nlattr *socks[NBD_SOCK_MAX+1];
-
-			if (nla_type(attr) != NBD_SOCK_ITEM) {
-				pr_err("socks must be embedded in a SOCK_ITEM attr\n");
-				ret = -EINVAL;
-				goto out;
-			}
-			ret = nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
-							  attr,
-							  nbd_sock_policy,
-							  info->extack);
-			if (ret != 0) {
-				pr_err("error processing sock list\n");
-				ret = -EINVAL;
-				goto out;
-			}
-			if (!socks[NBD_SOCK_FD])
-				continue;
-			fd = (int)nla_get_u32(socks[NBD_SOCK_FD]);
-			ret = nbd_add_socket(nbd, fd, true);
-			if (ret)
-				goto out;
-		}
-	}
+	ret = nbd_genl_foreach_sock(info, nbd_genl_connect_sock_cb, nbd);
+	if (ret < 0)
+		goto out;
 
 	if (info->attrs[NBD_ATTR_BACKEND_IDENTIFIER]) {
 		nbd->backend = nla_strdup(info->attrs[NBD_ATTR_BACKEND_IDENTIFIER],
@@ -2345,6 +2370,20 @@ static int nbd_genl_disconnect(struct sk_buff *skb, struct genl_info *info)
 	return 0;
 }
 
+static int nbd_genl_reconnect_sock_cb(struct nbd_device *nbd, int fd)
+{
+	int ret = nbd_reconnect_socket(nbd, fd);
+
+	if (!ret) {
+		dev_info(nbd_to_dev(nbd), "reconnected socket\n");
+		return 0;
+	}
+
+	if (ret == -ENOSPC)
+		return 1;
+	return ret;
+}
+
 static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info)
 {
 	struct nbd_device *nbd = NULL;
@@ -2441,40 +2480,10 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info)
 		}
 	}
 
-	if (info->attrs[NBD_ATTR_SOCKETS]) {
-		struct nlattr *attr;
-		int rem, fd;
-
-		nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS],
-				    rem) {
-			struct nlattr *socks[NBD_SOCK_MAX+1];
-
-			if (nla_type(attr) != NBD_SOCK_ITEM) {
-				pr_err("socks must be embedded in a SOCK_ITEM attr\n");
-				ret = -EINVAL;
-				goto out;
-			}
-			ret = nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
-							  attr,
-							  nbd_sock_policy,
-							  info->extack);
-			if (ret != 0) {
-				pr_err("error processing sock list\n");
-				ret = -EINVAL;
-				goto out;
-			}
-			if (!socks[NBD_SOCK_FD])
-				continue;
-			fd = (int)nla_get_u32(socks[NBD_SOCK_FD]);
-			ret = nbd_reconnect_socket(nbd, fd);
-			if (ret) {
-				if (ret == -ENOSPC)
-					ret = 0;
-				goto out;
-			}
-			dev_info(nbd_to_dev(nbd), "reconnected socket\n");
-		}
-	}
+	ret = nbd_genl_foreach_sock(info, nbd_genl_reconnect_sock_cb, nbd);
+	/* foreach_sock returns a positive count on success; doit must return 0 */
+	if (ret >= 0)
+		ret = 0;
 out:
 	mutex_unlock(&nbd->config_lock);
 	nbd_config_put(nbd);
-- 
2.52.0

Re: [PATCH v5 6/8] nbd: factor out a nbd_genl_foreach_sock

[yu kuai]

Hi,

在 2026/7/30 16:20, Yang Erkun 写道:
> The NBD_ATTR_SOCKETS walk is duplicated in nbd_genl_connect (add sockets)
> and nbd_genl_reconfigure (reconnect).  Factor out a single helper that
> walks the list and calls a callback per fd; with a NULL callback it is a
> pure counter, used by a later patch to learn nr_hw_queues before the
> device exists.  Returns the number of fds walked (>= 0) or a negative
> errno; a callback >0 stops early as success (reconnect's -ENOSPC).
>
> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
> ---
>   drivers/block/nbd.c | 137 +++++++++++++++++++++++---------------------
>   1 file changed, 73 insertions(+), 64 deletions(-)

This patch LGTM, two nits below.

>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 3b7363b11d0b..34b84fc1c61f 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -2108,6 +2108,58 @@ static int nbd_genl_size_set(struct genl_info *info, struct nbd_device *nbd)
>   	return 0;
>   }
>   
> +/*
> + * Walk the NBD_ATTR_SOCKETS nested list can call @cb for each socket fd.
> + *
> + * Return the number of fds walked, or a negative errno.
> + */
> +static int nbd_genl_foreach_sock(struct genl_info *info,
> +		int (*cb)(struct nbd_device *nbd, int fd),
> +		struct nbd_device *nbd)
> +{
> +	struct nlattr *attr;
> +	int rem, count = 0;
> +
> +	if (!info->attrs[NBD_ATTR_SOCKETS])
> +		return 0;
> +
> +	nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS], rem) {
> +		struct nlattr *socks[NBD_SOCK_MAX + 1];
> +		int ret;
> +
> +		if (nla_type(attr) != NBD_SOCK_ITEM) {
> +			pr_err("socks must be embedded in a SOCK_ITEM attr\n");
> +			return -EINVAL;
> +		}
> +
> +		if (nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
> +						attr,
> +						nbd_sock_policy,
> +						info->extack)) {
> +			pr_err("error processing sock list\n");
> +			return -EINVAL;
> +		}
> +
> +		if (!socks[NBD_SOCK_FD])
> +			continue;
> +
> +		count++;
> +		if (cb) {
> +			ret = cb(nbd, (int)nla_get_u32(socks[NBD_SOCK_FD]));
> +			if (ret > 0)
> +				return count;
> +			if (ret < 0)
> +				return ret;
> +		}
> +	}
> +	return count;
> +}
> +
> +static int nbd_genl_connect_sock_cb(struct nbd_device *nbd, int fd)
> +{
> +	return nbd_add_socket(nbd, fd, true);
> +}
> +
>   static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
>   {
>   	struct nbd_device *nbd;
> @@ -2227,36 +2279,9 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
>   		}
>   	}
>   
> -	if (info->attrs[NBD_ATTR_SOCKETS]) {
> -		struct nlattr *attr;
> -		int rem, fd;
> -
> -		nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS],
> -				    rem) {
> -			struct nlattr *socks[NBD_SOCK_MAX+1];
> -
> -			if (nla_type(attr) != NBD_SOCK_ITEM) {
> -				pr_err("socks must be embedded in a SOCK_ITEM attr\n");
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			ret = nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
> -							  attr,
> -							  nbd_sock_policy,
> -							  info->extack);
> -			if (ret != 0) {
> -				pr_err("error processing sock list\n");
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			if (!socks[NBD_SOCK_FD])
> -				continue;
> -			fd = (int)nla_get_u32(socks[NBD_SOCK_FD]);
> -			ret = nbd_add_socket(nbd, fd, true);
> -			if (ret)
> -				goto out;
> -		}
> -	}
> +	ret = nbd_genl_foreach_sock(info, nbd_genl_connect_sock_cb, nbd);
> +	if (ret < 0)
> +		goto out;
>   
>   	if (info->attrs[NBD_ATTR_BACKEND_IDENTIFIER]) {
>   		nbd->backend = nla_strdup(info->attrs[NBD_ATTR_BACKEND_IDENTIFIER],
> @@ -2345,6 +2370,20 @@ static int nbd_genl_disconnect(struct sk_buff *skb, struct genl_info *info)
>   	return 0;
>   }
>   
> +static int nbd_genl_reconnect_sock_cb(struct nbd_device *nbd, int fd)
> +{
> +	int ret = nbd_reconnect_socket(nbd, fd);
> +
> +	if (!ret) {
> +		dev_info(nbd_to_dev(nbd), "reconnected socket\n");
> +		return 0;
> +	}
> +
> +	if (ret == -ENOSPC)
> +		return 1;
> +	return ret;
> +}

Since there is only one caller for nbd_reconnect_socket(), you might as well just fold above
changes into nbd_reconnect_socket() directly.

> +
>   static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info)
>   {
>   	struct nbd_device *nbd = NULL;
> @@ -2441,40 +2480,10 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info)
>   		}
>   	}
>   
> -	if (info->attrs[NBD_ATTR_SOCKETS]) {
> -		struct nlattr *attr;
> -		int rem, fd;
> -
> -		nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS],
> -				    rem) {
> -			struct nlattr *socks[NBD_SOCK_MAX+1];
> -
> -			if (nla_type(attr) != NBD_SOCK_ITEM) {
> -				pr_err("socks must be embedded in a SOCK_ITEM attr\n");
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			ret = nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
> -							  attr,
> -							  nbd_sock_policy,
> -							  info->extack);
> -			if (ret != 0) {
> -				pr_err("error processing sock list\n");
> -				ret = -EINVAL;
> -				goto out;
> -			}
> -			if (!socks[NBD_SOCK_FD])
> -				continue;
> -			fd = (int)nla_get_u32(socks[NBD_SOCK_FD]);
> -			ret = nbd_reconnect_socket(nbd, fd);
> -			if (ret) {
> -				if (ret == -ENOSPC)
> -					ret = 0;
> -				goto out;
> -			}
> -			dev_info(nbd_to_dev(nbd), "reconnected socket\n");
> -		}
> -	}
> +	ret = nbd_genl_foreach_sock(info, nbd_genl_reconnect_sock_cb, nbd);
> +	/* foreach_sock returns a positive count on success; doit must return 0 */
> +	if (ret >= 0)
> +		ret = 0;
>   out:
>   	mutex_unlock(&nbd->config_lock);
>   	nbd_config_put(nbd);

-- 
Thanks,
Kuai

Re: [PATCH v5 6/8] nbd: factor out a nbd_genl_foreach_sock

[yangerkun]

在 2026/8/2 20:14, yu kuai 写道:
> Hi,
> 
> 在 2026/7/30 16:20, Yang Erkun 写道:
>> The NBD_ATTR_SOCKETS walk is duplicated in nbd_genl_connect (add sockets)
>> and nbd_genl_reconfigure (reconnect).  Factor out a single helper that
>> walks the list and calls a callback per fd; with a NULL callback it is a
>> pure counter, used by a later patch to learn nr_hw_queues before the
>> device exists.  Returns the number of fds walked (>= 0) or a negative
>> errno; a callback >0 stops early as success (reconnect's -ENOSPC).
>>
>> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
>> ---
>>    drivers/block/nbd.c | 137 +++++++++++++++++++++++---------------------
>>    1 file changed, 73 insertions(+), 64 deletions(-)
> 
> This patch LGTM, two nits below.
> 
>>
>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>> index 3b7363b11d0b..34b84fc1c61f 100644
>> --- a/drivers/block/nbd.c
>> +++ b/drivers/block/nbd.c
>> @@ -2108,6 +2108,58 @@ static int nbd_genl_size_set(struct genl_info *info, struct nbd_device *nbd)
>>    	return 0;
>>    }
>>    
>> +/*
>> + * Walk the NBD_ATTR_SOCKETS nested list can call @cb for each socket fd.
>> + *
>> + * Return the number of fds walked, or a negative errno.
>> + */
>> +static int nbd_genl_foreach_sock(struct genl_info *info,
>> +		int (*cb)(struct nbd_device *nbd, int fd),
>> +		struct nbd_device *nbd)
>> +{
>> +	struct nlattr *attr;
>> +	int rem, count = 0;
>> +
>> +	if (!info->attrs[NBD_ATTR_SOCKETS])
>> +		return 0;
>> +
>> +	nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS], rem) {
>> +		struct nlattr *socks[NBD_SOCK_MAX + 1];
>> +		int ret;
>> +
>> +		if (nla_type(attr) != NBD_SOCK_ITEM) {
>> +			pr_err("socks must be embedded in a SOCK_ITEM attr\n");
>> +			return -EINVAL;
>> +		}
>> +
>> +		if (nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
>> +						attr,
>> +						nbd_sock_policy,
>> +						info->extack)) {
>> +			pr_err("error processing sock list\n");
>> +			return -EINVAL;
>> +		}
>> +
>> +		if (!socks[NBD_SOCK_FD])
>> +			continue;
>> +
>> +		count++;
>> +		if (cb) {
>> +			ret = cb(nbd, (int)nla_get_u32(socks[NBD_SOCK_FD]));
>> +			if (ret > 0)
>> +				return count;
>> +			if (ret < 0)
>> +				return ret;
>> +		}
>> +	}
>> +	return count;
>> +}
>> +
>> +static int nbd_genl_connect_sock_cb(struct nbd_device *nbd, int fd)
>> +{
>> +	return nbd_add_socket(nbd, fd, true);
>> +}
>> +
>>    static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
>>    {
>>    	struct nbd_device *nbd;
>> @@ -2227,36 +2279,9 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
>>    		}
>>    	}
>>    
>> -	if (info->attrs[NBD_ATTR_SOCKETS]) {
>> -		struct nlattr *attr;
>> -		int rem, fd;
>> -
>> -		nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS],
>> -				    rem) {
>> -			struct nlattr *socks[NBD_SOCK_MAX+1];
>> -
>> -			if (nla_type(attr) != NBD_SOCK_ITEM) {
>> -				pr_err("socks must be embedded in a SOCK_ITEM attr\n");
>> -				ret = -EINVAL;
>> -				goto out;
>> -			}
>> -			ret = nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
>> -							  attr,
>> -							  nbd_sock_policy,
>> -							  info->extack);
>> -			if (ret != 0) {
>> -				pr_err("error processing sock list\n");
>> -				ret = -EINVAL;
>> -				goto out;
>> -			}
>> -			if (!socks[NBD_SOCK_FD])
>> -				continue;
>> -			fd = (int)nla_get_u32(socks[NBD_SOCK_FD]);
>> -			ret = nbd_add_socket(nbd, fd, true);
>> -			if (ret)
>> -				goto out;
>> -		}
>> -	}
>> +	ret = nbd_genl_foreach_sock(info, nbd_genl_connect_sock_cb, nbd);
>> +	if (ret < 0)
>> +		goto out;
>>    
>>    	if (info->attrs[NBD_ATTR_BACKEND_IDENTIFIER]) {
>>    		nbd->backend = nla_strdup(info->attrs[NBD_ATTR_BACKEND_IDENTIFIER],
>> @@ -2345,6 +2370,20 @@ static int nbd_genl_disconnect(struct sk_buff *skb, struct genl_info *info)
>>    	return 0;
>>    }
>>    
>> +static int nbd_genl_reconnect_sock_cb(struct nbd_device *nbd, int fd)
>> +{
>> +	int ret = nbd_reconnect_socket(nbd, fd);
>> +
>> +	if (!ret) {
>> +		dev_info(nbd_to_dev(nbd), "reconnected socket\n");
>> +		return 0;
>> +	}
>> +
>> +	if (ret == -ENOSPC)
>> +		return 1;
>> +	return ret;
>> +}
> 
> Since there is only one caller for nbd_reconnect_socket(), you might as well just fold above
> changes into nbd_reconnect_socket() directly.

OK, will do it next version!

> 
>> +
>>    static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info)
>>    {
>>    	struct nbd_device *nbd = NULL;
>> @@ -2441,40 +2480,10 @@ static int nbd_genl_reconfigure(struct sk_buff *skb, struct genl_info *info)
>>    		}
>>    	}
>>    
>> -	if (info->attrs[NBD_ATTR_SOCKETS]) {
>> -		struct nlattr *attr;
>> -		int rem, fd;
>> -
>> -		nla_for_each_nested(attr, info->attrs[NBD_ATTR_SOCKETS],
>> -				    rem) {
>> -			struct nlattr *socks[NBD_SOCK_MAX+1];
>> -
>> -			if (nla_type(attr) != NBD_SOCK_ITEM) {
>> -				pr_err("socks must be embedded in a SOCK_ITEM attr\n");
>> -				ret = -EINVAL;
>> -				goto out;
>> -			}
>> -			ret = nla_parse_nested_deprecated(socks, NBD_SOCK_MAX,
>> -							  attr,
>> -							  nbd_sock_policy,
>> -							  info->extack);
>> -			if (ret != 0) {
>> -				pr_err("error processing sock list\n");
>> -				ret = -EINVAL;
>> -				goto out;
>> -			}
>> -			if (!socks[NBD_SOCK_FD])
>> -				continue;
>> -			fd = (int)nla_get_u32(socks[NBD_SOCK_FD]);
>> -			ret = nbd_reconnect_socket(nbd, fd);
>> -			if (ret) {
>> -				if (ret == -ENOSPC)
>> -					ret = 0;
>> -				goto out;
>> -			}
>> -			dev_info(nbd_to_dev(nbd), "reconnected socket\n");
>> -		}
>> -	}
>> +	ret = nbd_genl_foreach_sock(info, nbd_genl_reconnect_sock_cb, nbd);
>> +	/* foreach_sock returns a positive count on success; doit must return 0 */
>> +	if (ret >= 0)
>> +		ret = 0;
>>    out:
>>    	mutex_unlock(&nbd->config_lock);
>>    	nbd_config_put(nbd);
> 

[PATCH v5 7/8] nbd: remove queue freeze for newly created nbd from netlink path

[Yang Erkun]

Previous commits has removed the queue freeze in nbd_add_socket and
nbd_set_size during nbd device setup. However, a queue freeze can still
occur when nbd_start_device calls blk_mq_update_nr_hw_queues if the
socket connection count does not match nbd->tag_set->nr_hw_queues.

The nbd_start_device function can be invoked through either the ioctl or
netlink paths. The ioctl path only allows reusing an existing inactivate
nbd device, there is nothing more we can do to prevent the queue freeze
since the old nbd->tag_set->nr_hw_queues may not match the new socket
connection count. Similarly, the netlink path can reuse a preferred
inactivate nbd device, and again, we cannot do more in this scenario.
However, the netlink path can also add a new nbd device using
nbd_dev_add. In this case, we can obtain the new number of socket
connections, and by adding a new argument representing the expected
nr_hw_queues in nbd_dev_add, we can ensure the queue freeze is avoided
for this situation.

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 34b84fc1c61f..6369a409e10c 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1942,7 +1942,8 @@ static const struct blk_mq_ops nbd_mq_ops = {
 	.timeout	= nbd_xmit_timeout,
 };
 
-static struct nbd_device *nbd_dev_add(int index, unsigned int refs)
+static struct nbd_device *nbd_dev_add(int index, unsigned int refs,
+				       int nr_hw_queues)
 {
 	struct queue_limits lim = {
 		.max_hw_sectors		= 65536,
@@ -1959,7 +1960,7 @@ static struct nbd_device *nbd_dev_add(int index, unsigned int refs)
 		goto out;
 
 	nbd->tag_set.ops = &nbd_mq_ops;
-	nbd->tag_set.nr_hw_queues = 1;
+	nbd->tag_set.nr_hw_queues = nr_hw_queues;
 	nbd->tag_set.queue_depth = 128;
 	nbd->tag_set.numa_node = NUMA_NO_NODE;
 	nbd->tag_set.cmd_size = sizeof(struct nbd_cmd);
@@ -2212,7 +2213,11 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
 	mutex_unlock(&nbd_index_mutex);
 
 	if (!nbd) {
-		nbd = nbd_dev_add(index, 2);
+		ret = nbd_genl_foreach_sock(info, NULL, NULL);
+		if (ret < 0)
+			return ret;
+
+		nbd = nbd_dev_add(index, 2, ret > 0 ? ret : 1);
 		if (IS_ERR(nbd)) {
 			pr_err("failed to add new device\n");
 			return PTR_ERR(nbd);
@@ -2736,7 +2741,7 @@ static int __init nbd_init(void)
 	nbd_dbg_init();
 
 	for (i = 0; i < nbds_max; i++)
-		nbd_dev_add(i, 1);
+		nbd_dev_add(i, 1, 1);
 	return 0;
 }
 
-- 
2.52.0

Re: [PATCH v5 7/8] nbd: remove queue freeze for newly created nbd from netlink path

[yu kuai]

在 2026/7/30 16:20, Yang Erkun 写道:

> Previous commits has removed the queue freeze in nbd_add_socket and
> nbd_set_size during nbd device setup. However, a queue freeze can still
> occur when nbd_start_device calls blk_mq_update_nr_hw_queues if the
> socket connection count does not match nbd->tag_set->nr_hw_queues.
>
> The nbd_start_device function can be invoked through either the ioctl or
> netlink paths. The ioctl path only allows reusing an existing inactivate
> nbd device, there is nothing more we can do to prevent the queue freeze
> since the old nbd->tag_set->nr_hw_queues may not match the new socket
> connection count. Similarly, the netlink path can reuse a preferred
> inactivate nbd device, and again, we cannot do more in this scenario.
> However, the netlink path can also add a new nbd device using
> nbd_dev_add. In this case, we can obtain the new number of socket
> connections, and by adding a new argument representing the expected
> nr_hw_queues in nbd_dev_add, we can ensure the queue freeze is avoided
> for this situation.
>
> Signed-off-by: Yang Erkun<yangerkun@huawei.com>
> ---
>   drivers/block/nbd.c | 13 +++++++++----
>   1 file changed, 9 insertions(+), 4 deletions(-)
Reviewed-by: Yu Kuai <yukuai@fygo.io>

-- 
Thanks,
Kuai

[PATCH v5 8/8] nbd: add nr_hw_queues module parameter for pre-created devices

[Yang Erkun]

The function blk_mq_update_nr_hw_queues in nbd_start_device may causes a
queue freeze. The previous commit addressed this issue for newly created
nbd device via setting the expected nr_hw_queues in nbd_dev_add.
However, when reusing an old inactive nbd device, the queue freeze can
still occur when old nbd->tag_set->nr_hw_queues not match the new socket
connection count. Inactive nbd devices can originate from two sources:
loading the nbd module with nbds_max, which sets the default nr_hw_queues
to 1, and the netlink method, which sets nr_hw_queues according to the
expected number of socket connections. For the first case, we can add a
module parameter to allow changing the default nr_hw_queues. This way,
users who know their expected number of connections can prevent queue
freezes on pre-created devices via nbds_max.

Before this patchset:
real    0m2.195s
user    0m0.005s
sys     0m0.022s

After this patchset:
real    0m0.090s
user    0m0.004s
sys     0m0.018s

Signed-off-by: Yang Erkun <yangerkun@huawei.com>
---
 drivers/block/nbd.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 6369a409e10c..2194c7d7b2ea 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -166,6 +166,7 @@ static struct dentry *nbd_dbg_dir;
 
 static unsigned int nbds_max = 16;
 static int max_part = 16;
+static int nr_hw_queues = 1;
 static int part_shift;
 
 static int nbd_dev_dbg_init(struct nbd_device *nbd);
@@ -2740,8 +2741,10 @@ static int __init nbd_init(void)
 	}
 	nbd_dbg_init();
 
+	if (nr_hw_queues < 1)
+		nr_hw_queues = 1;
 	for (i = 0; i < nbds_max; i++)
-		nbd_dev_add(i, 1, 1);
+		nbd_dev_add(i, 1, nr_hw_queues);
 	return 0;
 }
 
@@ -2802,3 +2805,6 @@ module_param(nbds_max, int, 0444);
 MODULE_PARM_DESC(nbds_max, "number of network block devices to initialize (default: 16)");
 module_param(max_part, int, 0444);
 MODULE_PARM_DESC(max_part, "number of partitions per device (default: 16)");
+module_param(nr_hw_queues, int, 0444);
+MODULE_PARM_DESC(nr_hw_queues,
+"number of hardware queues for devices pre-created at module load (default: 1). ");
-- 
2.52.0

Re: [PATCH v5 8/8] nbd: add nr_hw_queues module parameter for pre-created devices

[yu kuai]

Hi,

在 2026/7/30 16:20, Yang Erkun 写道:
> The function blk_mq_update_nr_hw_queues in nbd_start_device may causes a
> queue freeze. The previous commit addressed this issue for newly created
> nbd device via setting the expected nr_hw_queues in nbd_dev_add.
> However, when reusing an old inactive nbd device, the queue freeze can
> still occur when old nbd->tag_set->nr_hw_queues not match the new socket
> connection count. Inactive nbd devices can originate from two sources:
> loading the nbd module with nbds_max, which sets the default nr_hw_queues
> to 1, and the netlink method, which sets nr_hw_queues according to the
> expected number of socket connections. For the first case, we can add a
> module parameter to allow changing the default nr_hw_queues. This way,
> users who know their expected number of connections can prevent queue
> freezes on pre-created devices via nbds_max.
>
> Before this patchset:
> real    0m2.195s
> user    0m0.005s
> sys     0m0.022s
>
> After this patchset:
> real    0m0.090s
> user    0m0.004s
> sys     0m0.018s
>
> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
> ---
>   drivers/block/nbd.c | 8 +++++++-
>   1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 6369a409e10c..2194c7d7b2ea 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -166,6 +166,7 @@ static struct dentry *nbd_dbg_dir;
>   
>   static unsigned int nbds_max = 16;
>   static int max_part = 16;
> +static int nr_hw_queues = 1;
>   static int part_shift;
>   
>   static int nbd_dev_dbg_init(struct nbd_device *nbd);
> @@ -2740,8 +2741,10 @@ static int __init nbd_init(void)
>   	}
>   	nbd_dbg_init();
>   
> +	if (nr_hw_queues < 1)
> +		nr_hw_queues = 1;
>   	for (i = 0; i < nbds_max; i++)
> -		nbd_dev_add(i, 1, 1);
> +		nbd_dev_add(i, 1, nr_hw_queues);

I still feel the module param name nr_hw_queues really confusing. How about rename it like
pre_defined_connections? And add comments here that user should config the exact number of
connections to avoid queue freeze from nbd_start_device().

>   	return 0;
>   }
>   
> @@ -2802,3 +2805,6 @@ module_param(nbds_max, int, 0444);
>   MODULE_PARM_DESC(nbds_max, "number of network block devices to initialize (default: 16)");
>   module_param(max_part, int, 0444);
>   MODULE_PARM_DESC(max_part, "number of partitions per device (default: 16)");
> +module_param(nr_hw_queues, int, 0444);
> +MODULE_PARM_DESC(nr_hw_queues,
> +"number of hardware queues for devices pre-created at module load (default: 1). ");

-- 
Thanks,
Kuai

Re: [PATCH v5 8/8] nbd: add nr_hw_queues module parameter for pre-created devices

[yangerkun]

在 2026/8/2 20:21, yu kuai 写道:
> Hi,
> 
> 在 2026/7/30 16:20, Yang Erkun 写道:
>> The function blk_mq_update_nr_hw_queues in nbd_start_device may causes a
>> queue freeze. The previous commit addressed this issue for newly created
>> nbd device via setting the expected nr_hw_queues in nbd_dev_add.
>> However, when reusing an old inactive nbd device, the queue freeze can
>> still occur when old nbd->tag_set->nr_hw_queues not match the new socket
>> connection count. Inactive nbd devices can originate from two sources:
>> loading the nbd module with nbds_max, which sets the default nr_hw_queues
>> to 1, and the netlink method, which sets nr_hw_queues according to the
>> expected number of socket connections. For the first case, we can add a
>> module parameter to allow changing the default nr_hw_queues. This way,
>> users who know their expected number of connections can prevent queue
>> freezes on pre-created devices via nbds_max.
>>
>> Before this patchset:
>> real    0m2.195s
>> user    0m0.005s
>> sys     0m0.022s
>>
>> After this patchset:
>> real    0m0.090s
>> user    0m0.004s
>> sys     0m0.018s
>>
>> Signed-off-by: Yang Erkun <yangerkun@huawei.com>
>> ---
>>    drivers/block/nbd.c | 8 +++++++-
>>    1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>> index 6369a409e10c..2194c7d7b2ea 100644
>> --- a/drivers/block/nbd.c
>> +++ b/drivers/block/nbd.c
>> @@ -166,6 +166,7 @@ static struct dentry *nbd_dbg_dir;
>>    
>>    static unsigned int nbds_max = 16;
>>    static int max_part = 16;
>> +static int nr_hw_queues = 1;
>>    static int part_shift;
>>    
>>    static int nbd_dev_dbg_init(struct nbd_device *nbd);
>> @@ -2740,8 +2741,10 @@ static int __init nbd_init(void)
>>    	}
>>    	nbd_dbg_init();
>>    
>> +	if (nr_hw_queues < 1)
>> +		nr_hw_queues = 1;
>>    	for (i = 0; i < nbds_max; i++)
>> -		nbd_dev_add(i, 1, 1);
>> +		nbd_dev_add(i, 1, nr_hw_queues);
> 
> I still feel the module param name nr_hw_queues really confusing. How about rename it like
> pre_defined_connections? And add comments here that user should config the exact number of
> connections to avoid queue freeze from nbd_start_device().

OK, will do it next version. Thanks a lot for your review for this series!

> 
>>    	return 0;
>>    }
>>    
>> @@ -2802,3 +2805,6 @@ module_param(nbds_max, int, 0444);
>>    MODULE_PARM_DESC(nbds_max, "number of network block devices to initialize (default: 16)");
>>    module_param(max_part, int, 0444);
>>    MODULE_PARM_DESC(max_part, "number of partitions per device (default: 16)");
>> +module_param(nr_hw_queues, int, 0444);
>> +MODULE_PARM_DESC(nr_hw_queues,
>> +"number of hardware queues for devices pre-created at module load (default: 1). ");
>