* [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect
@ 2026-09-22 20:07 Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte
Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
allocation and startup"), I/O queue allocation and start steps
(including authentication) can be parallelized for nvme-tcp as well.
Patch 1 fixes a pre-existing bug: nvme_tcp_alloc_queue() used
current->nsproxy->net_ns to create sockets during initial connect
while it used kernel's init_net for reconnect and error recovery
paths. The fix captures the caller's network namespace at controller
allocation time.
Patch 2 is a preparatory refactor: it splits the queue-count
negotiation and TLS PSK checks into helpers so that the individual
steps can be called directly from nvme_tcp_configure_io_queues(). No
functional change.
Patch 3 has the async implementation: it fans the per-queue alloc and
start out over an async domain, serializes the per-cpu queue-count in
nvme_tcp_set_queue_io_cpu() that can now run concurrently, and uses
ctrl->net for all socket creation.
Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
time reduced from 61ms to 11ms.
blktests - nvme suite, TCP transport (nvme_trtype=tcp)
Tested with:
- Ubuntu 22.04 LTS (Jammy Jellyfish), x86_64, KubeVirt VM
- nvme-cli v2.13 / libnvme v1.14 (built with libkeyutils)
- ktls-utils tag ktls-utils-1.4.0 (c63c04b) (tlshd) linked
against gnutls 3.8.5
---
v4:
- Moved conditional from nvme_tcp_configure_io_queues() to
nvme_tcp_setup_io_queues().
v3:
- Added patch 1 fixing a network namespace bug: reconnect and error
recovery paths were creating sockets in init_net instead of the
controller's namespace. Patch 3 now uses ctrl->net (from patch 1)
instead of hardcoding init_net.
- Patches renumbered accordingly (v2 patch 1->2, patch 2->3).
v2:
- Resending the entire series to fix missing numbering in the
original patch 2 subject line. No functional code changes.
v1: https://lore.kernel.org/20260824222939.301887-1-sgogte@purestorage.com/
v2: https://lore.kernel.org/20260824225648.316962-1-sgogte@purestorage.com/
v3: https://lore.kernel.org/20260910202812.1642832-1-sgogte@purestorage.com/
---
Surabhi Gogte (3):
nvme-tcp: store and use the caller's network namespace
nvme-tcp: refactor I/O queue setup path
nvme-tcp: parallelize I/O queue allocation and startup
drivers/nvme/host/tcp.c | 139 +++++++++++++++++++++++++++-------------
1 file changed, 96 insertions(+), 43 deletions(-)
--
2.55.0
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
@ 2026-09-22 20:07 ` Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path Surabhi Gogte
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte, Hannes Reinecke
nvme_tcp_alloc_queue() creates each socket using
current->nsproxy->net_ns, which is correct for the initial connect that
runs in userspace context. However, the reconnect and error recovery
paths run from a workqueue where current netns is a kernel thread in
init_net, so sockets created during reconnect end up in a different
namespace than the one the controller was originally connected from.
Fix this by capturing the caller's network namespace at controller
allocation time. Use ctrl->net for all socket creation so that every
queue, on initial connect, reset, or reconnect, is created in the same
namespace.
Fixes: 3f2304f8c6d6 ("nvme-tcp: add NVMe over TCP host driver")
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/tcp.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 921934028e0b..f916bcca304f 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -171,6 +171,7 @@ struct nvme_tcp_ctrl {
struct delayed_work connect_work;
struct nvme_tcp_request async_req;
u32 io_queues[HCTX_MAX_TYPES];
+ struct net *net;
};
static struct workqueue_struct *nvme_tcp_wq;
@@ -1858,7 +1859,7 @@ static int nvme_tcp_alloc_queue(struct nvme_ctrl *nctrl, int qid,
queue->cmnd_capsule_len = sizeof(struct nvme_command) +
NVME_TCP_ADMIN_CCSZ;
- ret = sock_create_kern(current->nsproxy->net_ns,
+ ret = sock_create_kern(ctrl->net,
ctrl->addr.ss_family, SOCK_STREAM,
IPPROTO_TCP, &queue->sock);
if (ret) {
@@ -2650,6 +2651,7 @@ static void nvme_tcp_free_ctrl(struct nvme_ctrl *nctrl)
nvmf_free_options(nctrl->opts);
free_ctrl:
+ put_net(ctrl->net);
kfree(ctrl->queues);
kfree(ctrl);
}
@@ -3048,12 +3050,15 @@ static struct nvme_tcp_ctrl *nvme_tcp_alloc_ctrl(struct device *dev,
goto out_free_ctrl;
}
+ ctrl->net = get_net(current->nsproxy->net_ns);
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_tcp_ctrl_ops, 0);
if (ret)
goto out_kfree_queues;
return ctrl;
out_kfree_queues:
+ put_net(ctrl->net);
kfree(ctrl->queues);
out_free_ctrl:
kfree(ctrl);
--
2.55.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
@ 2026-09-22 20:07 ` Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
2026-10-13 15:33 ` [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Keith Busch
3 siblings, 0 replies; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte, Hannes Reinecke
Split the I/O queue setup helpers apart so that the individual steps can
be called directly from nvme_tcp_configure_io_queues():
- Queue count negotiation moves out of nvme_tcp_alloc_io_queues() into a
new nvme_tcp_set_io_queue_count(), leaving the allocator with just the
per-queue allocation loop.
- TLS PSK validation moves out of __nvme_tcp_alloc_io_queues() into a
new nvme_tcp_tls_check_psk().
- nvme_tcp_configure_io_queues() now calls the three steps directly in
the same order as before.
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/tcp.c | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index f916bcca304f..9308feba8a74 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2184,10 +2184,8 @@ static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
return ret;
}
-static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+static int nvme_tcp_tls_check_psk(struct nvme_ctrl *ctrl)
{
- int i, ret;
-
if (nvme_tcp_tls_configured(ctrl)) {
if (ctrl->opts->concat) {
/*
@@ -2209,6 +2207,13 @@ static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
}
}
+ return 0;
+}
+
+static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+{
+ int i, ret;
+
for (i = 1; i < ctrl->queue_count; i++) {
ret = nvme_tcp_alloc_queue(ctrl, i,
ctrl->tls_pskid);
@@ -2225,7 +2230,7 @@ static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
return ret;
}
-static int nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+static int nvme_tcp_set_io_queue_count(struct nvme_ctrl *ctrl)
{
unsigned int nr_io_queues;
int ret;
@@ -2247,14 +2252,22 @@ static int nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
nvmf_set_io_queues(ctrl->opts, nr_io_queues,
to_tcp_ctrl(ctrl)->io_queues);
- return __nvme_tcp_alloc_io_queues(ctrl);
+ return 0;
}
static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
{
int ret, nr_queues;
- ret = nvme_tcp_alloc_io_queues(ctrl);
+ ret = nvme_tcp_set_io_queue_count(ctrl);
+ if (ret)
+ return ret;
+
+ ret = nvme_tcp_tls_check_psk(ctrl);
+ if (ret)
+ return ret;
+
+ ret = __nvme_tcp_alloc_io_queues(ctrl);
if (ret)
return ret;
--
2.55.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path Surabhi Gogte
@ 2026-09-22 20:07 ` Surabhi Gogte
2026-10-18 12:25 ` Shin'ichiro Kawasaki
2026-10-13 15:33 ` [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Keith Busch
3 siblings, 1 reply; 6+ messages in thread
From: Surabhi Gogte @ 2026-09-22 20:07 UTC (permalink / raw)
To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Surabhi Gogte, Hannes Reinecke
Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
allocation and startup"), refactor nvme tcp I/O queue setup to use async
API, combining allocation and startup into a single parallel operation
per queue. This reduces connection and reconnection setup time when
there are delays in establishing connections, which is especially
important for high-core-count hosts.
Key changes:
- Use async API to facilitate parallel calls for io queue setup.
- Add nvme_tcp_queue_setup_ctx for propagating errors from async
workers.
- Remove nvme_tcp_start_io_queues() and __nvme_tcp_alloc_io_queues();
their logic is folded into nvme_tcp_setup_io_queues() and
nvme_tcp_configure_io_queues().
- Allocate the io tag set before the queues so that the queue range is
known, and only set up the reconnect grow case if the queue count
actually increased.
- Serialize the cpu scan and claim in nvme_tcp_set_queue_io_cpu() with a
spinlock, as concurrent callers would otherwise select the same cpu.
The per-cpu counters no longer need to be atomics.
Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
time reduced from 61ms to 11ms.
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
---
drivers/nvme/host/tcp.c | 119 ++++++++++++++++++++++++++--------------
1 file changed, 77 insertions(+), 42 deletions(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 9308feba8a74..ec678ed07af8 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -7,6 +7,7 @@
#include <linux/module.h>
#include <linux/init.h>
#include <linux/slab.h>
+#include <linux/async.h>
#include <linux/err.h>
#include <linux/crc32.h>
#include <linux/nvme-tcp.h>
@@ -54,7 +55,8 @@ MODULE_PARM_DESC(tls_handshake_timeout,
"nvme TLS handshake timeout in seconds (default 10)");
#endif
-static atomic_t nvme_tcp_cpu_queues[NR_CPUS];
+static int nvme_tcp_cpu_queues[NR_CPUS];
+static DEFINE_SPINLOCK(nvme_tcp_cpu_queues_lock);
enum nvme_tcp_send_state {
NVME_TCP_SEND_CMD_PDU = 0,
@@ -154,6 +156,12 @@ struct nvme_tcp_queue {
static DEFINE_MUTEX(nvme_tcp_ctrl_mutex);
static LIST_HEAD_GUARDED(nvme_tcp_ctrl_list, nvme_tcp_ctrl_mutex);
+struct nvme_tcp_queue_setup_ctx {
+ struct nvme_ctrl *ctrl;
+ int qid;
+ int *err;
+};
+
struct nvme_tcp_ctrl {
/* read only in the hot path */
struct nvme_tcp_queue *queues;
@@ -1731,9 +1739,10 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
goto out;
/* Search for the least used cpu from the mq_map */
+ spin_lock(&nvme_tcp_cpu_queues_lock);
io_cpu = WORK_CPU_UNBOUND;
for_each_online_cpu(cpu) {
- int num_queues = atomic_read(&nvme_tcp_cpu_queues[cpu]);
+ int num_queues = nvme_tcp_cpu_queues[cpu];
if (mq_map[cpu] != qid)
continue;
@@ -1744,9 +1753,10 @@ static void nvme_tcp_set_queue_io_cpu(struct nvme_tcp_queue *queue)
}
if (io_cpu != WORK_CPU_UNBOUND) {
queue->io_cpu = io_cpu;
- atomic_inc(&nvme_tcp_cpu_queues[io_cpu]);
+ nvme_tcp_cpu_queues[io_cpu]++;
set_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags);
}
+ spin_unlock(&nvme_tcp_cpu_queues_lock);
out:
dev_dbg(ctrl->ctrl.device, "queue %d: using cpu %d\n",
qid, queue->io_cpu);
@@ -2023,8 +2033,11 @@ static void nvme_tcp_stop_queue_nowait(struct nvme_ctrl *nctrl, int qid)
if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
return;
- if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags))
- atomic_dec(&nvme_tcp_cpu_queues[queue->io_cpu]);
+ if (test_and_clear_bit(NVME_TCP_Q_IO_CPU_SET, &queue->flags)) {
+ spin_lock(&nvme_tcp_cpu_queues_lock);
+ nvme_tcp_cpu_queues[queue->io_cpu]--;
+ spin_unlock(&nvme_tcp_cpu_queues_lock);
+ }
mutex_lock(&queue->queue_lock);
if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
@@ -2131,25 +2144,6 @@ static void nvme_tcp_stop_io_queues(struct nvme_ctrl *ctrl)
nvme_tcp_wait_queue(ctrl, i);
}
-static int nvme_tcp_start_io_queues(struct nvme_ctrl *ctrl,
- int first, int last)
-{
- int i, ret;
-
- for (i = first; i < last; i++) {
- ret = nvme_tcp_start_queue(ctrl, i);
- if (ret)
- goto out_stop_queues;
- }
-
- return 0;
-
-out_stop_queues:
- for (i--; i >= first; i--)
- nvme_tcp_stop_queue(ctrl, i);
- return ret;
-}
-
static int nvme_tcp_alloc_admin_queue(struct nvme_ctrl *ctrl)
{
int ret;
@@ -2210,22 +2204,67 @@ static int nvme_tcp_tls_check_psk(struct nvme_ctrl *ctrl)
return 0;
}
-static int __nvme_tcp_alloc_io_queues(struct nvme_ctrl *ctrl)
+static void nvme_tcp_setup_queue_async(void *data, async_cookie_t cookie)
{
- int i, ret;
+ struct nvme_tcp_queue_setup_ctx *ctx = data;
+ struct nvme_ctrl *ctrl = ctx->ctrl;
+ int ret;
- for (i = 1; i < ctrl->queue_count; i++) {
- ret = nvme_tcp_alloc_queue(ctrl, i,
- ctrl->tls_pskid);
- if (ret)
- goto out_free_queues;
+ ret = nvme_tcp_alloc_queue(ctrl, ctx->qid, ctrl->tls_pskid);
+ if (ret)
+ goto out_err;
+
+ ret = nvme_tcp_start_queue(ctrl, ctx->qid);
+ if (ret)
+ goto out_err;
+
+ return;
+
+out_err:
+ WRITE_ONCE(*ctx->err, ret);
+}
+
+static int nvme_tcp_setup_io_queues(struct nvme_ctrl *ctrl, unsigned int first,
+ unsigned int last)
+{
+ ASYNC_DOMAIN_EXCLUSIVE(queue_domain);
+ struct nvme_tcp_queue_setup_ctx *ctxs;
+ int nr_queues = last - first;
+ int err = 0, i, ret;
+
+ if (first >= last)
+ return 0;
+
+ ctxs = kmalloc_objs(*ctxs, nr_queues);
+ if (!ctxs)
+ return -ENOMEM;
+
+ for (i = 0; i < nr_queues; i++) {
+ ctxs[i].ctrl = ctrl;
+ ctxs[i].qid = first + i;
+ ctxs[i].err = &err;
+ async_schedule_domain(nvme_tcp_setup_queue_async, &ctxs[i],
+ &queue_domain);
}
+ async_synchronize_full_domain(&queue_domain);
+ kfree(ctxs);
+
+ ret = READ_ONCE(err);
+ if (ret)
+ goto out_free_queues;
+
return 0;
out_free_queues:
- for (i--; i >= 1; i--)
- nvme_tcp_free_queue(ctrl, i);
+ for (i = first; i < last; i++) {
+ struct nvme_tcp_queue *queue = &to_tcp_ctrl(ctrl)->queues[i];
+
+ if (test_bit(NVME_TCP_Q_LIVE, &queue->flags))
+ nvme_tcp_stop_queue(ctrl, i);
+ if (test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+ nvme_tcp_free_queue(ctrl, i);
+ }
return ret;
}
@@ -2267,10 +2306,6 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
if (ret)
return ret;
- ret = __nvme_tcp_alloc_io_queues(ctrl);
- if (ret)
- return ret;
-
if (new) {
ret = nvme_alloc_io_tag_set(ctrl, &to_tcp_ctrl(ctrl)->tag_set,
&nvme_tcp_mq_ops,
@@ -2281,12 +2316,12 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
}
/*
- * Only start IO queues for which we have allocated the tagset
+ * Only setup IO queues for which we have allocated the tagset
* and limited it to the available queues. On reconnects, the
* queue number might have changed.
*/
nr_queues = min(ctrl->tagset->nr_hw_queues + 1, ctrl->queue_count);
- ret = nvme_tcp_start_io_queues(ctrl, 1, nr_queues);
+ ret = nvme_tcp_setup_io_queues(ctrl, 1, nr_queues);
if (ret)
goto out_cleanup_connect_q;
@@ -2310,9 +2345,9 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
/*
* If the number of queues has increased (reconnect case)
- * start all new queues now.
+ * setup all new queues now.
*/
- ret = nvme_tcp_start_io_queues(ctrl, nr_queues,
+ ret = nvme_tcp_setup_io_queues(ctrl, nr_queues,
ctrl->tagset->nr_hw_queues + 1);
if (ret)
goto out_wait_freeze_timed_out;
@@ -3157,7 +3192,7 @@ static int __init nvme_tcp_init_module(void)
return -ENOMEM;
for_each_possible_cpu(cpu)
- atomic_set(&nvme_tcp_cpu_queues[cpu], 0);
+ nvme_tcp_cpu_queues[cpu] = 0;
nvmf_register_transport(&nvme_tcp_transport);
return 0;
--
2.55.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
` (2 preceding siblings ...)
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
@ 2026-10-13 15:33 ` Keith Busch
3 siblings, 0 replies; 6+ messages in thread
From: Keith Busch @ 2026-10-13 15:33 UTC (permalink / raw)
To: Surabhi Gogte
Cc: Jens Axboe, Christoph Hellwig, Sagi Grimberg, linux-nvme,
linux-kernel, mkhalfella, randyj, adailey
On Tue, Sep 22, 2026 at 02:07:15PM -0600, Surabhi Gogte wrote:
> Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
> allocation and startup"), I/O queue allocation and start steps
> (including authentication) can be parallelized for nvme-tcp as well.
>
> Patch 1 fixes a pre-existing bug: nvme_tcp_alloc_queue() used
> current->nsproxy->net_ns to create sockets during initial connect
> while it used kernel's init_net for reconnect and error recovery
> paths. The fix captures the caller's network namespace at controller
> allocation time.
>
> Patch 2 is a preparatory refactor: it splits the queue-count
> negotiation and TLS PSK checks into helpers so that the individual
> steps can be called directly from nvme_tcp_configure_io_queues(). No
> functional change.
>
> Patch 3 has the async implementation: it fans the per-queue alloc and
> start out over an async domain, serializes the per-cpu queue-count in
> nvme_tcp_set_queue_io_cpu() that can now run concurrently, and uses
> ctrl->net for all socket creation.
>
> Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
> time reduced from 61ms to 11ms.
Thanks, applied to nvme-7.4.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
@ 2026-10-18 12:25 ` Shin'ichiro Kawasaki
0 siblings, 0 replies; 6+ messages in thread
From: Shin'ichiro Kawasaki @ 2026-10-18 12:25 UTC (permalink / raw)
To: Surabhi Gogte
Cc: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg,
linux-nvme, linux-kernel, mkhalfella, randyj, adailey,
Hannes Reinecke
On Sep 22, 2026 / 14:07, Surabhi Gogte wrote:
> Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
> allocation and startup"), refactor nvme tcp I/O queue setup to use async
> API, combining allocation and startup into a single parallel operation
> per queue. This reduces connection and reconnection setup time when
> there are delays in establishing connections, which is especially
> important for high-core-count hosts.
>
> Key changes:
> - Use async API to facilitate parallel calls for io queue setup.
> - Add nvme_tcp_queue_setup_ctx for propagating errors from async
> workers.
> - Remove nvme_tcp_start_io_queues() and __nvme_tcp_alloc_io_queues();
> their logic is folded into nvme_tcp_setup_io_queues() and
> nvme_tcp_configure_io_queues().
> - Allocate the io tag set before the queues so that the queue range is
> known, and only set up the reconnect grow case if the queue count
> actually increased.
> - Serialize the cpu scan and claim in nvme_tcp_set_queue_io_cpu() with a
> spinlock, as concurrent callers would otherwise select the same cpu.
> The per-cpu counters no longer need to be atomics.
>
> Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
> time reduced from 61ms to 11ms.
Recently, blktests-CI runs for the kernel at block/for-next branch started
reporting failures of the test case nvme/004 (with tcp transport) and md/001.
The kernel reported a WARN related to lock->magic, and it caused the failures.
FYI, here I share the failure run log at md/001 [1] and dmesg at the failure.
The md/001 confirms a bug fix in md driver. The bug was triggered when a
nvme device with tcp transport is used as its backend. Then the test case
exercises the nvme-tcp driver, which this patch touches.
I bisected and found that this commit is the trigger of the failures. When I
revert this commit from the block/for-next branch tip with git hash
34fbfb9d8d1f, the failure goes away.
Surabhi, could you take a look in the failure?
[1] failure of md/001
md/001 (Raid with bitmap on tcp nvmet with opt-io-size over bitmap size) [failed]
runtime 3.021s ... 2.970s
something found in dmesg:
[ 79.786635] [ T966] run blktests md/001 at 2026-10-18 20:35:29.594227680
[ 80.157789] [ T984] brd: module loaded
[ 80.566933] [ T1008] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
[ 80.637888] [ T1011] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
[ 80.925883] [ T312] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
[ 80.940866] [ T1018] nvme nvme5: creating 4 I/O queues.
[ 80.942243] [ T1018] nvme nvme5: mapped 4/0/0 default/read/poll queues.
[ 80.942689] [ T1018] ------------[ cut here ]------------
[ 80.942879] [ T1018] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[ 80.942882] [ T1018] WARNING: kernel/locking/mutex.c:625 at __mutex_lock+0x1008/0x2510, CPU#0: nvme/1018
...
(See '/home/shin/Blktests/blktests/results/nodev/md/001.dmesg' for the entire message)
[2] dmesg during md/001
2026-10-17T22:40:06Z {detected_level="unknown"} [ 2568.828164] run blktests md/001 at 2026-10-17 22:40:06.674885632
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.162185] brd: module loaded
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.587993] nvmet: adding nsid 1 to subsystem blktests-subsystem-1
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.629198] nvmet_tcp: enabling port 0 (127.0.0.1:4420)
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.848587] nvmet: Created nvm controller 1 for subsystem blktests-subsystem-1 for NQN nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349.
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.875225] nvme nvme2: creating 4 I/O queues.
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.878288] nvme nvme2: mapped 4/0/0 default/read/poll queues.
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.883211] ------------[ cut here ]------------
2026-10-17T22:40:07Z {detected_level="debug"} [ 2569.883660] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
2026-10-17T22:40:07Z {detected_level="warn"} [ 2569.883666] WARNING: kernel/locking/mutex.c:625 at __mutex_lock+0xb73/0x2350, CPU#2: nvme/55686
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.887533] Modules linked in: nvme_tcp(E) nvme_fabrics(E) nvmet_tcp(E) nvmet(E) brd(E) raid1(E) dm_dust(E) loop(E) xfs(E) tun(E) nvme(E) iTCO_wdt(E) intel_rapl_msr(E) intel_pmc_bxt(E) intel_rapl_common(E) pcspkr(E) i2c_i801(E) i2c_smbus(E) nvme_core(E) nvme_keyring(E) virtio_balloon(E) nvme_auth(E) virtio_scsi(E) lpc_ich(E) bochs(E) serio_raw(E) qemu_fw_cfg(E) vmw_vsock_virtio_transport(E) vmw_vsock_virtio_transport_common(E) vsock(E)
2026-10-17T22:40:07Z {detected_level="debug"} [ 2569.887693] Unloaded tainted modules: nvme_tcp(E):1 nvmet(E):63 nvme_fabrics(E):63 nvme_loop(E):63 loop(E):14 ublk_drv(E):2 null_blk(E):37 scsi_debug(E):53 amd_atl(E):1 edac_mce_amd(E):1 bpf_preload(E):1 [last unloaded: scsi_debug(E)]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.899624] CPU: 2 UID: 0 PID: 55686 Comm: nvme Tainted: G E 7.3.0-rc7 #1 PREEMPT(lazy)
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.902229] Tainted: [E]=UNSIGNED_MODULE
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.903507] Hardware name: KubeVirt None/RHEL, BIOS 1.16.3-4.el9 04/01/2014
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.905532] RIP: 0010:__mutex_lock+0xb7a/0x2350
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.907336] Code: 38 d0 7c 08 84 d2 0f 85 52 17 00 00 44 8b 05 8d af 8d 02 45 85 c0 0f 85 0a f6 ff ff 48 8d 3d ad cc 8e 02 48 c7 c6 c0 fd 2f 85 <67> 48 0f b9 3a e9 f2 f5 ff ff 4d 8d a7 e0 0e 00 00 4c 89 e7 e8 3d
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.912320] RSP: 0018:ffff8881438c7370 EFLAGS: 00010246
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.914185] RAX: 0000000000000003 RBX: ffff8881face6460 RCX: 1ffffffff0f35008
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.916368] RDX: 0000000000000000 RSI: ffffffff852ffdc0 RDI: ffffffff879b9d70
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.918675] RBP: ffff8881438c74e0 R08: 0000000000000000 R09: fffffbfff0f35008
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.920803] R10: ffffffff879a8047 R11: ffff88812d120008 R12: 0000000000000000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.923039] R13: 0000000000000000 R14: dffffc0000000000 R15: ffff88812d3a0000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.925258] FS: 00007fd431b6b640(0000) GS:ffff8882ac2ea000(0000) knlGS:0000000000000000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.927640] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.929481] CR2: 00007f50088d0c1c CR3: 000000010d0d4006 CR4: 0000000000770ef0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.932053] PKRU: 55555554
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.933507] Call Trace:
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.934813] <TASK>
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.936625] ? nvme_tcp_init_request+0xc8/0x300 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.938805] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.940675] ? __alloc_frozen_pages_noprof+0x543/0x7c0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.942464] ? __pfx___mutex_lock+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.944123] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.945814] ? alloc_pages_node_noprof+0xae/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.947474] ? nvme_tcp_init_request+0xc8/0x300 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.949268] nvme_tcp_init_request+0xc8/0x300 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.951141] blk_mq_alloc_rqs+0x3fc/0x6f0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.952747] blk_mq_alloc_map_and_rqs+0x168/0x320
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.954450] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.956260] blk_mq_alloc_set_map_and_rqs+0x1be/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.958088] blk_mq_alloc_tag_set+0x59a/0xc40
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.959710] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.961321] ? dev_printk_emit+0xbe/0xf3
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.962736] nvme_alloc_io_tag_set+0x37d/0x6a0 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.964446] ? __pfx_nvme_alloc_io_tag_set+0x10/0x10 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.966345] ? percpu_ref_put_many.constprop.0+0x7f/0x1b0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.968109] ? __nvme_submit_sync_cmd+0x1e2/0x320 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.970040] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.971714] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="info"} [ 2569.973339] ? _dev_info+0xe6/0x11a
2026-10-17T22:40:07Z {detected_level="info"} [ 2569.974760] ? __pfx__dev_info+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.976153] ? __pfx_nvme_set_queue_count+0x10/0x10 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.977938] ? lock_acquire.part.0+0xc7/0x240
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.979582] nvme_tcp_configure_io_queues.cold+0x27f/0x8ee [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.981436] ? __pfx_nvme_enable_ctrl+0x10/0x10 [nvme_core]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.983132] ? lockdep_hardirqs_on_prepare.part.0+0x9b/0x140
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.984762] ? __pfx_nvme_tcp_configure_io_queues+0x10/0x10 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.986525] nvme_tcp_setup_ctrl+0x281/0x5f0 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.988113] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.989665] nvme_tcp_create_ctrl+0xf6/0x120 [nvme_tcp]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.991236] nvmf_create_ctrl+0x34e/0x920 [nvme_fabrics]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.992832] nvmf_dev_write+0xd5/0x180 [nvme_fabrics]
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.994357] vfs_write+0x1c7/0x1060
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.995622] ? __pfx_vfs_write+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.996900] ksys_write+0xf9/0x1d0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.998093] ? __pfx_ksys_write+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2569.999349] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.000692] ? lockdep_hardirqs_on_prepare.part.0+0x9b/0x140
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.002249] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.003675] ? trace_hardirqs_on+0x18/0x160
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.005046] do_syscall_64+0xef/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.006137] ? kmem_cache_free+0x143/0x600
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.007405] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.008722] ? do_sys_openat2+0x107/0x180
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.009946] ? __pfx_do_sys_openat2+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.011180] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.012539] ? fput_close_sync+0xe5/0x1a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.013752] ? __pfx_fput_close_sync+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.015062] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.016455] ? __x64_sys_openat+0x108/0x1d0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.017722] ? __pfx___x64_sys_openat+0x10/0x10
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.019017] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.020393] ? do_syscall_64+0x1d7/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.021623] ? do_syscall_64+0x1b9/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.022876] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.024209] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.025581] ? do_syscall_64+0x1b9/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.026867] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.028225] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.029718] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.031169] ? do_syscall_64+0x1d7/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.032408] ? do_syscall_64+0x1b9/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.033704] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.035099] ? trace_hardirqs_on_prepare+0x119/0x150
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.036605] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.037956] ? do_syscall_64+0x1d7/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.039181] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.040575] ? __lock_release.isra.0+0x5b/0x180
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.041947] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.043389] ? do_user_addr_fault+0x8d3/0xe80
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.044575] ? irqentry_exit+0xb8/0x810
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.045690] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.046990] ? do_syscall_64+0x2d/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.048067] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.049302] ? srso_alias_return_thunk+0x5/0xfbef5
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.050545] ? do_syscall_64+0x5c/0x5a0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.051649] entry_SYSCALL_64_after_hwframe+0x76/0x7e
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.052916] RIP: 0033:0x7fd431c3d56e
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.053948] Code: 4d 89 d8 e8 d4 bd 00 00 4c 8b 5d f8 41 8b 93 08 03 00 00 59 5e 48 83 f8 fc 74 11 c9 c3 0f 1f 80 00 00 00 00 48 8b 45 10 0f 05 <c9> c3 83 e2 39 83 fa 08 75 e7 e8 03 ff ff ff 0f 1f 00 f3 0f 1e fa
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.057937] RSP: 002b:00007ffdaa469710 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.059733] RAX: ffffffffffffffda RBX: 00005609fa418740 RCX: 00007fd431c3d56e
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.061425] RDX: 00000000000000cf RSI: 00005609fa418740 RDI: 0000000000000003
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.063121] RBP: 00007ffdaa469720 R08: 0000000000000000 R09: 0000000000000000
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.064826] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000003
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.066523] R13: 00005609fa4175b0 R14: 00000000000000cf R15: 00005609fa4175b0
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.068235] </TASK>
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.069130] irq event stamp: 11409
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.070225] hardirqs last enabled at (11409): [<ffffffff850e58f5>] _raw_spin_unlock_irqrestore+0x45/0x60
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.072350] hardirqs last disabled at (11408): [<ffffffff817db728>] __raw_spin_lock_irqsave+0x58/0x60
2026-10-17T22:40:07Z {detected_level="unknown"} [ 2570.074391] softirqs last enabled at (11372): [<ffffffff815ffc88>] handle_softirqs+0x638/0x880
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.076357] softirqs last disabled at (11365): [<ffffffff8160003f>] __irq_exit_rcu+0x15f/0x290
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.078305] ---[ end trace 0000000000000000 ]---
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.084017] nvme nvme2: new ctrl: NQN "blktests-subsystem-1", addr 127.0.0.1:4420, hostnqn: nqn.2014-08.org.nvmexpress:uuid:0f01fb42-9f7f-4856-b0b3-51e60b8de349
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.400433] md127: array will not be assembled in old kernels that lack configurable LBS support (<= 6.18)
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.401014] md/raid1:md127: active with 1 out of 2 mirrors
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.404015] md127: detected capacity change from 0 to 2093056
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.559964] md127: detected capacity change from 2093056 to 0
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.560710] md: md127 stopped.
2026-10-17T22:40:08Z {detected_level="unknown"} [ 2570.632174] nvme nvme2: Removing ctrl: NQN "blktests-subsystem-1"
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2571.118121] brd: module unloaded
2026-10-17T22:40:09Z {detected_level="debug"} [ 2572.039779] scsi_debug:sdebug_driver_probe: scsi_debug: trim poll_queues to 0. poll_q/nr_hw = (0/1)
2026-10-17T22:40:09Z {detected_level="debug"} [ 2572.040354] scsi host66: scsi_debug: version 0191 [20210520]
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.040354] dev_size_mb=8, opts=0x0, submit_queues=1, statistics=0
2026-10-17T22:40:09Z {detected_level="debug"} [ 2572.046424] scsi 66:0:0:0: Direct-Access Linux scsi_debug 0191 PQ: 0 ANSI: 7
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.053271] scsi 66:0:0:0: Power-on or device reset occurred
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.063350] sd 66:0:0:0: [sda] 16384 512-byte logical blocks: (8.39 MB/8.00 MiB)
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.063603] sd 66:0:0:0: Attached scsi generic sg1 type 0
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.064965] sd 66:0:0:0: [sda] Write Protect is off
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.065470] sd 66:0:0:0: [sda] Mode Sense: 73 00 10 08
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.068116] sd 66:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
2026-10-17T22:40:09Z {detected_level="unknown"} [ 2572.075669] sd 66:0:0:0: [sda] permanent stream count = 5
2026-10-17T22:40:10Z {detected_level="unknown"} [ 2572.080471] sd 66:0:0:0: [sda] Preferred minimum I/O size 512 bytes
2026-10-17T22:40:10Z {detected_level="unknown"} [ 2572.082032] sd 66:0:0:0: [sda] Optimal transfer size 524288 bytes
2026-10-17T22:40:10Z {detected_level="unknown"} [ 2572.131009] sd 66:0:0:0: [sda] Attached SCSI disk
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2026-10-18 12:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-22 20:07 [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 1/3] nvme-tcp: store and use the caller's network namespace Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 2/3] nvme-tcp: refactor I/O queue setup path Surabhi Gogte
2026-09-22 20:07 ` [PATCH v4 3/3] nvme-tcp: parallelize I/O queue allocation and startup Surabhi Gogte
2026-10-18 12:25 ` Shin'ichiro Kawasaki
2026-10-13 15:33 ` [PATCH v4 0/3] nvme-tcp: parallelize I/O setup queue in connect Keith Busch
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.