* [PATCH v3 0/2] nvme: fabrics and target fixes @ 2026-11-16 8:12 Guixin Liu 2026-11-16 8:12 ` [PATCH v3 1/2] nvme-fc: fix use-after-free in nvme_fc_match_disconn_ls() Guixin Liu 2026-11-16 8:12 ` [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction Guixin Liu 0 siblings, 2 replies; 5+ messages in thread From: Guixin Liu @ 2026-11-16 8:12 UTC (permalink / raw) To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg, Nilay Shroff, Daniel Wagner, Hannes Reinecke Cc: linux-nvme Two remaining fixes from the previous rounds of this series: - a use-after-free in nvme_fc_match_disconn_ls(): no iterator is sound across the rport lock drop inside the loop, so the loop is reworked to run entirely under the rport lock and to take a reference only for the match; reproduced under KASAN through fcloop - a data race on the shared authentication transaction counter, reworked per review to a per-queue counter, dropping the shared ctrl->transaction field entirely The other three patches from v2 are already resolved: the GFP_KERNEL persistent reservation change and the tbkas configfs locking are in nvme-7.4 now, and the reconnect_delay fix was superseded by an earlier identical patch posted by someone else. Both patches are verified on an initiator/target VM pair: the fc loop is compile tested and exercised under KASAN through fcloop with six controllers on one remote port across tbkas force-reconnect cycles and concurrent disconnects with no reports; the DH-HMAC-CHAP authenticated session including I/O covers the auth path. Changes since v2: - rework the nvme-auth fix to a per-queue counter (Keith Busch), drop the now unused shared field, and add the Fixes tag (sashiko); the Reviewed-by on v2 covered the atomic_t variant and is dropped - resend the unchanged nvme-fc patch, still awaiting review - drop the reconnect_delay patch (superseded), the GFP_KERNEL patch (applied) and the tbkas patch (applied) Link: https://lore.kernel.org/all/20261109022646.290792-1-kanie@linux.alibaba.com/ Guixin Liu (2): nvme-fc: fix use-after-free in nvme_fc_match_disconn_ls() nvme-auth: fix data race on ctrl->transaction drivers/nvme/host/auth.c | 3 +-- drivers/nvme/host/fc.c | 22 ++++++++++++---------- drivers/nvme/host/nvme.h | 1 - 3 files changed, 13 insertions(+), 13 deletions(-) -- 2.43.7 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v3 1/2] nvme-fc: fix use-after-free in nvme_fc_match_disconn_ls() 2026-11-16 8:12 [PATCH v3 0/2] nvme: fabrics and target fixes Guixin Liu @ 2026-11-16 8:12 ` Guixin Liu 2026-11-16 8:12 ` [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction Guixin Liu 1 sibling, 0 replies; 5+ messages in thread From: Guixin Liu @ 2026-11-16 8:12 UTC (permalink / raw) To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg, Nilay Shroff, Daniel Wagner, Hannes Reinecke Cc: linux-nvme Commit b71cbcf7d170e ("nvme-fc: don't hold rport lock when putting ctrl") dropped the rport lock inside the loop to put the reference of a non-matching controller, because nvme_fc_ctrl_free() takes the rport lock to unlink the controller. But no iterator is sound across that drop: with list_for_each_entry_safe() the prefetched next entry can be freed in the window, and with a plain iterator the loop's own nvme_fc_ctrl_put() can be the final reference, freeing the current entry before the iterator reads its ->next pointer. The reference dance is unnecessary in the first place. An entry cannot leave ctrl_list while rport->lock is held, as the final put unlinks it under that very lock, so association_id can be inspected under ctrl->lock without holding a reference. Take a reference only for the matching controller, still under the rport lock, and hand it to the caller as before. The loop then no longer needs to drop the lock at all, which also removes the reason the lock was dropped for in the first place. Fixes: b71cbcf7d170e ("nvme-fc: don't hold rport lock when putting ctrl") Cc: stable@vger.kernel.org Signed-off-by: Guixin Liu <kanie@linux.alibaba.com> --- Reproduced under KASAN in a VM through fcloop with several controllers on one remote port and a tbkas force-reconnect racing concurrent controller disconnects; the lock-drop window was widened with a test-only mdelay(200) to make the microsecond window deterministic: BUG: KASAN: slab-use-after-free in nvme_fc_match_disconn_ls Read of size 8 by task kworker Workqueue: events nvme_fc_handle_ls_rqst_work Call Trace: nvme_fc_match_disconn_ls nvme_fc_ls_disconnect_assoc nvme_fc_handle_ls_rqst_work process_one_work The freed object was the prefetched next entry, allocated by nvme_fc_alloc_ctrl() on connect and freed by nvme_free_ctrl() on the concurrent teardown. The loop increment dereferences it before nvme_fc_ctrl_get() gets a chance to reject it, which is why the get-unless-zero check in the loop body does not help. The reworked loop does not drop the lock, so no such window is left to widen. --- drivers/nvme/host/fc.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index 2cbd744ffd22..ce4598dd227f 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -1469,29 +1469,31 @@ nvme_fc_match_disconn_ls(struct nvme_fc_rport *rport, { struct fcnvme_ls_disconnect_assoc_rqst *rqst = &lsop->rqstbuf->rq_dis_assoc; - struct nvme_fc_ctrl *ctrl, *tmp, *ret = NULL; + struct nvme_fc_ctrl *ctrl, *ret = NULL; struct nvmefc_ls_rcv_op *oldls = NULL; u64 association_id = be64_to_cpu(rqst->associd.association_id); unsigned long flags; spin_lock_irqsave(&rport->lock, flags); - list_for_each_entry_safe(ctrl, tmp, &rport->ctrl_list, ctrl_list) { - if (!nvme_fc_ctrl_get(ctrl)) - continue; + list_for_each_entry(ctrl, &rport->ctrl_list, ctrl_list) { spin_lock(&ctrl->lock); if (association_id == ctrl->association_id) { - oldls = ctrl->rcv_disconn; - ctrl->rcv_disconn = lsop; - ret = ctrl; + /* + * Entries cannot leave the list while rport->lock is + * held, so they can be inspected without a reference. + * Only the match needs one, for the caller. + */ + if (nvme_fc_ctrl_get(ctrl)) { + oldls = ctrl->rcv_disconn; + ctrl->rcv_disconn = lsop; + ret = ctrl; + } } spin_unlock(&ctrl->lock); if (ret) /* leave the ctrl get reference */ break; - spin_unlock_irqrestore(&rport->lock, flags); - nvme_fc_ctrl_put(ctrl); - spin_lock_irqsave(&rport->lock, flags); } spin_unlock_irqrestore(&rport->lock, flags); -- 2.43.7 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction 2026-11-16 8:12 [PATCH v3 0/2] nvme: fabrics and target fixes Guixin Liu 2026-11-16 8:12 ` [PATCH v3 1/2] nvme-fc: fix use-after-free in nvme_fc_match_disconn_ls() Guixin Liu @ 2026-11-16 8:12 ` Guixin Liu 2026-11-16 13:22 ` Hannes Reinecke 1 sibling, 1 reply; 5+ messages in thread From: Guixin Liu @ 2026-11-16 8:12 UTC (permalink / raw) To: Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg, Nilay Shroff, Daniel Wagner, Hannes Reinecke Cc: linux-nvme nvme_ctrl_auth_work() queues the DH-HMAC-CHAP exchange of every queue on nvme-auth-wq without waiting for them, so multiple nvme_queue_auth_work() instances run concurrently and all increment ctrl->transaction in nvme_queue_auth_work(). The plain increment is a data race, and the lost updates hand the same transaction id to concurrent queues. The counter does not need to be shared at all: the controller only echoes the transaction id within a queue, so a per-queue counter is sufficient. Increment chap->transaction instead, drop the reset in nvme_auth_reset_dhchap() so the id stays monotonic across re-authentication, and remove the now unused ctrl->transaction field. Fixes: f50fff73d620 ("nvme: implement In-Band authentication") Cc: stable@vger.kernel.org Signed-off-by: Guixin Liu <kanie@linux.alibaba.com> --- Changes since v2: reworked per review to use a per-queue counter and drop the shared field entirely (Keith Busch), and the Fixes tag was added (sashiko). The Reviewed-by on v2 covered the atomic_t version of this change and is dropped. --- drivers/nvme/host/auth.c | 3 +-- drivers/nvme/host/nvme.h | 1 - 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/nvme/host/auth.c b/drivers/nvme/host/auth.c index 9a3da74dec5a..54f3354ecae9 100644 --- a/drivers/nvme/host/auth.c +++ b/drivers/nvme/host/auth.c @@ -644,7 +644,6 @@ static void nvme_auth_reset_dhchap(struct nvme_dhchap_queue_context *chap) chap->s1 = 0; chap->s2 = 0; chap->bi_directional = false; - chap->transaction = 0; memset(chap->c1, 0, sizeof(chap->c1)); memset(chap->c2, 0, sizeof(chap->c2)); mempool_free(chap->buf, nvme_chap_buf_pool); @@ -764,7 +763,7 @@ static void nvme_queue_auth_work(struct work_struct *work) return; } - chap->transaction = ctrl->transaction++; + chap->transaction++; /* DH-HMAC-CHAP Step 1: send negotiate */ dev_dbg(ctrl->device, "%s: qid %d send negotiate\n", diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 2543e52d6358..e015d3a3e29c 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -463,7 +463,6 @@ struct nvme_ctrl { struct nvme_dhchap_queue_context *dhchap_ctxs; struct nvme_dhchap_key *host_key; struct nvme_dhchap_key *ctrl_key; - u16 transaction; #endif key_serial_t tls_pskid; -- 2.43.7 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction 2026-11-16 8:12 ` [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction Guixin Liu @ 2026-11-16 13:22 ` Hannes Reinecke 2026-11-16 15:37 ` Keith Busch 0 siblings, 1 reply; 5+ messages in thread From: Hannes Reinecke @ 2026-11-16 13:22 UTC (permalink / raw) To: Guixin Liu, Keith Busch, Jens Axboe, Christoph Hellwig, Sagi Grimberg, Nilay Shroff, Daniel Wagner Cc: linux-nvme On 11/16/26 9:12 AM, Guixin Liu wrote: > nvme_ctrl_auth_work() queues the DH-HMAC-CHAP exchange of every queue > on nvme-auth-wq without waiting for them, so multiple > nvme_queue_auth_work() instances run concurrently and all increment > ctrl->transaction in nvme_queue_auth_work(). The plain increment is a > data race, and the lost updates hand the same transaction id to > concurrent queues. > > The counter does not need to be shared at all: the controller only > echoes the transaction id within a queue, so a per-queue counter is > sufficient. Increment chap->transaction instead, drop the reset in > nvme_auth_reset_dhchap() so the id stays monotonic across > re-authentication, and remove the now unused ctrl->transaction field. > I disagree. The authentication transaction ID identifies the transaction itself, and is used to differentiate between different transactions. As such it needs to be unique within each queue. But as it carries controller-specific data we should also ensure the ID is unique per controller, ensuring we catch any possible misdirected authentication packet. It should be possible to move the controller value to an atomic value, and then we could do chap->transaction = atomic_inc_return(&ctrl->transaction); Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction 2026-11-16 13:22 ` Hannes Reinecke @ 2026-11-16 15:37 ` Keith Busch 0 siblings, 0 replies; 5+ messages in thread From: Keith Busch @ 2026-11-16 15:37 UTC (permalink / raw) To: Hannes Reinecke Cc: Guixin Liu, Jens Axboe, Christoph Hellwig, Sagi Grimberg, Nilay Shroff, Daniel Wagner, linux-nvme On Mon, Nov 16, 2026 at 02:22:51PM +0100, Hannes Reinecke wrote: > On 11/16/26 9:12 AM, Guixin Liu wrote: > > nvme_ctrl_auth_work() queues the DH-HMAC-CHAP exchange of every queue > > on nvme-auth-wq without waiting for them, so multiple > > nvme_queue_auth_work() instances run concurrently and all increment > > ctrl->transaction in nvme_queue_auth_work(). The plain increment is a > > data race, and the lost updates hand the same transaction id to > > concurrent queues. > > > > The counter does not need to be shared at all: the controller only > > echoes the transaction id within a queue, so a per-queue counter is > > sufficient. Increment chap->transaction instead, drop the reset in > > nvme_auth_reset_dhchap() so the id stays monotonic across > > re-authentication, and remove the now unused ctrl->transaction field. > > > I disagree. The authentication transaction ID identifies the > transaction itself, and is used to differentiate between different > transactions. As such it needs to be unique within each queue. But the proposal *is* unique within the queue. It's just not unique at the controller level, and it sounds like the protocol doesn't need to be. > But as it carries controller-specific data we should also ensure > the ID is unique per controller, ensuring we catch any possible > misdirected authentication packet. I don't understand. The qid:T_ID tuple already identifies the transaction. If a controller mixes up transactions across queues, it's already broken in ways we can't fix, so we shouldn't design around it. > It should be possible to move the controller value to an atomic > value, and then we could do If there's no protocol reason to require this type of uniqueness, then I'm not seeing why the driver is putting this constraint on itself. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-16 15:37 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-16 8:12 [PATCH v3 0/2] nvme: fabrics and target fixes Guixin Liu 2026-11-16 8:12 ` [PATCH v3 1/2] nvme-fc: fix use-after-free in nvme_fc_match_disconn_ls() Guixin Liu 2026-11-16 8:12 ` [PATCH v3 2/2] nvme-auth: fix data race on ctrl->transaction Guixin Liu 2026-11-16 13:22 ` Hannes Reinecke 2026-11-16 15:37 ` Keith Busch
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox