From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4DEDAC43458 for ; Thu, 2 Jul 2026 12:38:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=lKI7fmOaYipG31xvygjbfgJ816qdaKPkIEG+rDNU3b4=; b=jQbvowqi68IEEkEgIhEZjDQVR6 05Js0hC+QtnE2aIe8RyiriOG9l7dh84EOewBqqOTZrPDizBPWNP+OFOpu6PkusJR6tD9/YY5CTUU+ /GCo9w5l6WSCISDDj/Bm0MMRwfOgKhJASaIjrDwUFLi+5R7f6bAVNiIzldZxYgxrCxHwveKfhkQ6F Vm2BkMO0FTlskEbnd+3mdeZz7PUeLIXrmyTFb2LGNCQBVO7GCWYuDnAT9fREQdGJo+gy7g7S4CG0C PrQKux9woFvJaIzQUGPhOu5XmAJbTXpf2v2jF9n16UGNY++3R04eitjjBYe47AoX5ooCtUZMOe5+K jBX5bDvA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wfGgn-00000004S5F-3CZ7; Thu, 02 Jul 2026 12:38:45 +0000 Received: from out30-98.freemail.mail.aliyun.com ([115.124.30.98]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wfGgl-00000004S4k-2PQY for linux-nvme@lists.infradead.org; Thu, 02 Jul 2026 12:38:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1782995921; h=From:To:Subject:Date:Message-Id:MIME-Version; bh=lKI7fmOaYipG31xvygjbfgJ816qdaKPkIEG+rDNU3b4=; b=wI7uLnPt9rnpQArrmE/BXinoB+Rg6G2wj4gJB0oFi5m4YYWc26Gb+NcSiBd6Pb8GGZDnTOIItmNIyMcXDKyu05N9YMbBEKEhAkwyM0bydSs9AqYvtkCiMTFHtczDEF1v+GyFnb6wBBa3cOuG7tBd+I7YohI1xJWpb9CUthKOJKQ= X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037026112;MF=lulie@linux.alibaba.com;NM=1;PH=DS;RN=11;SR=0;TI=SMTPD_---0X6En153_1782995919; Received: from localhost(mailfrom:lulie@linux.alibaba.com fp:SMTPD_---0X6En153_1782995919 cluster:ay36) by smtp.aliyun-inc.com; Thu, 02 Jul 2026 20:38:40 +0800 From: Philo Lu To: stable@vger.kernel.org Cc: hch@lst.de, sagi@grimberg.me, kch@nvidia.com, gregkh@linuxfoundation.org, skumar47@syr.edu, kumar.shivam43666@gmail.com, kbusch@kernel.org, dust.li@linux.alibaba.com, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 5.10.y] nvmet-tcp: fix race between ICReq handling and queue teardown Date: Thu, 2 Jul 2026 20:38:39 +0800 Message-Id: <20260702123839.100640-1-lulie@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260702_053843_788397_7D31019B X-CRM114-Status: GOOD ( 15.14 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org From: Chaitanya Kulkarni commit 5293a8882c549fab4a878bc76b0b6c951f980a61 upstream. nvmet_tcp_handle_icreq() updates queue->state after sending an Initialization Connection Response (ICResp), but it does so without serializing against target-side queue teardown. If an NVMe/TCP host sends an Initialization Connection Request (ICReq) and immediately closes the connection, target-side teardown may start in softirq context before io_work drains the already buffered ICReq. In that case, nvmet_tcp_schedule_release_queue() sets queue->state to NVMET_TCP_Q_DISCONNECTING and drops the queue reference under state_lock. If io_work later processes that ICReq, nvmet_tcp_handle_icreq() can still overwrite the state back to NVMET_TCP_Q_LIVE. That defeats the DISCONNECTING-state guard in nvmet_tcp_schedule_release_queue() and allows a later socket state change to re-enter teardown and issue a second kref_put() on an already released queue. The ICResp send failure path has the same problem. If teardown has already moved the queue to DISCONNECTING, a send error can still overwrite the state with NVMET_TCP_Q_FAILED, again reopening the window for a second teardown path to drop the queue reference. Fix this by serializing both post-send state transitions with state_lock and bailing out if teardown has already started. Use -ESHUTDOWN as an internal sentinel for that bail-out path rather than propagating it as a transport error like -ECONNRESET. Keep nvmet_tcp_socket_error() setting rcv_state to NVMET_TCP_RECV_ERR before honoring that sentinel so receive-side parsing stays quiesced until the existing release path completes. Fixes: c46a6465bac2 ("nvmet-tcp: add NVMe over TCP target driver") Cc: stable@vger.kernel.org Reported-by: Shivam Kumar Tested-by: Shivam Kumar Signed-off-by: Chaitanya Kulkarni Signed-off-by: Keith Busch [ context diff adaptation: drop `queue->state = NVMET_TCP_Q_FAILED` since the enum introduced in 6.7, 675b453e0241 ("nvmet-tcp: enable TLS handshake upcall" ] Signed-off-by: Philo Lu --- drivers/nvme/target/tcp.c | 29 ++++++++++++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-) diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c index edcc0662c3e5..4bd82e9b113f 100644 --- a/drivers/nvme/target/tcp.c +++ b/drivers/nvme/target/tcp.c @@ -345,6 +345,19 @@ static void nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd) static void nvmet_tcp_fatal_error(struct nvmet_tcp_queue *queue) { + /* + * Keep rcv_state at RECV_ERR even for the internal -ESHUTDOWN path. + * nvmet_tcp_handle_icreq() can return -ESHUTDOWN after the ICReq has + * already been consumed and queue teardown has started. + * + * If nvmet_tcp_data_ready() or nvmet_tcp_write_space() queues + * nvmet_tcp_io_work() again before nvmet_tcp_release_queue_work() + * cancels it, the queue must not keep that old receive state. + * Otherwise the next nvmet_tcp_io_work() run can reach + * nvmet_tcp_done_recv_pdu() and try to handle the same ICReq again. + * + * That is why queue->rcv_state needs to be updated before we return. + */ queue->rcv_state = NVMET_TCP_RECV_ERR; if (queue->nvme_sq.ctrl) nvmet_ctrl_fatal_error(queue->nvme_sq.ctrl); @@ -888,10 +901,24 @@ static int nvmet_tcp_handle_icreq(struct nvmet_tcp_queue *queue) iov.iov_base = icresp; iov.iov_len = sizeof(*icresp); ret = kernel_sendmsg(queue->sock, &msg, &iov, 1, iov.iov_len); - if (ret < 0) + if (ret < 0) { + spin_lock_bh(&queue->state_lock); + if (queue->state == NVMET_TCP_Q_DISCONNECTING) { + spin_unlock_bh(&queue->state_lock); + return -ESHUTDOWN; + } + spin_unlock_bh(&queue->state_lock); return ret; /* queue removal will cleanup */ + } + spin_lock_bh(&queue->state_lock); + if (queue->state == NVMET_TCP_Q_DISCONNECTING) { + spin_unlock_bh(&queue->state_lock); + /* Tell nvmet_tcp_socket_error() teardown is in progress. */ + return -ESHUTDOWN; + } queue->state = NVMET_TCP_Q_LIVE; + spin_unlock_bh(&queue->state_lock); nvmet_prepare_receive_pdu(queue); return 0; } -- 2.47.3