From: Sasha Levin <sashal@kernel.org>
To: patches@lists.linux.dev, stable@vger.kernel.org
Cc: Hannes Reinecke <hare@kernel.org>, Yi Zhang <yi.zhang@redhat.com>,
Sagi Grimberg <sagi@grimberg.me>, Keith Busch <kbusch@kernel.org>,
Sasha Levin <sashal@kernel.org>,
kch@nvidia.com, linux-nvme@lists.infradead.org
Subject: [PATCH AUTOSEL 6.18-6.6] nvmet-tcp: fixup hang in nvmet_tcp_listen_data_ready()
Date: Tue, 20 Jan 2026 14:34:50 -0500 [thread overview]
Message-ID: <20260120193456.865383-7-sashal@kernel.org> (raw)
In-Reply-To: <20260120193456.865383-1-sashal@kernel.org>
From: Hannes Reinecke <hare@kernel.org>
[ Upstream commit 2fa8961d3a6a1c2395d8d560ffed2c782681bade ]
When the socket is closed while in TCP_LISTEN a callback is run to
flush all outstanding packets, which in turns calls
nvmet_tcp_listen_data_ready() with the sk_callback_lock held.
So we need to check if we are in TCP_LISTEN before attempting
to get the sk_callback_lock() to avoid a deadlock.
Link: https://lore.kernel.org/linux-nvme/CAHj4cs-zu7eVB78yUpFjVe2UqMWFkLk8p+DaS3qj+uiGCXBAoA@mail.gmail.com/
Tested-by: Yi Zhang <yi.zhang@redhat.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
Let me understand the context of the commit better - it might be that
the commit requires a minor adjustment for backport since it uses
`nvmet_wq` but in the current tree (and stable trees), the workqueue is
called `nvmet_tcp_wq`. Looking at the diff more carefully:
The diff shows:
```c
+ if (port)
queue_work(nvmet_wq, &port->accept_work);
```
Wait, the space before `queue_work` in the diff context shows this is
the EXISTING code (no + sign). But wait, looking at the current code:
```c
schedule_work(&port->accept_work);
```
So there's a discrepancy. The commit I'm analyzing might be from a tree
where `schedule_work` was changed to `queue_work(nvmet_wq, ...)` in
another commit.
For the purpose of backporting analysis, the KEY FIX is:
1. **Check `sk->sk_state != TCP_LISTEN` BEFORE acquiring the lock** -
this is the essential fix for the deadlock
2. The change from `schedule_work` to `queue_work(nvmet_wq, ...)` is
secondary
For stable backports, minor adjustments may be needed (using
`schedule_work` instead of `queue_work(nvmet_wq, ...)`), but the core
fix (early state check before lock acquisition) is applicable.
## SUMMARY
**What the commit fixes**: A deadlock that occurs when
`nvmet_tcp_listen_data_ready()` is called during socket cleanup with
`sk_callback_lock` already held. The fix checks the socket state before
acquiring the lock.
**Why it matters for stable users**: This is a deadlock/hang that can
occur during NVMe-over-TCP disconnect scenarios. Systems using NVMe-
over-TCP will hang, requiring a hard reset.
**Meets stable kernel rules**:
- Obviously correct: Yes - simple early return check before lock
acquisition
- Fixes a real bug: Yes - deadlock reported and reproduced via blktests
- Small scope: Yes - 10 lines changed in one function
- No new features: Correct - pure bug fix
- Tested: Yes - "Tested-by" tag from Red Hat QE
**Risk assessment**: Very low. The fix simplifies the logic and cannot
cause regression.
**Dependencies/Backport notes**: The commit uses `queue_work(nvmet_wq,
...)` but older stable trees use `schedule_work(&port->accept_work)`. A
minor adjustment may be needed for clean backport, but the essential fix
(early state check) applies cleanly.
**YES** - This is a clear deadlock fix that should be backported to
stable kernels. The commit:
1. Fixes a real, reproducible deadlock
2. Is minimal and obviously correct
3. Has been tested and reviewed
4. Affects users of NVMe-over-TCP (a production storage protocol)
5. Has very low regression risk
**YES**
drivers/nvme/target/tcp.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/drivers/nvme/target/tcp.c b/drivers/nvme/target/tcp.c
index 470bf37e5a637..2e9a3e698b700 100644
--- a/drivers/nvme/target/tcp.c
+++ b/drivers/nvme/target/tcp.c
@@ -1992,14 +1992,13 @@ static void nvmet_tcp_listen_data_ready(struct sock *sk)
trace_sk_data_ready(sk);
+ if (sk->sk_state != TCP_LISTEN)
+ return;
+
read_lock_bh(&sk->sk_callback_lock);
port = sk->sk_user_data;
- if (!port)
- goto out;
-
- if (sk->sk_state == TCP_LISTEN)
+ if (port)
queue_work(nvmet_wq, &port->accept_work);
-out:
read_unlock_bh(&sk->sk_callback_lock);
}
--
2.51.0
next prev parent reply other threads:[~2026-01-20 19:35 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-01-20 19:34 [PATCH AUTOSEL 6.18] ALSA: usb-audio: Prevent excessive number of frames Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18-6.6] ASoC: amd: yc: Fix microphone on ASUS M6500RE Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18-5.10] ASoC: tlv320adcx140: Propagate error codes during probe Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18-6.1] nvme-fc: release admin tagset if init fails Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18] dmaengine: mmp_pdma: Fix race condition in mmp_pdma_residue() Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18-5.10] ASoC: davinci-evm: Fix reference leak in davinci_evm_probe Sasha Levin
2026-01-20 19:34 ` Sasha Levin [this message]
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18-6.12] ASoC: simple-card-utils: Check device node before overwrite direction Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18] ASoC: Intel: sof_sdw: Add new quirks for PTL on Dell with CS42L43 Sasha Levin
2026-01-20 19:34 ` [PATCH AUTOSEL 6.18] ALSA: hda/tas2781: Add newly-released HP laptop Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260120193456.865383-7-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=hare@kernel.org \
--cc=kbusch@kernel.org \
--cc=kch@nvidia.com \
--cc=linux-nvme@lists.infradead.org \
--cc=patches@lists.linux.dev \
--cc=sagi@grimberg.me \
--cc=stable@vger.kernel.org \
--cc=yi.zhang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox