From: Sasha Levin <sashal@kernel.org>
To: patches@lists.linux.dev, stable@vger.kernel.org
Cc: Maurizio Lombardi <mlombard@redhat.com>,
Daniel Wagner <dwagner@suse.de>, Keith Busch <kbusch@kernel.org>,
Sasha Levin <sashal@kernel.org>,
justin.tee@broadcom.com, nareshgottumukkala83@gmail.com,
paul.ely@broadcom.com, axboe@kernel.dk, sagi@grimberg.me,
linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: [PATCH AUTOSEL 6.18-5.10] nvme-fc: do not warn on controller removal race
Date: Mon, 19 Oct 2026 09:52:25 -0400 [thread overview]
Message-ID: <20261019135441.1420968-454-sashal@kernel.org> (raw)
In-Reply-To: <20261019135441.1420968-1-sashal@kernel.org>
From: Maurizio Lombardi <mlombard@redhat.com>
[ Upstream commit 62b70e6c87121ae5b79ca20c3a5b005590336ec5 ]
In nvme_fc_reconnect_or_delete(), the system attempts to remove
the controller when reconnect limits are reached by calling
nvme_delete_ctrl(). This call is currently wrapped in a WARN_ON().
If a user concurrently deletes the controller via sysfs, the
controller's state may have already transitioned out of the
CONNECTING state. When this happens, nvme_delete_ctrl() correctly
rejects the operation and returns -EBUSY. Because of the WARN_ON()
wrapper, this race condition triggers an unnecessary kernel warning
stack trace.
Remove the WARN_ON() macro around nvme_delete_ctrl();
this allows the system to safely ignore the -EBUSY return value
if the controller is already being deleted.
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
Reviewed-by: Daniel Wagner <dwagner@suse.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
I have the local-tree, diff, history, and review evidence. Below is the
full nine-phase analysis for this 6.18.y checkout.
---
## PHASE 1: COMMIT MESSAGE FORENSICS
**Step 1.1: PARSE THE SUBJECT LINE**
Record: `[nvme-fc] [do not warn] suppress a false-positive WARN_ON when
controller deletion races with reconnect exhaustion`
**Step 1.2: PARSE ALL COMMIT MESSAGE TAGS**
Record:
- `Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>` (author)
- `Reviewed-by: Daniel Wagner <dwagner@suse.de>`
- `Signed-off-by: Keith Busch <kbusch@kernel.org>` (NVMe maintainer)
- No `Fixes:`, `Reported-by:`, `Tested-by:`, `Link:`, or `Cc: stable`
(absence is expected)
- Pipeline-only tags ignored: `[Upstream commit 62b70e6c8712...]`,
`Signed-off-by: Sasha Levin`
**Step 1.3: ANALYZE THE COMMIT BODY TEXT**
Record:
- **Bug:** `nvme_fc_reconnect_or_delete()` wraps `nvme_delete_ctrl()` in
`WARN_ON()`.
- **Symptom:** concurrent sysfs delete can move the controller out of
`CONNECTING`; `nvme_delete_ctrl()` then correctly returns `-EBUSY`,
and `WARN_ON()` dumps a kernel warning stack.
- **Root cause:** TOCTOU between the `NVME_CTRL_CONNECTING` check and
the delete call. `-EBUSY` is a valid result, not a programming error.
- **Version info:** none stated. The `WARN_ON` has been on this call
since 2017.
**Step 1.4: DETECT HIDDEN BUG FIXES**
Record: Not disguised. It is an explicit false-positive `WARN_ON` fix
for a real race. Functional delete behavior is unchanged.
---
## PHASE 2: DIFF ANALYSIS
**Step 2.1: INVENTORY THE CHANGES**
Record:
- File: `drivers/nvme/host/fc.c` (`1 insertion, 1 deletion`)
- Function: `nvme_fc_reconnect_or_delete()`
- Scope: single-file, one-line surgical change
**Step 2.2: UNDERSTAND THE CODE FLOW CHANGE**
Record:
- **Before:** after reconnect limits / `dev_loss_tmo` expiry,
`WARN_ON(nvme_delete_ctrl(&ctrl->ctrl))` fires on any non-zero return.
- **After:** `nvme_delete_ctrl()` is called and `-EBUSY` is ignored.
- Path: reconnect-failure / teardown, not the successful connect path.
**Step 2.3: IDENTIFY THE BUG MECHANISM**
Record:
- **Category:** race / incorrect assertion (`WARN_ON` on a legitimate
`-EBUSY`)
- **Mechanism:** `nvme_delete_ctrl()` fails `nvme_change_ctrl_state(...,
NVME_CTRL_DELETING)` or fails to queue `delete_work` if delete is
already in progress. The reconnect path already checked
`nvme_ctrl_state() == NVME_CTRL_CONNECTING` without a lock, so a
concurrent sysfs/`nvme_delete_ctrl()` can win. The warning is the bug;
deletion is already handled.
**Step 2.4: ASSESS THE FIX QUALITY**
Record:
- Obviously correct: other `nvme_delete_ctrl()` sites in the same file
(remoteport unregister, module unload) and RDMA/TCP already omit
`WARN_ON`.
- Minimal, no API/data-structure change.
- Regression risk is very low: the only behavior change is not warning
on an expected `-EBUSY`. A still-`CONNECTING` controller is still
deleted.
---
## PHASE 3: GIT HISTORY INVESTIGATION
**Step 3.1: BLAME THE CHANGED LINES**
Record:
- `WARN_ON(nvme_delete_ctrl(...))` blamed to `c5017e85705bfe` (“nvme:
move controller deletion to common code”, Christoph Hellwig,
2017-10-29).
- That commit replaced `WARN_ON(__nvme_fc_del_ctrl(ctrl))` with
`WARN_ON(nvme_delete_ctrl(...))`. The common helper can return
`-EBUSY` on a concurrent delete; the old wrapper’s assumption no
longer holds.
- Present since ~v4.15-era NVMe-FC code; long-lived in this 6.18 series.
**Step 3.2: FOLLOW THE FIXES: TAG**
Record: No `Fixes:` tag. Not applicable. Introducer is `c5017e85705bfe`,
which is in this tree.
**Step 3.3: CHECK FILE HISTORY FOR RELATED CHANGES**
Record:
- Recent `fc.c` work is unrelated (DMA unmap, double-free, tagset, rport
locking).
- Precedent in-tree: `6622f9acd29cd` “nvme-loop: do not warn for deleted
controllers during reset” (2021) — same class of false `WARN_ON`
during concurrent delete.
- Standalone; not `patch X/Y`.
**Step 3.4: CHECK THE AUTHOR'S OTHER COMMITS**
Record: Maurizio Lombardi (Red Hat) is a regular NVMe contributor.
Applied by Keith Busch (NVMe maintainer). Reviewed by Daniel Wagner
(SUSE, active NVMe-FC). Thread also has `Reviewed-by: Hannes Reinecke`.
**Step 3.5: CHECK FOR DEPENDENT/PREREQUISITE COMMITS**
Record: None. Uses existing `nvme_delete_ctrl()`. Applies standalone.
---
## PHASE 4: MAILING LIST AND EXTERNAL RESEARCH
**Step 4.1: FIND THE ORIGINAL PATCH DISCUSSION**
Record:
- `b4 dig -c 62b70e6c87121ae5b79ca20c3a5b005590336ec5` →
https://patch.msgid.link/20260908143434.105513-1-mlombard@redhat.com
- `b4 dig -a`: single v1, no later revisions
- Full thread saved via `b4 dig -m`
- Daniel Wagner: “Yes, this makes sense.” + Reviewed-by
- Hannes Reinecke: Reviewed-by
- No NAKs, no risk flags, no `Cc: stable` in review (not a negative)
**Step 4.2: CHECK WHO REVIEWED THE PATCH**
Record: Recipients included `kbusch@kernel.org`, `dwagner@suse.de`,
`linux-nvme@lists.infradead.org`. Right maintainers and list. Keith
Busch signed off.
**Step 4.3: SEARCH FOR THE BUG REPORT**
Record: No `Reported-by` / bugzilla / syzbot. Description is a concrete
sysfs-vs-reconnect race, not theoretical cleanup.
**Step 4.4: CHECK FOR RELATED PATCHES AND SERIES**
Record: Standalone one-patch series. Sibling transports already call
`nvme_delete_ctrl()` without `WARN_ON`.
**Step 4.5: CHECK STABLE MAILING LIST HISTORY**
Record: lore.kernel.org WebFetch returned 403. The captured NVMe thread
has no stable discussion. **UNVERIFIED:** whether `stable@` discussed
this commit.
---
## PHASE 5: CODE SEMANTIC ANALYSIS
**Step 5.1: IDENTIFY KEY FUNCTIONS**
Record: `nvme_fc_reconnect_or_delete()`; callee `nvme_delete_ctrl()`.
**Step 5.2: TRACE CALLERS**
Record:
- `nvme_fc_reset_ctrl_work()` → `nvme_fc_reconnect_or_delete(ctrl,
-ENOTCONN)` when the remoteport is not `ONLINE`
- `nvme_fc_connect_ctrl_work()` → `nvme_fc_reconnect_or_delete(ctrl,
ret)` when `nvme_fc_create_association()` fails
Both run on `nvme_wq` after link loss / failed reconnect — a normal SAN
path.
**Step 5.3: TRACE CALLEES**
Record: `nvme_delete_ctrl()` does `nvme_change_ctrl_state(ctrl,
NVME_CTRL_DELETING)` then `queue_work(nvme_delete_wq,
&ctrl->delete_work)`, returning `-EBUSY` if either fails. Sysfs
`delete_controller` uses `nvme_delete_ctrl_sync()` (same `DELETING`
transition).
**Step 5.4: FOLLOW THE CALL CHAIN**
Record:
```
fabric reconnect work / reset work
→ nvme_fc_reconnect_or_delete()
[state == CONNECTING?]
→ nvme_delete_ctrl() // races with:
sysfs delete_controller (CAP_SYS_ADMIN, S_IWUSR)
nvme_fc_unregister_remoteport()
nvme_fc_delete_controllers() (module unload)
```
Reachable in production NVMe-FC setups. Sysfs delete needs privilege;
reconnect exhaustion does not.
**Step 5.5: SEARCH FOR SIMILAR PATTERNS**
Record: At `v6.18.57`, `WARN_ON(nvme_delete_ctrl` exists only in
`drivers/nvme/host/fc.c`. RDMA (`nvme_rdma_reconnect_or_remove`), TCP,
and the other two FC call sites already ignore the return. This site is
the leftover.
semcode MCP: not available; used `git grep` / `rg`.
---
## PHASE 6: CROSS-REFERENCING AGAINST THE LOCAL TREE
**Step 6.1: DOES THE BUGGY CODE EXIST IN THIS TREE?**
Record:
- Local tree: `v6.18.57` (`VERSION=6`, `PATCHLEVEL=18`, `SUBLEVEL=57`;
`git describe HEAD` = `v6.18.57-155-gcdf239ca885b2`)
- At `v6.18.57`, `drivers/nvme/host/fc.c:3337` is
`WARN_ON(nvme_delete_ctrl(&ctrl->ctrl));`
- Upstream fix `62b70e6c8712` is **not** an ancestor of `v6.18.57`
- Bug introduced 2017; present in this series
**Step 6.2: CHECK FOR BACKPORT COMPLICATIONS**
Record: Clean apply. Context around the line at `v6.18.57` matches the
patch. `git apply --check` against the `v6.18.57` file succeeded.
Difficulty: clean apply.
**Step 6.3: CHECK IF RELATED FIXES ARE ALREADY HERE**
Record: No other in-tree commit removes this FC `WARN_ON`. The loop
analog (`6622f9acd29cd`) is already present and does not replace this
fix.
---
## PHASE 7: SUBSYSTEM AND MAINTAINER CONTEXT
**Step 7.1: IDENTIFY THE SUBSYSTEM AND ITS CRITICALITY**
Record: NVMe over Fabrics FC host (`drivers/nvme/host/fc.c`).
**IMPORTANT** (enterprise SAN storage), not core mm/VFS, not staging.
**Step 7.2: ASSESS SUBSYSTEM ACTIVITY**
Record: Actively maintained (recent FC locking, tagset, DMA fixes). This
line itself has been stable since 2017.
---
## PHASE 8: IMPACT AND RISK ASSESSMENT
**Step 8.1: DETERMINE WHO IS AFFECTED**
Record: Config-specific / driver-specific: `CONFIG_NVME_FC` users.
Typical audience is enterprise FC-SAN, i.e. LTS/stable deployments.
**Step 8.2: DETERMINE THE TRIGGER CONDITIONS**
Record:
- Trigger: reconnect attempts exhausted **and** a concurrent delete
(sysfs, remoteport teardown, or module unload)
- Not every I/O; realistic during link flaps / admin teardown
- Unprivileged users cannot write `delete_controller`; they can
contribute to reconnect failure. Privilege needed for the sysfs side
of the described race.
**Step 8.3: DETERMINE THE FAILURE MODE SEVERITY**
Record:
- Default: `WARN_ON` stack dump (no data loss; delete still happens on
the other path)
- With `panic_on_warn`: kernel panic during controller teardown
- Severity: **MEDIUM** normally, **HIGH/CRITICAL** if `panic_on_warn` is
enabled (CI, hardened, some Android/enterprise)
**Step 8.4: CALCULATE RISK-BENEFIT RATIO**
Record:
- **Benefit:** medium — stops a user-visible warning and a possible
`panic_on_warn` crash on a path enterprise NVMe-FC users actually hit
- **Risk:** very low — one line, no locking/API change, matches other
transports
- **Ratio:** benefit exceeds risk
---
## PHASE 9: FINAL SYNTHESIS
**Step 9.1: COMPILE THE EVIDENCE**
**FOR:**
- Real race; `-EBUSY` is correct, `WARN_ON` is not
- `WARN_ON` is a listed strong-yes class (and can panic)
- One-line, obviously correct, reviewed by Wagner/Reinecke, applied by
Busch
- Buggy line exists in `v6.18.57`; applies cleanly; no prerequisites
- Same file’s other delete sites and RDMA/TCP already omit the warning
- Precedent: nvme-loop “do not warn for deleted controllers”
**AGAINST:**
- Not data corruption / UAF / deadlock
- No public user report or syzbot
- Default impact is a warning, not a functional hang
- NVMe-FC is not universal
**UNRESOLVED:**
- Whether `stable@` discussed it (lore search 403)
- How often the race is hit in the field (no `Reported-by`)
**Step 9.2: APPLY THE STABLE RULES CHECKLIST**
1. Obviously correct and tested? **PASS** — one-line, two NVMe
reviewers, maintainer SOB
2. Fixes a real bug that affects users? **PASS** — false `WARN_ON` on a
reachable race
3. Important issue? **PASS** — `WARN_ON` / possible `panic_on_warn`
panic; not silent corruption, but in the documented yes class
4. Small and contained? **PASS** — 1 line, 1 file
5. No new features or APIs? **PASS**
6. Can apply to the local tree? **PASS** — clean on `v6.18.57`
**Step 9.3: CHECK FOR EXCEPTION CATEGORIES**
Record: None (not a device ID / quirk / DT / build / docs exception).
Qualifies as a normal small bugfix.
**Step 9.4: DECISION**
This tree is 6.18.y. The `WARN_ON` is still in `v6.18.57`. The patch is
a one-line removal of a false assertion on a real reconnect-vs-delete
race. It does not change successful-delete behavior, matches every other
NVMe transport, and is the kind of warning-to-panic issue stable should
take.
---
## What the commit solves, and why it matters here
On 6.18.y NVMe-FC, when reconnects are exhausted,
`nvme_fc_reconnect_or_delete()` tries to delete the controller. If
userspace (`delete_controller`) or another teardown path already moved
the controller to `DELETING`, `nvme_delete_ctrl()` returns `-EBUSY` by
design. Wrapping that in `WARN_ON()` turns a handled race into a kernel
warning (or a panic if `panic_on_warn` is set). Removing the wrapper is
the same policy RDMA, TCP, and the other FC delete sites already use.
Risk is negligible; benefit is avoiding a production warning/panic on
enterprise FC fabrics that 6.18.y actually ships.
---
## Verification
- [Phase 1] Parsed tags: author SOB Maurizio Lombardi; Reviewed-by
Daniel Wagner; SOB Keith Busch; no Fixes/Reported-by/Cc:stable
- [Phase 2] Diff: 1 line in `nvme_fc_reconnect_or_delete()`;
`WARN_ON(nvme_delete_ctrl())` → `nvme_delete_ctrl()`
- [Phase 2] `nvme_delete_ctrl()` at `drivers/nvme/host/core.c:257`
returns `-EBUSY` if state change or `queue_work` fails
- [Phase 2] State machine: `CONNECTING` → `DELETING` or `LIVE` only;
concurrent delete explains `-EBUSY`
- [Phase 3] `git blame` at parent: line from `c5017e85705bfe` (2017);
originally `WARN_ON(__nvme_fc_del_ctrl)`
- [Phase 3] `git log -S 'WARN_ON(nvme_delete_ctrl'` on `fc.c`:
introducer + this removal
- [Phase 3] Related: `6622f9acd29cd` nvme-loop “do not warn…” is already
an ancestor
- [Phase 3] No later `fc.c` commits after the candidate rewrite this
line
- [Phase 4] `b4 dig -c 62b70e6c8712`:
https://patch.msgid.link/20260908143434.105513-1-mlombard@redhat.com
- [Phase 4] `b4 dig -a`: v1 only
- [Phase 4] `b4 dig -w`: kbusch, dwagner, linux-nvme CC’d
- [Phase 4] `b4 dig -m`: Wagner + Reinecke Reviewed-by; no NAK
- [Phase 4] lore WebFetch 403 — used b4 mbox instead
- [Phase 4] UNVERIFIED: stable@ mailing list history (403)
- [Phase 5] Callers verified: `nvme_fc_reset_ctrl_work`,
`nvme_fc_connect_ctrl_work`
- [Phase 5] Concurrent deleters: `nvme_sysfs_delete` →
`nvme_delete_ctrl_sync`; `nvme_fc_unregister_remoteport`;
`nvme_fc_delete_controllers`
- [Phase 5] `git grep` at v6.18.57: only FC reconnect path uses
`WARN_ON(nvme_delete_ctrl`
- [Phase 6] `git describe HEAD` / `make kernelversion`: 6.18.57
- [Phase 6] `git show v6.18.57:drivers/nvme/host/fc.c` line 3337:
`WARN_ON` present
- [Phase 6] `git merge-base --is-ancestor 62b70e6c8712 v6.18.57` / HEAD:
not in the 6.18.57 base
- [Phase 6] Apply check of the patch onto the v6.18.57 file: success
- [Phase 7] `CONFIG_NVME_FC`; MAINTAINERS: Keith Busch + NVMe FC
transport
- [Phase 8] Failure mode: WARN_ON stack / panic_on_warn; severity MEDIUM
(HIGH if panic_on_warn)
- UNVERIFIED: field frequency (no Reported-by)
The fix is small, surgical, present as a real bug in this 6.18.y tree,
and meets the stable criteria.
**YES**
drivers/nvme/host/fc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index fc7aa3fc6c2b5..186d1beecf32b 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3334,7 +3334,7 @@ nvme_fc_reconnect_or_delete(struct nvme_fc_ctrl *ctrl, int status)
ctrl->cnum, min_t(int, portptr->dev_loss_tmo,
(ctrl->ctrl.opts->max_reconnects *
ctrl->ctrl.opts->reconnect_delay)));
- WARN_ON(nvme_delete_ctrl(&ctrl->ctrl));
+ nvme_delete_ctrl(&ctrl->ctrl);
}
}
--
2.53.0
prev parent reply other threads:[~2026-10-19 14:05 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20261019135441.1420968-1-sashal@kernel.org>
2026-10-19 13:45 ` [PATCH AUTOSEL 6.18-5.10] nvme: zns: cap zone report nr_zones by DMA buffer size Sasha Levin
2026-10-19 13:45 ` [PATCH AUTOSEL 6.18] nvme: clamp FDP nruhsd to allocated RUH status descriptor count Sasha Levin
2026-10-19 13:45 ` [PATCH AUTOSEL 6.18-5.10] nvme: ratelimit the completion-path messages driven by device data Sasha Levin
2026-10-19 13:46 ` [PATCH AUTOSEL 6.18-5.15] nvmet: zns: reject full zone report when buffer is too small Sasha Levin
2026-10-19 14:15 ` sashiko-bot
2026-10-19 13:46 ` [PATCH AUTOSEL 6.18-5.10] nvme: bound ns descriptor header and body to identify buffer Sasha Levin
2026-10-19 13:48 ` [PATCH AUTOSEL 6.18-6.6] nvme-auth: use crypto_memneq for DH-HMAC-CHAP response comparison Sasha Levin
2026-10-19 13:48 ` [PATCH AUTOSEL 6.18-5.15] nvme-tcp: look up host_iface in the current netns Sasha Levin
2026-10-19 14:43 ` sashiko-bot
2026-10-19 13:49 ` [PATCH AUTOSEL 6.18] nvme-tcp: delay nvme_tcp_reclassify_socket() Sasha Levin
2026-10-19 14:50 ` sashiko-bot
2026-10-19 13:50 ` [PATCH AUTOSEL 6.18-6.12] nvme: handle positive error codes in nuse_show() Sasha Levin
2026-10-19 13:51 ` [PATCH AUTOSEL 6.18-6.6] nvme: don't reference NS after unlocking in nvme_ns_head_ctrl_ioctl() Sasha Levin
2026-10-19 13:51 ` [PATCH AUTOSEL 6.18-6.1] nvmet: passthru: fix OOB reads when parsing ns id descriptor list Sasha Levin
2026-10-19 13:52 ` Sasha Levin [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261019135441.1420968-454-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=axboe@kernel.dk \
--cc=dwagner@suse.de \
--cc=justin.tee@broadcom.com \
--cc=kbusch@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=mlombard@redhat.com \
--cc=nareshgottumukkala83@gmail.com \
--cc=patches@lists.linux.dev \
--cc=paul.ely@broadcom.com \
--cc=sagi@grimberg.me \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox