From: sashiko-bot@kernel.org
To: "Srivatsa S. Bhat (Microsoft)" <srivatsa@csail.mit.edu>
Cc: linux-hyperv@vger.kernel.org
Subject: Re: [PATCH] block: Fix NULL pointer dereference in blk_mq_complete_request_remote
Date: Wed, 18 Nov 2026 19:57:27 +0000 [thread overview]
Message-ID: <sashiko-outbox-221909@kernel.org> (raw)
In-Reply-To: <20261119-fix-blk-mq-null-ptr-deref-v1-1-9bcbcfcfaff8@csail.mit.edu>
Thank you for your contribution! Sashiko AI review found 3 potential
issues to consider:
- [High] block: incomplete and racy freed-request check in
blk_mq_complete_request_remote()
- [Medium] nvme: missing in-flight state check in nvme_find_rq() allows
prior crash and corruption
- [Low] block: inaccurate Fixes: trailer in commit message
--
commit 65327008502cbbac3ada696570a4b5ff52feb917
Author: Srivatsa S. Bhat (Microsoft) <srivatsa@csail.mit.edu>
> block: Fix NULL pointer dereference in blk_mq_complete_request_remote
>
> Add NULL pointer validation in blk_mq_complete_request_remote() to
> prevent kernel crashes when block drivers pass freed request structures.
>
> When NVMe devices experience hardware failures or firmware bugs,
> completion queue corruption can occur, leading to scenarios where
> drivers attempt to complete already-freed requests. The NVMe genctr
> validation in nvme_find_rq() catches most stale completions, but when
> corruption produces a command_id whose genctr bits happen to match,
> the freed request passes validation and reaches the block layer. The
> block layer's __blk_mq_free_request() sets rq->mq_hctx to NULL,
> but blk_mq_complete_request_remote() lacks validation before
> dereferencing this pointer.
>
> This manifests as a kernel crash like shown below (the stack trace
> was obtained from a crash observed in a Linux VM on Hyper-V):
>
> nvme nvme0: resetting controller due to AER
> nvme nvme0: request 0xe3 genctr mismatch (got 0x0 expected 0x5)
> nvme nvme0: invalid id 227 completed on queue 6
> nvme nvme0: request 0x26 genctr mismatch (got 0xb expected 0x9)
> nvme nvme0: invalid id 45094 completed on queue 94
>
> BUG: kernel NULL pointer dereference, address: 00000000000000fe
> RIP: 0010:blk_mq_complete_request_remote+0x1b/0x150
> Call Trace:
> <IRQ>
> nvme_irq+0x182/0x320 [nvme]
> __handle_irq_event_percpu+0x47/0x180
> handle_irq_event+0x38/0x60
> handle_edge_irq+0x87/0x1a0
> __common_interrupt+0x3f/0xa0
> common_interrupt+0x42/0xa0
>
> The crash occurs at offset 0xfe when accessing rq->mq_hctx->nr_ctx,
> where mq_hctx is NULL.
>
> Add a NULL check that returns true (signaling to callers that the
> request has been handled) to gracefully drop freed requests, with a
> WARN_ONCE to flag the condition for diagnosis.
>
> Fixes: 40d09b53bfc5 ("blk-mq: add a new blk_mq_complete_request_remote API")
> Signed-off-by: Srivatsa S. Bhat (Microsoft) <srivatsa@csail.mit.edu>
[Severity: Low]
In commit 40d09b53bfc5 ("blk-mq: add a new blk_mq_complete_request_remote API"),
blk_mq_complete_request_remote() did not dereference rq->mq_hctx. The
rq->mq_hctx->nr_ctx dereference was added later, so should the Fixes tag
instead be:
Fixes: f168420c62e7 ("blk-mq: don't redirect completion for hctx withs only one ctx mapping")
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 5ae22d5565404..5dc9d7d806b59 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -1307,6 +1307,15 @@ static void blk_mq_raise_softirq(struct request *rq)
>
> bool blk_mq_complete_request_remote(struct request *rq)
> {
> + /*
> + * Validate request is not freed (__blk_mq_free_request() sets
> + * mq_hctx = NULL).
> + */
> + if (unlikely(!rq->mq_hctx)) {
> + WARN_ONCE(1, "%s: attempt to complete freed request", __func__);
> + return true;
> + }
> +
> WRITE_ONCE(rq->state, MQ_RQ_COMPLETE);
>
> /*
[Severity: High]
Is checking !rq->mq_hctx sufficient and race-free for detecting requests
that are not in flight?
First, the batched completion fast path used by nvme-pci frees requests
without ever clearing rq->mq_hctx:
block/blk-mq.c:blk_mq_end_request_batch() {
...
WRITE_ONCE(rq->state, MQ_RQ_IDLE);
if (!req_ref_put_and_test(rq))
continue;
blk_crypto_free_request(rq);
blk_pm_mark_last_busy(rq);
if (nr_tags == TAG_COMP_BATCH || cur_hctx != rq->mq_hctx) {
if (cur_hctx)
blk_mq_flush_tag_batch(cur_hctx, tags, nr_tags);
nr_tags = 0;
cur_hctx = rq->mq_hctx;
}
tags[nr_tags++] = rq->tag;
...
}
Requests freed through blk_mq_end_request_batch(), requests reallocated
or cached in a plug via blk_mq_rq_ctx_init() (still in MQ_RQ_IDLE before
blk_mq_start_request()), and requests already in MQ_RQ_COMPLETE awaiting
remote IPI or softirq completion all have a non-NULL rq->mq_hctx. A
spurious or duplicate CQE matching one of those requests bypasses the
!rq->mq_hctx check and completes the request a second time (causing
double DMA unmapping in nvme_pci_unmap_rq(), refcount and sbitmap tag
underflow, or cyclic corruption of the per-CPU blk_cpu_done llist if
rq->ipi_list is queued twice).
Second, because __blk_mq_free_request() clears rq->mq_hctx without
synchronization against a spurious completion on another CPU, the unlocked
check still races:
CPU0 CPU1
blk_mq_free_request(rq)
WRITE_ONCE(rq->state, MQ_RQ_IDLE)
blk_mq_complete_request_remote(rq)
if (unlikely(!rq->mq_hctx))
/* observes non-NULL */
WRITE_ONCE(rq->state, MQ_RQ_COMPLETE)
__blk_mq_free_request(rq)
rq->mq_hctx = NULL
if (rq->mq_hctx->nr_ctx == 1 ...)
<-- NULL pointer dereference
In addition to crashing at rq->mq_hctx->nr_ctx, CPU1 overwrites the freed
request's rq->state back to MQ_RQ_COMPLETE after CPU0 set it to
MQ_RQ_IDLE.
[Severity: Medium]
Should stale completions be rejected earlier in nvme_find_rq() by checking
that blk_mq_rq_state(rq) == MQ_RQ_IN_FLIGHT (similar to
scsi_host_find_tag() and nbd_read_reply())?
When nvme_find_rq() returns a request whose 4-bit genctr collided with a
corrupted CQE, callers act on that request before
blk_mq_complete_request_remote() is reached:
drivers/nvme/host/pci.c:nvme_handle_cqe() {
...
trace_nvme_sq(req, cqe->sq_head, data_race(nvmeq->sq_tail));
if (!nvme_try_complete_req(req, cqe->status, cqe->result) &&
...
}
If the nvme_sq tracepoint is enabled, trace_nvme_sq() calls
nvme_req_qid(req), which dereferences req->mq_hctx->queue_num and hits the
same NULL pointer dereference when req->mq_hctx is NULL. Next,
nvme_try_complete_req() increments nvme_req(req)->genctr and overwrites
status and result before calling blk_mq_complete_request_remote(), which
corrupts the request if it has already been reallocated or is awaiting
remote completion and causes its real completion to fail genctr
validation.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261119-fix-blk-mq-null-ptr-deref-v1-1-9bcbcfcfaff8@csail.mit.edu?part=1
next prev parent reply other threads:[~2026-11-18 19:57 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 19:28 [PATCH] block: Fix NULL pointer dereference in blk_mq_complete_request_remote Srivatsa S. Bhat (Microsoft)
2026-11-18 19:57 ` sashiko-bot [this message]
2026-11-18 20:12 ` Jens Axboe
2026-11-18 20:53 ` Hamza Mahfooz
2026-11-18 21:43 ` Keith Busch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=sashiko-outbox-221909@kernel.org \
--to=sashiko-bot@kernel.org \
--cc=linux-hyperv@vger.kernel.org \
--cc=sashiko-reviews@lists.linux.dev \
--cc=srivatsa@csail.mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox