[PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Daniel Wagner]

blk_mq_alloc_request_hctx uses the first online CPU in hctx->cpumask to
select the software context for a request.

Since commit a46c27026da1 ("blk-mq: don't schedule block kworker on
isolated CPUs"), hctx->cpumask excludes isolated CPUs. If a hardware
context is mapped only to isolated CPUs, the mask is empty and
blk_mq_alloc_request_hctx fails with -EXDEV.

The NVMe fabrics transports use blk_mq_alloc_request_hctx to allocate
the Connect command for each I/O queue. As a result, connecting to a
controller can fail when using isolcpus=domain or an isolated cpuset
partition.

Passthrough requests bypass the software queues and go directly to
hctx->dispatch. Their software context is only used to select the
completion CPU and as a hint when freeing the tag.

Use the software context of an online housekeeping CPU for passthrough
requests.

Fixes: a46c27026da1 ("blk-mq: don't schedule block kworker on isolated CPUs")
Tested-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Daniel Wagner <dwagner@suse.de>
---
Connecting a fabrics controller fails when all CPUs of a hardware
context are isolated with isolcpus=domain or an isolated cpuset
partition. Since commit a46c27026da1 ("blk-mq: don't schedule block
kworker on isolated CPUs") the CPU mask of such a hardware context is
empty and blk_mq_alloc_request_hctx() fails the Connect command with
-EXDEV. This patch lets passthrough requests use the software context
of a housekeeping CPU instead.

The nvme fabrics fixes from v1 (patches 1 and 3-5) are in nvme-7.4, so
only the blk-mq patch is left. The blktest for this patch is nvme/070,
posted separately.
---
Changes in v2:
- Only the blk-mq patch left, the nvme patches are in nvme-7.4.
- Reword the commit message to describe what the fix does (Hannes).
- Add Tested-by from Maurizio.
- Link to v1: https://patch.msgid.link/20261014-nvme-fabrics-isolcpu-v1-0-355156be27b2@kernel.org
---
 block/blk-mq.c | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 5fca78778c31..77685e6283fa 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -717,6 +717,25 @@ struct request *blk_mq_alloc_request(struct request_queue *q, blk_opf_t opf,
 }
 EXPORT_SYMBOL(blk_mq_alloc_request);
 
+/*
+ * Return an online CPU which is neither isolated from the scheduler domains
+ * nor from managed interrupts, or nr_cpu_ids if there is none.
+ */
+static unsigned int blk_mq_housekeeping_cpu(void)
+{
+	unsigned int cpu;
+
+	rcu_read_lock();
+	for_each_online_cpu(cpu) {
+		if (!cpu_is_isolated(cpu) &&
+		    housekeeping_test_cpu(cpu, HK_TYPE_MANAGED_IRQ))
+			break;
+	}
+	rcu_read_unlock();
+
+	return cpu;
+}
+
 struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	blk_opf_t opf, blk_mq_req_flags_t flags, unsigned int hctx_idx)
 {
@@ -767,8 +786,22 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
 	if (!blk_mq_hw_queue_mapped(data.hctx))
 		goto out_queue_exit;
 	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
-	if (cpu >= nr_cpu_ids)
-		goto out_queue_exit;
+	if (cpu >= nr_cpu_ids) {
+		/*
+		 * Isolated CPUs are not part of hctx->cpumask, so it can be
+		 * empty although online CPUs are mapped to this hardware
+		 * context. Passthrough requests are inserted directly into
+		 * hctx->dispatch, so use the software context of a
+		 * housekeeping CPU for them. Inactive hardware contexts have
+		 * no online CPU left.
+		 */
+		if (!blk_op_is_passthrough(opf) ||
+		    test_bit(BLK_MQ_S_INACTIVE, &data.hctx->state))
+			goto out_queue_exit;
+		cpu = blk_mq_housekeeping_cpu();
+		if (cpu >= nr_cpu_ids)
+			goto out_queue_exit;
+	}
 	data.ctx = __blk_mq_get_ctx(q, cpu);
 
 	if (q->elevator)

---
base-commit: c8c53d669d4e45b3a804013251a34387d7e11bfb
change-id: 20261013-nvme-fabrics-isolcpu-5b4948362f88

Best regards,
--  
Daniel Wagner <dwagner@suse.de>

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] blk-mq: blk_mq_alloc_request_hctx() allocates on offline hctxs when BLK_MQ_S_INACTIVE is unset
--

commit f6824e249591ee431516fa47caa778074d8f4d22
Author: Daniel Wagner <dwagner@suse.de>

blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

When hctx->cpumask contains no online CPUs because all CPUs mapped to the
hardware context are isolated, this commit allows passthrough requests in
blk_mq_alloc_request_hctx() to fall back to the software context of an
online housekeeping CPU unless BLK_MQ_S_INACTIVE is set on the hctx.

> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 5fca78778c31c..77685e6283fa1 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c

[ ... ]

> @@ -767,8 +786,22 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>  	if (!blk_mq_hw_queue_mapped(data.hctx))
>  		goto out_queue_exit;
>  	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
> -	if (cpu >= nr_cpu_ids)
> -		goto out_queue_exit;
> +	if (cpu >= nr_cpu_ids) {
> +		/*
> +		 * Isolated CPUs are not part of hctx->cpumask, so it can be
> +		 * empty although online CPUs are mapped to this hardware
> +		 * context. Passthrough requests are inserted directly into
> +		 * hctx->dispatch, so use the software context of a
> +		 * housekeeping CPU for them. Inactive hardware contexts have
> +		 * no online CPU left.
> +		 */
> +		if (!blk_op_is_passthrough(opf) ||
> +		    test_bit(BLK_MQ_S_INACTIVE, &data.hctx->state))
> +			goto out_queue_exit;
> +		cpu = blk_mq_housekeeping_cpu();
> +		if (cpu >= nr_cpu_ids)
> +			goto out_queue_exit;
> +	}

[Severity: Medium]
Can this allocate a passthrough request on a hardware context that has no
online CPUs mapped to it instead of returning ERR_PTR(-EXDEV)?

BLK_MQ_S_INACTIVE is only set in the CPU hotplug callback
blk_mq_hctx_notify_offline() when the last online CPU mapped to an hctx
transitions to offline:

block/blk-mq.c:blk_mq_hctx_notify_offline() {
	...
	if (!hctx->nr_ctx || blk_mq_hctx_has_online_cpu(hctx, cpu))
		return 0;

	set_bit(BLK_MQ_S_INACTIVE, &hctx->state);
	...
}

However, during queue initialization in blk_mq_init_allocated_queue() and
queue reconfiguration in blk_mq_update_nr_hw_queues(), blk_mq_init_hctx()
unconditionally clears BLK_MQ_S_INACTIVE, __blk_mq_add_cpuhp() registers the
hotplug state via cpuhp_state_add_instance_nocalls() (and skips registration
for BLK_MQ_F_STACKING), and blk_mq_map_swqueue() maps all present and
possible CPUs to hardware contexts without setting BLK_MQ_S_INACTIVE on
hctxs whose mapped CPUs are all currently offline.

If a queue is initialized or remapped while all CPUs assigned to data.hctx
are already offline, blk_mq_hw_queue_mapped(data.hctx) is true and
BLK_MQ_S_INACTIVE remains clear. When blk_mq_alloc_request_hctx() is called
for that hctx_idx with a passthrough operation, test_bit(BLK_MQ_S_INACTIVE,
&data.hctx->state) evaluates to false and the request is allocated using
blk_mq_housekeeping_cpu() on an offline hardware queue instead of failing
with -EXDEV.

Should this check !blk_mq_hctx_has_online_cpu(data.hctx, nr_cpu_ids) (or
ensure blk_mq_map_swqueue() sets BLK_MQ_S_INACTIVE when an hctx has no
online CPUs mapped)?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261103-nvme-fabrics-isolcpu-v2-1-95ec70d58254@suse.de?part=1

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Aaron Tomlin]

On Tue, Nov 03, 2026 at 03:57:22PM +0100, Daniel Wagner wrote:
> blk_mq_alloc_request_hctx uses the first online CPU in hctx->cpumask to
> select the software context for a request.
> 
> Since commit a46c27026da1 ("blk-mq: don't schedule block kworker on
> isolated CPUs"), hctx->cpumask excludes isolated CPUs. If a hardware
> context is mapped only to isolated CPUs, the mask is empty and
> blk_mq_alloc_request_hctx fails with -EXDEV.
> 
> The NVMe fabrics transports use blk_mq_alloc_request_hctx to allocate
> the Connect command for each I/O queue. As a result, connecting to a
> controller can fail when using isolcpus=domain or an isolated cpuset
> partition.
> 
> Passthrough requests bypass the software queues and go directly to
> hctx->dispatch. Their software context is only used to select the
> completion CPU and as a hint when freeing the tag.
> 
> Use the software context of an online housekeeping CPU for passthrough
> requests.
> 
> Fixes: a46c27026da1 ("blk-mq: don't schedule block kworker on isolated CPUs")
> Tested-by: Maurizio Lombardi <mlombard@redhat.com>
> Signed-off-by: Daniel Wagner <dwagner@suse.de>
> ---
> Connecting a fabrics controller fails when all CPUs of a hardware
> context are isolated with isolcpus=domain or an isolated cpuset
> partition. Since commit a46c27026da1 ("blk-mq: don't schedule block
> kworker on isolated CPUs") the CPU mask of such a hardware context is
> empty and blk_mq_alloc_request_hctx() fails the Connect command with
> -EXDEV. This patch lets passthrough requests use the software context
> of a housekeeping CPU instead.
> 
> The nvme fabrics fixes from v1 (patches 1 and 3-5) are in nvme-7.4, so
> only the blk-mq patch is left. The blktest for this patch is nvme/070,
> posted separately.
> ---
> Changes in v2:
> - Only the blk-mq patch left, the nvme patches are in nvme-7.4.
> - Reword the commit message to describe what the fix does (Hannes).
> - Add Tested-by from Maurizio.
> - Link to v1: https://patch.msgid.link/20261014-nvme-fabrics-isolcpu-v1-0-355156be27b2@kernel.org
> ---
>  block/blk-mq.c | 37 +++++++++++++++++++++++++++++++++++--
>  1 file changed, 35 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 5fca78778c31..77685e6283fa 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -717,6 +717,25 @@ struct request *blk_mq_alloc_request(struct request_queue *q, blk_opf_t opf,
>  }
>  EXPORT_SYMBOL(blk_mq_alloc_request);
>  
> +/*
> + * Return an online CPU which is neither isolated from the scheduler domains
> + * nor from managed interrupts, or nr_cpu_ids if there is none.
> + */
> +static unsigned int blk_mq_housekeeping_cpu(void)
> +{
> +	unsigned int cpu;
> +
> +	rcu_read_lock();
> +	for_each_online_cpu(cpu) {
> +		if (!cpu_is_isolated(cpu) &&
> +		    housekeeping_test_cpu(cpu, HK_TYPE_MANAGED_IRQ))
> +			break;
> +	}
> +	rcu_read_unlock();
> +
> +	return cpu;
> +}
> +
>  struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>  	blk_opf_t opf, blk_mq_req_flags_t flags, unsigned int hctx_idx)
>  {
> @@ -767,8 +786,22 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>  	if (!blk_mq_hw_queue_mapped(data.hctx))
>  		goto out_queue_exit;
>  	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
> -	if (cpu >= nr_cpu_ids)
> -		goto out_queue_exit;
> +	if (cpu >= nr_cpu_ids) {
> +		/*
> +		 * Isolated CPUs are not part of hctx->cpumask, so it can be
> +		 * empty although online CPUs are mapped to this hardware
> +		 * context. Passthrough requests are inserted directly into
> +		 * hctx->dispatch, so use the software context of a
> +		 * housekeeping CPU for them. Inactive hardware contexts have
> +		 * no online CPU left.
> +		 */
> +		if (!blk_op_is_passthrough(opf) ||
> +		    test_bit(BLK_MQ_S_INACTIVE, &data.hctx->state))
> +			goto out_queue_exit;
> +		cpu = blk_mq_housekeeping_cpu();
> +		if (cpu >= nr_cpu_ids)
> +			goto out_queue_exit;
> +	}
>  	data.ctx = __blk_mq_get_ctx(q, cpu);
>  
>  	if (q->elevator)
> 
> ---
> base-commit: c8c53d669d4e45b3a804013251a34387d7e11bfb
> change-id: 20261013-nvme-fabrics-isolcpu-5b4948362f88
> 
> Best regards,
> --  
> Daniel Wagner <dwagner@suse.de>
> 

Thank you.

Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>

-- 
Aaron Tomlin

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Sebastian Andrzej Siewior]

On 2026-11-03 15:57:22 [+0100], Daniel Wagner wrote:
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -717,6 +717,25 @@ struct request *blk_mq_alloc_request(struct request_queue *q, blk_opf_t opf,
>  }
>  EXPORT_SYMBOL(blk_mq_alloc_request);
>  
> +/*
> + * Return an online CPU which is neither isolated from the scheduler domains
> + * nor from managed interrupts, or nr_cpu_ids if there is none.
> + */
> +static unsigned int blk_mq_housekeeping_cpu(void)
> +{
> +	unsigned int cpu;
> +
> +	rcu_read_lock();
> +	for_each_online_cpu(cpu) {
> +		if (!cpu_is_isolated(cpu) &&
> +		    housekeeping_test_cpu(cpu, HK_TYPE_MANAGED_IRQ))
> +			break;
> +	}

What about
	guard(rcu);
	for_each_cpu_and(cpu, cpu_online_mask, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ)) {
		if (!cpu_is_isolated(cpu)
			break;
	}

this ends up with one less function call ;)

> +	rcu_read_unlock();
> +
> +	return cpu;
> +}
> +
>  struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>  	blk_opf_t opf, blk_mq_req_flags_t flags, unsigned int hctx_idx)
>  {

Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>

Sebastian

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Nilay Shroff]

On 11/3/26 8:27 PM, Daniel Wagner wrote:
> blk_mq_alloc_request_hctx uses the first online CPU in hctx->cpumask to
> select the software context for a request.
> 
> Since commit a46c27026da1 ("blk-mq: don't schedule block kworker on
> isolated CPUs"), hctx->cpumask excludes isolated CPUs. If a hardware
> context is mapped only to isolated CPUs, the mask is empty and
> blk_mq_alloc_request_hctx fails with -EXDEV.
> 
> The NVMe fabrics transports use blk_mq_alloc_request_hctx to allocate
> the Connect command for each I/O queue. As a result, connecting to a
> controller can fail when using isolcpus=domain or an isolated cpuset
> partition.
> 
> Passthrough requests bypass the software queues and go directly to
> hctx->dispatch. Their software context is only used to select the
> completion CPU and as a hint when freeing the tag.
> 
> Use the software context of an online housekeeping CPU for passthrough
> requests.
> 
> Fixes: a46c27026da1 ("blk-mq: don't schedule block kworker on isolated CPUs")
> Tested-by: Maurizio Lombardi<mlombard@redhat.com>
> Signed-off-by: Daniel Wagner<dwagner@suse.de>

Looks good to me:
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Christoph Hellwig]

On Tue, Nov 03, 2026 at 05:37:48PM +0100, Sebastian Andrzej Siewior wrote:
> What about
> 	guard(rcu);
> 	for_each_cpu_and(cpu, cpu_online_mask, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ)) {
> 		if (!cpu_is_isolated(cpu)
> 			break;
> 	}
> 
> this ends up with one less function call ;)

Please avoid this completely unreadable and unaditable guard crap.

How is anyone supposed to understand that above mess?

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Sebastian Andrzej Siewior]

On 2026-11-04 08:17:03 [+0100], Christoph Hellwig wrote:
> On Tue, Nov 03, 2026 at 05:37:48PM +0100, Sebastian Andrzej Siewior wrote:
> > What about
> > 	guard(rcu);
> > 	for_each_cpu_and(cpu, cpu_online_mask, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ)) {
> > 		if (!cpu_is_isolated(cpu)
> > 			break;
> > 	}
> > 
> > this ends up with one less function call ;)
> 
> Please avoid this completely unreadable and unaditable guard crap.

If you/ block prefers to avoid guard() then fine. I usually get punched
the other way around.

> How is anyone supposed to understand that above mess?

I can show you code where you start crying. Anyway. My main selling
point was for_each_cpu_and().

Sebastian

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Christoph Hellwig]

On Wed, Nov 04, 2026 at 08:37:10AM +0100, Sebastian Andrzej Siewior wrote:
> On 2026-11-04 08:17:03 [+0100], Christoph Hellwig wrote:
> > On Tue, Nov 03, 2026 at 05:37:48PM +0100, Sebastian Andrzej Siewior wrote:
> > > What about
> > > 	guard(rcu);
> > > 	for_each_cpu_and(cpu, cpu_online_mask, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ)) {
> > > 		if (!cpu_is_isolated(cpu)
> > > 			break;
> > > 	}
> > > 
> > > this ends up with one less function call ;)
> > 
> > Please avoid this completely unreadable and unaditable guard crap.
> 
> If you/ block prefers to avoid guard() then fine. I usually get punched
> the other way around.
> 
> > How is anyone supposed to understand that above mess?
> 
> I can show you code where you start crying.

I don't cry easily, but I've basically given up trying to understand
or fix code infested too much by at and other CLASS/__cleanup magic
like the core VFS code, because my brain can't cope with it...

> Anyway. My main selling
> point was for_each_cpu_and().

I guess the advantage would be to reduce the iteration, so it's 
probably ok.  I don't find it particularly easier to read, and given
that this is a slow path it might not matter.

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Hannes Reinecke]

On 11/3/26 3:57 PM, Daniel Wagner wrote:
> blk_mq_alloc_request_hctx uses the first online CPU in hctx->cpumask to
> select the software context for a request.
> 
> Since commit a46c27026da1 ("blk-mq: don't schedule block kworker on
> isolated CPUs"), hctx->cpumask excludes isolated CPUs. If a hardware
> context is mapped only to isolated CPUs, the mask is empty and
> blk_mq_alloc_request_hctx fails with -EXDEV.
> 
> The NVMe fabrics transports use blk_mq_alloc_request_hctx to allocate
> the Connect command for each I/O queue. As a result, connecting to a
> controller can fail when using isolcpus=domain or an isolated cpuset
> partition.
> 
> Passthrough requests bypass the software queues and go directly to
> hctx->dispatch. Their software context is only used to select the
> completion CPU and as a hint when freeing the tag.
> 
> Use the software context of an online housekeeping CPU for passthrough
> requests.
> 
> Fixes: a46c27026da1 ("blk-mq: don't schedule block kworker on isolated CPUs")
> Tested-by: Maurizio Lombardi <mlombard@redhat.com>
> Signed-off-by: Daniel Wagner <dwagner@suse.de>
> ---
> Connecting a fabrics controller fails when all CPUs of a hardware
> context are isolated with isolcpus=domain or an isolated cpuset
> partition. Since commit a46c27026da1 ("blk-mq: don't schedule block
> kworker on isolated CPUs") the CPU mask of such a hardware context is
> empty and blk_mq_alloc_request_hctx() fails the Connect command with
> -EXDEV. This patch lets passthrough requests use the software context
> of a housekeeping CPU instead.
> 
> The nvme fabrics fixes from v1 (patches 1 and 3-5) are in nvme-7.4, so
> only the blk-mq patch is left. The blktest for this patch is nvme/070,
> posted separately.
> ---
> Changes in v2:
> - Only the blk-mq patch left, the nvme patches are in nvme-7.4.
> - Reword the commit message to describe what the fix does (Hannes).
> - Add Tested-by from Maurizio.
> - Link to v1: https://patch.msgid.link/20261014-nvme-fabrics-isolcpu-v1-0-355156be27b2@kernel.org
> ---
>   block/blk-mq.c | 37 +++++++++++++++++++++++++++++++++++--
>   1 file changed, 35 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 5fca78778c31..77685e6283fa 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -717,6 +717,25 @@ struct request *blk_mq_alloc_request(struct request_queue *q, blk_opf_t opf,
>   }
>   EXPORT_SYMBOL(blk_mq_alloc_request);
>   
> +/*
> + * Return an online CPU which is neither isolated from the scheduler domains
> + * nor from managed interrupts, or nr_cpu_ids if there is none.
> + */
> +static unsigned int blk_mq_housekeeping_cpu(void)
> +{
> +	unsigned int cpu;
> +
> +	rcu_read_lock();
> +	for_each_online_cpu(cpu) {
> +		if (!cpu_is_isolated(cpu) &&
> +		    housekeeping_test_cpu(cpu, HK_TYPE_MANAGED_IRQ))
> +			break;
> +	}
> +	rcu_read_unlock();
> +
> +	return cpu;
> +}
> +
>   struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>   	blk_opf_t opf, blk_mq_req_flags_t flags, unsigned int hctx_idx)
>   {
> @@ -767,8 +786,22 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
>   	if (!blk_mq_hw_queue_mapped(data.hctx))
>   		goto out_queue_exit;
>   	cpu = cpumask_first_and(data.hctx->cpumask, cpu_online_mask);
> -	if (cpu >= nr_cpu_ids)
> -		goto out_queue_exit;
> +	if (cpu >= nr_cpu_ids) {
> +		/*
> +		 * Isolated CPUs are not part of hctx->cpumask, so it can be
> +		 * empty although online CPUs are mapped to this hardware
> +		 * context. Passthrough requests are inserted directly into
> +		 * hctx->dispatch, so use the software context of a
> +		 * housekeeping CPU for them. Inactive hardware contexts have
> +		 * no online CPU left.
> +		 */
> +		if (!blk_op_is_passthrough(opf) ||
> +		    test_bit(BLK_MQ_S_INACTIVE, &data.hctx->state))
> +			goto out_queue_exit;
> +		cpu = blk_mq_housekeeping_cpu();
> +		if (cpu >= nr_cpu_ids)
> +			goto out_queue_exit;
> +	}
>   	data.ctx = __blk_mq_get_ctx(q, cpu);
>   
>   	if (q->elevator)
> 
> ---
> base-commit: c8c53d669d4e45b3a804013251a34387d7e11bfb
> change-id: 20261013-nvme-fabrics-isolcpu-5b4948362f88
> 
> Best regards,
> --
> Daniel Wagner <dwagner@suse.de>
> 

Reviewed-by: Hannes Reinecke <hare@kernel.org>

Cheers,

Hannes
-- 
Dr. Hannes Reinecke                  Kernel Storage Architect
hare@suse.de                                +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Daniel Wagner]

On Tue, Nov 03, 2026 at 03:22:29PM +0000, sashiko-bot@kernel.org wrote:
> Should this check !blk_mq_hctx_has_online_cpu(data.hctx, nr_cpu_ids) (or
> ensure blk_mq_map_swqueue() sets BLK_MQ_S_INACTIVE when an hctx has no
> online CPUs mapped)?

Yes, this is correct. I'll extend the patch and add this to the commit message:

    When all mapped CPUs are offline at queue setup, BLK_MQ_S_INACTIVE is
    not set, as only the CPU offline callback sets it. The
    blk_mq_hctx_has_online_cpu test catches this case. It uses the queue
    map, so it also finds online isolated CPUs.

    When the last mapped CPU goes offline, the CPU offline callback sets
    BLK_MQ_S_INACTIVE while the CPU is still in cpu_online_mask, so
    blk_mq_hctx_has_online_cpu still finds it. The BLK_MQ_S_INACTIVE test
    catches this case.

Re: [PATCH v2] blk-mq: use a housekeeping CPU for passthrough requests on isolated hctxs

[Daniel Wagner]

On Wed, Nov 04, 2026 at 11:22:40AM +0100, Daniel Wagner wrote:
> On Tue, Nov 03, 2026 at 03:22:29PM +0000, sashiko-bot@kernel.org wrote:
> > Should this check !blk_mq_hctx_has_online_cpu(data.hctx, nr_cpu_ids) (or
> > ensure blk_mq_map_swqueue() sets BLK_MQ_S_INACTIVE when an hctx has no
> > online CPUs mapped)?
> 
> Yes, this is correct. I'll extend the patch and add this to the commit message:
> 
>     When all mapped CPUs are offline at queue setup, BLK_MQ_S_INACTIVE is
>     not set, as only the CPU offline callback sets it. The
>     blk_mq_hctx_has_online_cpu test catches this case. It uses the queue
>     map, so it also finds online isolated CPUs.
> 
>     When the last mapped CPU goes offline, the CPU offline callback sets
>     BLK_MQ_S_INACTIVE while the CPU is still in cpu_online_mask, so
>     blk_mq_hctx_has_online_cpu still finds it. The BLK_MQ_S_INACTIVE test
>     catches this case.

I have to correct myself. Allocating on a hardware context with only
offline CPUs is intended, so the patch shouldn't add the check.

blk-mq maps all possible CPUs to the hardware contexts since
20e4d8139319 ("blk-mq: simplify queue mapping & schedule with each
possisble CPU"). While CPUs are offline, a hardware context can end up
with only offline CPUs. blk_mq_alloc_request_hctx() then fails the
Connect command with -EXDEV, and connecting a fabrics controller fails,
for example when booted with maxcpus=4 on an 8-CPU machine. The queue is
still needed once the CPUs come online.

I wrote a blktest for this (nvme/080). It fails on current upstream.
With the patch the connect works, and I/O from the CPUs goes through
that queue after they come online. Hardware contexts whose last CPU
went offline are inactive and still fail.

v3 keeps the v2 code and adds this case to the commit message, with a
second Fixes tag for 20e4d8139319.