[RFC PATCH v1] blk-mq: isolate CPUs from hctx

[RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Costa Shulyupin]

The housekeeping CPU masks, set up by the "isolcpus" and "nohz_full"
boot command line options, are used at boot time to exclude selected
CPUs from running some kernel housekeeping subsystems to minimize
disturbance to latency sensitive userspace applications such as DPDK.
This options can only be changed with a reboot. This is a problem for
containerized workloads running on OpenShift/Kubernetes where a
mix of low latency and "normal" workloads can be created/destroyed
dynamically and the number of CPUs allocated to each workload is often
not known at boot time.

Cgroups allow configuring isolated_cpus at runtime.
However, blk-mq may still use managed interrupts on the
newly isolated CPUs.

Rebuild hctx->cpumask considering isolated CPUs to avoid
managed interrupts on those CPUs and reclaim non-isolated ones.

The patch is based on
isolation: Exclude dynamically isolated CPUs from housekeeping masks:
https://lore.kernel.org/lkml/20240821142312.236970-1-longman@redhat.com/

Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
---
 block/blk-mq.c         | 30 ++++++++++++++++++++++++++++++
 include/linux/blk-mq.h |  1 +
 kernel/cgroup/cpuset.c |  2 ++
 3 files changed, 33 insertions(+)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 12ee37986331..d5786b953d17 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -4145,6 +4145,36 @@ static void blk_mq_map_swqueue(struct request_queue *q)
 	}
 }
 
+/**
+ * blk_mq_isolate_cpus() - rebuild hctx->cpumask considering isolated CPUs
+ * to avoid managed interrupts on those CPUs.
+ */
+
+void blk_mq_isolate_cpus(const struct cpumask *isolcpus)
+{
+	struct class_dev_iter iter;
+	struct device *dev;
+
+	class_dev_iter_init(&iter, &block_class, NULL, &disk_type);
+	while ((dev = class_dev_iter_next(&iter))) {
+		struct request_queue *q = bdev_get_queue(dev_to_bdev(dev));
+		struct blk_mq_hw_ctx *hctx;
+		unsigned long i;
+
+		if (!queue_is_mq(q))
+			continue;
+
+		blk_mq_map_swqueue(q);
+		/*
+		 * Postcondition:
+		 * cpumask must not intersect with isolated CPUs.
+		 */
+		queue_for_each_hw_ctx(q, hctx, i)
+			WARN_ON_ONCE(cpumask_intersects(hctx->cpumask, isolcpus));
+	}
+	class_dev_iter_exit(&iter);
+}
+
 /*
  * Caller needs to ensure that we're either frozen/quiesced, or that
  * the queue isn't live yet.
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 2035fad3131f..a1f57b5ad46d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -924,6 +924,7 @@ void blk_freeze_queue_start_non_owner(struct request_queue *q);
 
 void blk_mq_map_queues(struct blk_mq_queue_map *qmap);
 void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues);
+void blk_mq_isolate_cpus(const struct cpumask *isolcpus);
 
 void blk_mq_quiesce_queue_nowait(struct request_queue *q);
 
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 5066397899c9..cad17f3f3315 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -41,6 +41,7 @@
 #include <linux/sched/isolation.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
+#include <linux/blk-mq.h>
 
 #undef pr_fmt
 #define pr_fmt(fmt)    "%s:%d: %s " fmt, __FILE__, __LINE__, __func__
@@ -1317,6 +1318,7 @@ static void update_isolation_cpumasks(bool isolcpus_updated)
 		return;
 	ret = housekeeping_exlude_isolcpus(isolated_cpus, HOUSEKEEPING_FLAGS);
 	WARN_ON_ONCE((ret < 0) && (ret != -EOPNOTSUPP));
+	blk_mq_isolate_cpus(isolated_cpus);
 }
 
 /**
-- 
2.47.0

Re: [RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Daniel Wagner]

On Fri, Nov 08, 2024 at 07:48:30AM +0200, Costa Shulyupin wrote:
> The housekeeping CPU masks, set up by the "isolcpus" and "nohz_full"
> boot command line options, are used at boot time to exclude selected
> CPUs from running some kernel housekeeping subsystems to minimize
> disturbance to latency sensitive userspace applications such as DPDK.
> This options can only be changed with a reboot. This is a problem for
> containerized workloads running on OpenShift/Kubernetes where a
> mix of low latency and "normal" workloads can be created/destroyed
> dynamically and the number of CPUs allocated to each workload is often
> not known at boot time.
>
> Cgroups allow configuring isolated_cpus at runtime.
> However, blk-mq may still use managed interrupts on the
> newly isolated CPUs.
>
> Rebuild hctx->cpumask considering isolated CPUs to avoid
> managed interrupts on those CPUs and reclaim non-isolated ones.

As far I understand this doesn't address the issue that the drivers
need also to be aware of isolcpu mask changes. That means even though
the cpumask is updated in the block layer, the driver doesn't know about
it and still runs on the isolated CPUs.

Re: [RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Michal Koutný]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

Hello.

On Fri, Nov 08, 2024 at 07:48:30AM GMT, Costa Shulyupin <costa.shul@redhat.com> wrote:
> Cgroups allow configuring isolated_cpus at runtime.
> However, blk-mq may still use managed interrupts on the
> newly isolated CPUs.
> 
> Rebuild hctx->cpumask considering isolated CPUs to avoid
> managed interrupts on those CPUs and reclaim non-isolated ones.
> 
> The patch is based on
> isolation: Exclude dynamically isolated CPUs from housekeeping masks:
> https://lore.kernel.org/lkml/20240821142312.236970-1-longman@redhat.com/

Even based on that this seems incomplete to me the CPUs that are part of
isolcpus mask on boot time won't be excluded from this?
IOW, isolating CPUs from blk_mq_hw_ctx would only be possible via cpuset
but not "statically" throught the cmdline option, or would it?

Thanks,
Michal

(-Cc: lizefan.x@bytedance.com)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Waiman Long]

On 11/15/24 10:45 AM, Michal Koutný wrote:
> Hello.
>
> On Fri, Nov 08, 2024 at 07:48:30AM GMT, Costa Shulyupin <costa.shul@redhat.com> wrote:
>> Cgroups allow configuring isolated_cpus at runtime.
>> However, blk-mq may still use managed interrupts on the
>> newly isolated CPUs.
>>
>> Rebuild hctx->cpumask considering isolated CPUs to avoid
>> managed interrupts on those CPUs and reclaim non-isolated ones.
>>
>> The patch is based on
>> isolation: Exclude dynamically isolated CPUs from housekeeping masks:
>> https://lore.kernel.org/lkml/20240821142312.236970-1-longman@redhat.com/
> Even based on that this seems incomplete to me the CPUs that are part of
> isolcpus mask on boot time won't be excluded from this?
> IOW, isolating CPUs from blk_mq_hw_ctx would only be possible via cpuset
> but not "statically" throught the cmdline option, or would it?

The cpuset had already been changed to take note of the statically 
isolated CPUs and included them in its consideration. They are recorded 
in the boot_hk_cpus mask. It relies on the fact that most users will set 
both nohz_full and isolcpus boot parameters. If only nohz_full is set 
without isolcpus, it will not be recorded.

Cheers,
Longman

Re: [RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Costa Shulyupin]

Hello Michal.

Isolation of CPUs from blk_mq_hw_ctx during boot is already handled on
call hierarchy:
...
        nvme_probe()
                nvme_alloc_admin_tag_set()
                        blk_mq_alloc_queue()
                                blk_mq_init_allocated_queue()
                                        blk_mq_map_swqueue()

blk_mq_map_swqueue() performs:
for_each_cpu(cpu, hctx->cpumask) {
        if (cpu_is_isolated(cpu))
                cpumask_clear_cpu(cpu, hctx->cpumask);
}

static inline bool cpu_is_isolated(int cpu)
{
        return !housekeeping_test_cpu(cpu, HK_TYPE_DOMAIN) ||
                !housekeeping_test_cpu(cpu, HK_TYPE_TICK) ||
               cpuset_cpu_is_isolated(cpu);
}

cpu_is_isolated() is introduced by  3232e7aad11e5.

Thanks,
Costa


On Fri, 15 Nov 2024 at 17:45, Michal Koutný <mkoutny@suse.com> wrote:
>
> Hello.
>
> On Fri, Nov 08, 2024 at 07:48:30AM GMT, Costa Shulyupin <costa.shul@redhat.com> wrote:
> > Cgroups allow configuring isolated_cpus at runtime.
> > However, blk-mq may still use managed interrupts on the
> > newly isolated CPUs.
> >
> > Rebuild hctx->cpumask considering isolated CPUs to avoid
> > managed interrupts on those CPUs and reclaim non-isolated ones.
> >
> > The patch is based on
> > isolation: Exclude dynamically isolated CPUs from housekeeping masks:
> > https://lore.kernel.org/lkml/20240821142312.236970-1-longman@redhat.com/
>
> Even based on that this seems incomplete to me the CPUs that are part of
> isolcpus mask on boot time won't be excluded from this?
> IOW, isolating CPUs from blk_mq_hw_ctx would only be possible via cpuset
> but not "statically" throught the cmdline option, or would it?
>
> Thanks,
> Michal
>
> (-Cc: lizefan.x@bytedance.com)

Re: [RFC PATCH v1] blk-mq: isolate CPUs from hctx

[Waiman Long]

On 11/15/24 3:25 PM, Costa Shulyupin wrote:
> Hello Michal.
>
> Isolation of CPUs from blk_mq_hw_ctx during boot is already handled on
> call hierarchy:
> ...
>          nvme_probe()
>                  nvme_alloc_admin_tag_set()
>                          blk_mq_alloc_queue()
>                                  blk_mq_init_allocated_queue()
>                                          blk_mq_map_swqueue()
>
> blk_mq_map_swqueue() performs:
> for_each_cpu(cpu, hctx->cpumask) {
>          if (cpu_is_isolated(cpu))
>                  cpumask_clear_cpu(cpu, hctx->cpumask);
> }
>
> static inline bool cpu_is_isolated(int cpu)
> {
>          return !housekeeping_test_cpu(cpu, HK_TYPE_DOMAIN) ||
>                  !housekeeping_test_cpu(cpu, HK_TYPE_TICK) ||
>                 cpuset_cpu_is_isolated(cpu);
> }

cpuset_cpu_is_isolated() can be removed once the cpumasks can be changed 
dynamically.

Cheers,
Longman