* [PATCH v8 09/11] virt/steal_governor: Add control knobs for handling steal values
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
These are the knobs to control the steal_governor.
interval_ms:
How often steal governor checks for steal time.
(Default: 1000 i.e 1 second)
This controls how fast steal governor driver reacts to changes to
the contention of physical CPUs.
Can be set between 100 to 100000. i.e. 100ms to 100seconds.
100ms is kept as minimum to ensure few meaningful steal values
accumulate even with HZ=100.
low_threshold:
lower threshold value in percentage * 100.
(Default: 200, i.e 2% steal is considered as low threshold)
This determines what values should be considered as nil/no steal values.
When steal governor see steal time is below or equal to this value, it
will increase the preferred CPUs by 1 core. Having value as zero
might cause oscillations
high_threshold:
higher threshold value in percentage * 100
(Default: 500, i.e 5% steal is considered as high threshold)
This determines what values should be considered as high steal values.
When steal governor sees steal time is higher than this value, it will
reduce the preferred CPUs by 1 core.
module_param_cb methods are used to do the validation checks.
This helps to ensure one configures sane values.
Since low and high are dependent, that check is done at module init.
Parameters values can't be changed at runtime. One has to unload
the module and change it. Hence recommended to build it as module.
Also available at: Documentation/driver-api/steal-governor.rst
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
drivers/virt/steal_governor/core.c | 73 +++++++++++++++++++++++++++++-
1 file changed, 71 insertions(+), 2 deletions(-)
diff --git a/drivers/virt/steal_governor/core.c b/drivers/virt/steal_governor/core.c
index 055f155d2045..1cb766a8ce28 100644
--- a/drivers/virt/steal_governor/core.c
+++ b/drivers/virt/steal_governor/core.c
@@ -17,7 +17,11 @@
#error "Steal Governor requires CONFIG_PREFERRED_CPU"
#endif
-static struct steal_governor sg_core_ctx;
+static struct steal_governor sg_core_ctx = {
+ .interval_ms = 1000, /* 1 second */
+ .high_threshold = 500, /* 5% */
+ .low_threshold = 200, /* 2% */
+};
static void restore_preferred_to_active(void)
{
@@ -28,9 +32,74 @@ static void restore_preferred_to_active(void)
set_cpu_preferred(cpu, true);
}
+static int param_set_interval_ms(const char *val, const struct kernel_param *kp)
+{
+ unsigned int interval;
+ int ret;
+
+ ret = kstrtouint(val, 0, &interval);
+ if (ret)
+ return ret;
+
+ if (interval < 100 || interval > 100000) {
+ pr_err("steal_governor: interval_ms must be between 100 and 100000\n");
+ return -EINVAL;
+ }
+
+ return param_set_uint(val, kp);
+}
+
+static const struct kernel_param_ops interval_ms_ops = {
+ .set = param_set_interval_ms,
+ .get = param_get_uint,
+};
+
+module_param_cb(interval_ms, &interval_ms_ops, &sg_core_ctx.interval_ms, 0444);
+MODULE_PARM_DESC(interval_ms,
+ "Sampling frequency in milliseconds. default: 1000");
+
+static int param_set_high_threshold(const char *val, const struct kernel_param *kp)
+{
+ unsigned int threshold;
+ int ret;
+
+ ret = kstrtouint(val, 0, &threshold);
+ if (ret)
+ return ret;
+
+ if (threshold >= 100 * 100) {
+ pr_err("steal_governor: high_threshold (%u) can't be more than 99.99%%\n",
+ threshold);
+ return -EINVAL;
+ }
+
+ return param_set_uint(val, kp);
+}
+
+static const struct kernel_param_ops high_threshold_ops = {
+ .set = param_set_high_threshold,
+ .get = param_get_uint,
+};
+
+module_param_cb(high_threshold, &high_threshold_ops, &sg_core_ctx.high_threshold, 0444);
+MODULE_PARM_DESC(high_threshold,
+ "High steal threshold. default: 500 i.e 5%. Must be > low_threshold");
+
+module_param_named(low_threshold, sg_core_ctx.low_threshold, uint, 0444);
+MODULE_PARM_DESC(low_threshold,
+ "Low steal threshold. default: 200 i.e 2%. Must be < high_threshold");
+
static int __init steal_governor_init(void)
{
- pr_info("steal_governor is enabled\n");
+ if (sg_core_ctx.low_threshold >= sg_core_ctx.high_threshold) {
+ pr_err("steal_governor: low_threshold (%u) must be less than high_threshold (%u)\n",
+ sg_core_ctx.low_threshold, sg_core_ctx.high_threshold);
+ return -EINVAL;
+ }
+
+ pr_info("steal_governor is enabled. interval: %ums, high_threshold: %u, low_threshold: %u\n",
+ sg_core_ctx.interval_ms, sg_core_ctx.high_threshold, sg_core_ctx.low_threshold);
+
return 0;
}
--
2.47.3
^ permalink raw reply related
* [PATCH v8 08/11] virt: Introduce steal governor driver
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
Introduce a new driver in virt named steal_governor. This driver
will compute the steal time and drive the policy decisions of preferred
CPU state.
More on it can be found in the Documentation/driver-api/steal-governor.rst
There is a new kconfig called STEAL_GOVERNOR which is introduced in
subsequent patches. That driver is going to select PREFERRED_CPU.
This makes configs driven by user preference.
When the driver is disabled, preferred CPUs is same as active CPUs.
File layout of the driver is being kept simple.
- core.c - contains main driver code. This includes the periodic
work function and take action on steal time which is introduced
in subsequent patches.
- core.h - header file which includes data structure.
Main structure of steal governor has,
- work: deferred periodic work function
- steal, time: To calculate the delta in periodic work.
- interval_ms, high_threshold, low_threshold: debug knobs of
steal_governor.
While there, Add MAINTAINERS entry for this new driver.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
Documentation/driver-api/index.rst | 1 +
Documentation/driver-api/steal-governor.rst | 117 ++++++++++++++++++++
MAINTAINERS | 9 ++
drivers/virt/steal_governor/core.c | 48 ++++++++
drivers/virt/steal_governor/core.h | 25 +++++
5 files changed, 200 insertions(+)
create mode 100644 Documentation/driver-api/steal-governor.rst
create mode 100644 drivers/virt/steal_governor/core.c
create mode 100644 drivers/virt/steal_governor/core.h
diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst
index eaf7161ff957..0a973b59cba3 100644
--- a/Documentation/driver-api/index.rst
+++ b/Documentation/driver-api/index.rst
@@ -138,6 +138,7 @@ Subsystem-specific APIs
sm501
soundwire/index
spi
+ steal-governor
surface_aggregator/index
switchtec
sync_file
diff --git a/Documentation/driver-api/steal-governor.rst b/Documentation/driver-api/steal-governor.rst
new file mode 100644
index 000000000000..1039d598fc2c
--- /dev/null
+++ b/Documentation/driver-api/steal-governor.rst
@@ -0,0 +1,117 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==============
+Steal Governor
+==============
+
+:Author: Shrikanth Hegde <sshegde@linux.ibm.com>
+
+Introduction
+============
+
+Steal governor is a driver aimed at solving the Noisy Neighbour problem
+in paravirtualized environments. The performance of workload
+running in one VM gets affected significantly due to other VMs and
+combined they make slower forward progress.
+
+When there is overcommit of CPU resources, i.e. sum of virtual CPUs (vCPUs)
+of all VMs is greater than number of physical CPUs (pCPUs) and
+when all or many VMs have high utilization, hypervisor won't be able
+to satisfy the CPU requirement and has to context switch within or
+across VMs. I.e. the hypervisor needs to preempt one vCPU to run
+another. This is called vCPU preemption.
+This is more expensive compared to task context switch within a vCPU.
+
+In such cases it is better that combined vCPU ask from all VMs is reduced
+by not using some of the vCPUs. vCPUs where workload can be safely
+scheduled which won't increase any contention for pCPU are called as
+"Preferred CPUs".
+
+See more on "Preferred CPUs" in Documentation/scheduler/sched-arch.rst.
+
+This driver makes CONFIG_PREFERRED_CPU=y which enables the scheduler core
+infrastructure to move tasks to Preferred CPUs where possible.
+
+Core idea
+=========
+
+steal time is an indication available today in Guest which shows contention
+for underlying physical CPU. Use it as a hint in the guest to fold the
+workload to a reduced set of vCPUs. When there is contention, steal time
+will show up in all the guests. When each guest honors the hint and folds
+the workload to a smaller set of vCPUs (Preferred CPUs), it reduces the
+contention and thereby reduces vCPU preemption.
+This is achieved without any cross-guest communication.
+
+Steal governor driver effectively does:
+
+1. Periodically computes steal time across the system.
+
+2. If steal time is greater than high threshold, reduce the number of
+ preferred CPUs by 1 core. Ensure at least one core is left always.
+
+3. If steal time is lower or equal to low threshold, increase the
+ number of preferred CPUs by 1 core. If preferred is same as active,
+ nothing to be done.
+
+4. Ensure preferred CPUs is always subset of active CPUs.
+ On feature disable it is same as active CPUs.
+
+This feature works best only when all the VMs enable the feature as
+it is a co-operative scheme. If a specific VM doesn't enable this feature
+it may end up with more CPUs than others, still should lead to better
+performance when seen from system view.
+Those who enable this driver must ensure it is enabled in all VMs.
+
+Module Parameters
+=================
+
+interval_ms
+-----------
+
+How often steal governor checks for steal time.
+Default: 1000 i.e 1 second. Value should be in between 100ms to 100sec.
+
+This controls how fast steal governor driver reacts to changes to
+the contention of physical CPUs. Since it does a fair amount of
+work, setting too low may have overhead. Setting it too
+high might render it ineffective.
+
+low_threshold
+-------------
+
+lower threshold value in percentage * 100.
+Default: 200, i.e 2% steal is considered as low threshold.
+Can't be higher than high_threshold.
+
+This determines what values should be considered as nil/no steal values.
+When steal governor see steal time is below or equal to this value, it
+will increase the preferred CPUs by 1 core. Having value as zero
+might cause oscillations.
+
+high_threshold
+--------------
+
+higher threshold value in percentage * 100
+Default: 500, i.e 5% steal is considered as high threshold.
+Can't be lower than low_threshold. Must be less than 10000.
+
+This determines what values should be considered as high steal values.
+When steal governor sees steal time is higher than this value, it will
+reduce the preferred CPUs by 1 core.
+
+Notes
+=====
+
+Selecting this driver makes CONFIG_PREFERRED_CPU=y. That makes configs
+driven by user preference.
+
+It is recommended to build CONFIG_STEAL_GOVERNOR=m due to below reasons:
+
+1. Doing periodic work has additional overheads. Enabling this driver
+ in systems where steal time cannot happen is of no use. There is no
+ benefit with additional overheads in such systems.
+
+2. This works well when all VMs work in co-operative manner. When an
+ administrative user enables it in one VM, he/she will likely enable
+ it all VMs.
diff --git a/MAINTAINERS b/MAINTAINERS
index 15011f5752a9..0906684b2243 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -25914,6 +25914,15 @@ F: rust/helpers/jump_label.c
F: rust/kernel/generated_arch_static_branch_asm.rs.S
F: rust/kernel/jump_label.rs
+STEAL GOVERNOR DRIVER
+M: Shrikanth Hegde <sshegde@linux.ibm.com>
+R: Yury Norov <yury.norov@gmail.com>
+L: linux-kernel@vger.kernel.org
+S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git sched/core
+F: Documentation/driver-api/steal-governor.rst
+F: drivers/virt/steal_governor/
+
STI AUDIO (ASoC) DRIVERS
M: Arnaud Pouliquen <arnaud.pouliquen@foss.st.com>
L: linux-sound@vger.kernel.org
diff --git a/drivers/virt/steal_governor/core.c b/drivers/virt/steal_governor/core.c
new file mode 100644
index 000000000000..055f155d2045
--- /dev/null
+++ b/drivers/virt/steal_governor/core.c
@@ -0,0 +1,48 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Steal time governor driver periodically computes steal time.
+ * Based on the thresholds it either reduce/increase the preferred
+ * CPUs which can be used by the workload to avoid vCPU preemption
+ * to an extent possible in paravirtualized environment.
+ *
+ * Available as module with CONFIG_STEAL_GOVERNOR
+ *
+ * Copyright (C) 2026 IBM
+ * Author: Shrikanth Hegde <sshegde@linux.ibm.com>
+ */
+
+#include "core.h"
+
+#if !IS_ENABLED(CONFIG_PREFERRED_CPU)
+#error "Steal Governor requires CONFIG_PREFERRED_CPU"
+#endif
+
+static struct steal_governor sg_core_ctx;
+
+static void restore_preferred_to_active(void)
+{
+ int cpu;
+
+ guard(cpus_read_lock)();
+ for_each_cpu(cpu, cpu_active_mask)
+ set_cpu_preferred(cpu, true);
+}
+
+static int __init steal_governor_init(void)
+{
+ pr_info("steal_governor is enabled\n");
+ return 0;
+}
+
+static void __exit steal_governor_exit(void)
+{
+ restore_preferred_to_active();
+ pr_info("steal_governor is disabled\n");
+}
+
+module_init(steal_governor_init);
+module_exit(steal_governor_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("IBM Corporation");
+MODULE_DESCRIPTION("Virtualization Steal Time Governor");
diff --git a/drivers/virt/steal_governor/core.h b/drivers/virt/steal_governor/core.h
new file mode 100644
index 000000000000..e27305284ac0
--- /dev/null
+++ b/drivers/virt/steal_governor/core.h
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __VIRT_STEAL_CORE_H
+#define __VIRT_STEAL_CORE_H
+
+#include <linux/types.h>
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/cpuhplock.h>
+#include <linux/cpumask.h>
+#include <linux/workqueue.h>
+#include <linux/ktime.h>
+#include <linux/kconfig.h>
+
+struct steal_governor {
+ struct delayed_work work;
+ ktime_t time;
+ u64 steal;
+ unsigned int interval_ms;
+ unsigned int high_threshold;
+ unsigned int low_threshold;
+};
+
+#endif /* __VIRT_STEAL_CORE_H */
--
2.47.3
^ permalink raw reply related
* [PATCH v8 07/11] sched/debug: Add migration stats due to non preferred CPUs
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
Add a new stat,
- nr_migrations_cpu_non_preferred: number of migrations happened since
a CPU was marked as non preferred due to high steal time.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
include/linux/sched.h | 1 +
kernel/sched/core.c | 9 +++++++--
kernel/sched/debug.c | 1 +
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 968b18a7f470..37849d2f1dbd 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -554,6 +554,7 @@ struct sched_statistics {
u64 nr_failed_migrations_running;
u64 nr_failed_migrations_hot;
u64 nr_forced_migrations;
+ u64 nr_migrations_cpu_non_preferred;
u64 nr_wakeups;
u64 nr_wakeups_sync;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 704043531b24..f0d9bcfd35e3 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -11324,8 +11324,13 @@ static int sched_non_preferred_cpu_push_stop(void *arg)
context_unsafe_alias(rq);
if (task_rq(p) == rq && task_on_rq_queued(p) &&
- !is_migration_disabled(p))
- rq = __migrate_task(rq, &rf, p, cpu);
+ !is_migration_disabled(p)) {
+ struct rq *dest_rq = __migrate_task(rq, &rf, p, cpu);
+
+ if (rq != dest_rq)
+ schedstat_inc(p->stats.nr_migrations_cpu_non_preferred);
+ rq = dest_rq;
+ }
rq_unlock(rq, &rf);
raw_spin_unlock_irq(&p->pi_lock);
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 72236db67983..5ebb2055e6d5 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -1446,6 +1446,7 @@ void proc_sched_show_task(struct task_struct *p, struct pid_namespace *ns,
P_SCHEDSTAT(nr_failed_migrations_running);
P_SCHEDSTAT(nr_failed_migrations_hot);
P_SCHEDSTAT(nr_forced_migrations);
+ P_SCHEDSTAT(nr_migrations_cpu_non_preferred);
P_SCHEDSTAT(nr_wakeups);
P_SCHEDSTAT(nr_wakeups_sync);
P_SCHEDSTAT(nr_wakeups_migrate);
--
2.47.3
^ permalink raw reply related
* [PATCH v8 06/11] sched/core: Push current task from non preferred CPU
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
Actively push out task running on a non-preferred CPU. Since the task is
running on the CPU, need to stop the cpu and push the task out.
However, if the task is pinned only to non-preferred CPUs, it will continue
running there. This will help in maintaining the userspace affinities
unlike CPU hotplug or isolated cpusets.
Though code is similar to __balance_push_cpu_stop and quite close to
push_cpu_stop, it is being kept separate as it provides a cleaner
implementation with CONFIG_PREFERRED_CPU.
Add push_task_work_done flag to protect work buffer.
Works only with FAIR class.
For now, only current running task is pushed out. This keeps the code
simpler. In future optimization maybe done to move all the queued
task on the rq.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
kernel/sched/core.c | 78 ++++++++++++++++++++++++++++++++++++++++++++
kernel/sched/sched.h | 8 +++++
2 files changed, 86 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 9e8eec4451b6..704043531b24 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5774,6 +5774,9 @@ void sched_tick(void)
unsigned long hw_pressure;
u64 resched_latency;
+ if (!cpu_preferred(cpu))
+ sched_push_current_non_preferred_cpu(rq);
+
if (housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE))
arch_scale_freq_tick();
@@ -11292,3 +11295,78 @@ void sched_change_end(struct sched_change_ctx *ctx)
p->sched_class->prio_changed(rq, p, ctx->prio);
}
}
+
+#ifdef CONFIG_PREFERRED_CPU
+static DEFINE_PER_CPU(struct cpu_stop_work, npc_push_task_work);
+
+static int sched_non_preferred_cpu_push_stop(void *arg)
+{
+ struct task_struct *p = arg;
+ struct rq *rq = this_rq();
+ struct rq_flags rf;
+ int cpu;
+
+ if (cpu_preferred(rq->cpu)) {
+ scoped_guard(rq_lock, rq)
+ rq->push_task_work_done = false;
+ put_task_struct(p);
+ return 0;
+ }
+
+ raw_spin_lock_irq(&p->pi_lock);
+
+ /* This could take rq lock. So call it before rq lock is taken */
+ cpu = select_fallback_rq(rq->cpu, p);
+ rq_lock(rq, &rf);
+ rq->push_task_work_done = false;
+ update_rq_clock(rq);
+
+ context_unsafe_alias(rq);
+
+ if (task_rq(p) == rq && task_on_rq_queued(p) &&
+ !is_migration_disabled(p))
+ rq = __migrate_task(rq, &rf, p, cpu);
+
+ rq_unlock(rq, &rf);
+ raw_spin_unlock_irq(&p->pi_lock);
+ put_task_struct(p);
+
+ return 0;
+}
+
+/*
+ * Push the current task running on non-preferred CPU(npc).
+ * Using this non preferred CPU will lead to more contention
+ * in the host. So it is better not to use this CPU.
+ *
+ * Since task is running, call a stopper to push the task out. This is
+ * similar to how task moves during hotplug. In select_fallback_rq a
+ * preferred CPU will be chosen and henceforth task shouldn't come back to
+ * this CPU again.
+ *
+ * Works for FAIR class only.
+ *
+ * If task is affined only non-preferred CPUs, no point in moving it out.
+ */
+void sched_push_current_non_preferred_cpu(struct rq *rq)
+{
+ struct task_struct *push_task = rq->curr;
+
+ scoped_guard(rq_lock, rq) {
+ /* Push the task if its explicit affinity allows */
+ if (!task_can_sched_on_preferred(rq->cpu, push_task))
+ return;
+
+ /* There is already a stopper thread. Don't race with it. */
+ if (rq->push_task_work_done)
+ return;
+
+ rq->push_task_work_done = true;
+ }
+
+ /* sched_tick runs with interrupts disabled. */
+ get_task_struct(push_task);
+ stop_one_cpu_nowait(rq->cpu, sched_non_preferred_cpu_push_stop,
+ push_task, this_cpu_ptr(&npc_push_task_work));
+}
+#endif
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 6de6366f2faa..80c02e2c09eb 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1277,6 +1277,8 @@ struct rq {
struct list_head cfs_tasks;
+ bool push_task_work_done;
+
struct sched_avg avg_rt;
struct sched_avg avg_dl;
#ifdef CONFIG_HAVE_SCHED_AVG_IRQ
@@ -4242,4 +4244,10 @@ static inline bool task_can_sched_on_preferred(int cpu, struct task_struct *p)
return cpumask_intersects(p->cpus_ptr, cpu_preferred_mask);
}
+#ifdef CONFIG_PREFERRED_CPU
+void sched_push_current_non_preferred_cpu(struct rq *rq);
+#else /* !CONFIG_PREFERRED_CPU */
+static inline void sched_push_current_non_preferred_cpu(struct rq *rq) { }
+#endif
+
#endif /* _KERNEL_SCHED_SCHED_H */
--
2.47.3
^ permalink raw reply related
* [PATCH v8 05/11] sched/fair: Load balance only among preferred CPUs
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
When cpu is marked as non preferred, any load pulled towards it is
pointless since in the next tick task will be pushed out again.
So, Consider only preferred CPUs for load balance.
This makes it not fight against the push task mechanism which happens
at tick. Also, this stops active balance to happen on non-preferred CPU
pulling the load.
This means there is no load balancing if the task is pinned only to
non-preferred CPUs. They will continue to run where they were previously
running before the CPUs was marked as non-preferred.
Bailout early for NEWIDLE and IDLE balance as load balancing is done
only on preferred CPUs.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
kernel/sched/fair.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index df8c9c2c7918..12f5b7de28d2 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -13399,7 +13399,7 @@ static int sched_balance_rq(int this_cpu, struct rq *this_rq,
};
bool need_unlock = false;
- cpumask_and(cpus, sched_domain_span(sd), cpu_active_mask);
+ cpumask_and(cpus, sched_domain_span(sd), cpu_preferred_mask);
schedstat_inc(sd->lb_count[idle]);
@@ -14337,7 +14337,8 @@ static void _nohz_idle_balance(struct rq *this_rq, unsigned int flags)
update_rq_clock(rq);
rq_unlock_irqrestore(rq, &rf);
- if (flags & NOHZ_BALANCE_KICK)
+ if (flags & NOHZ_BALANCE_KICK &&
+ cpu_preferred(balance_cpu))
sched_balance_domains(rq, CPU_IDLE);
}
@@ -14481,10 +14482,8 @@ static int sched_balance_newidle(struct rq *this_rq, struct rq_flags *rf)
*/
this_rq->idle_stamp = rq_clock(this_rq);
- /*
- * Do not pull tasks towards !active CPUs...
- */
- if (!cpu_active(this_cpu))
+ /* Do not pull tasks towards !preferred CPUs */
+ if (!cpu_preferred(this_cpu))
return 0;
/*
--
2.47.3
^ permalink raw reply related
* [PATCH v8 04/11] sched/core: Try to use a preferred CPU in is_cpu_allowed
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
When possible, choose a preferred CPUs to pick.
This is essential to maintain user affinities when preferred
CPUs change. A task pinned on non-preferred CPU should continue
to run there, since this is non-user triggered.
If CPU is non-preferred and task can run on other CPUs which are
currently preferred, then choose those other CPUs instead.
This is decided by checking cpus_ptr and cpu_preferred_mask
intersect or not. If yes, task has other preferred CPUs, it can
run there instead.
Overhead is minimal when CPU is preferred.
Push task mechanism uses stopper thread which going to call
select_fallback_rq and use this mechanism to pick only a preferred CPU.
This takes care of wakeup path for FAIR tasks too.
is_cpu_allowed is called to ensure wakeup happens on preferred CPUs.
With that, additional checks in available_idle_cpu is not necessary.
For majority of the cases this would still keep select_fallback_rq
as O(N). task_has_preferred_cpus which is O(N) is called only if
!cpu_preferred. Then task running there is expected to move out.
So subsequent it should run on preferred CPU. This becomes O(N**2)
only for tasks pinned only non preferred CPUs. That is rare case.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
kernel/sched/core.c | 12 ++++++++++--
kernel/sched/sched.h | 12 ++++++++++++
2 files changed, 22 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index a45f7c308329..9e8eec4451b6 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2509,8 +2509,12 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
return cpu_online(cpu);
/* Non kernel threads are not allowed during either online or offline. */
- if (!(p->flags & PF_KTHREAD))
+ if (!(p->flags & PF_KTHREAD)) {
+ /* Try to use preferred CPU if task's affinity allows */
+ if (task_can_sched_on_preferred(cpu, p))
+ return false;
return cpu_active(cpu);
+ }
/* KTHREAD_IS_PER_CPU is always allowed. */
if (kthread_is_per_cpu(p))
@@ -2520,7 +2524,11 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
if (cpu_dying(cpu))
return false;
- /* But are allowed during online. */
+ /* Try to keep unbound kthreads on a preferred CPU if possible. */
+ if (task_can_sched_on_preferred(cpu, p))
+ return false;
+
+ /* Otherwise, they are allowed to run on online CPU. */
return cpu_online(cpu);
}
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 26ae13c86b69..6de6366f2faa 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -4230,4 +4230,16 @@ DEFINE_CLASS_IS_UNCONDITIONAL(sched_change)
#include "ext/ext.h"
+static inline bool task_can_sched_on_preferred(int cpu, struct task_struct *p)
+{
+ if (cpu_preferred(cpu))
+ return false;
+
+ /* Only FAIR tasks honor preferred CPU state */
+ if (unlikely(p->sched_class != &fair_sched_class))
+ return false;
+
+ return cpumask_intersects(p->cpus_ptr, cpu_preferred_mask);
+}
+
#endif /* _KERNEL_SCHED_SCHED_H */
--
2.47.3
^ permalink raw reply related
* [PATCH v8 03/11] sysfs: Add preferred CPU file
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
Add "preferred" file in /sys/devices/system/cpu
This would help
- Users to quickly check which CPUs are marked as preferred.
- Userspace daemons such as irqbalance to use this mask to
send irq into preferred CPUs.
For example:
cat /sys/devices/system/cpu/online
0-719
cat /sys/devices/system/cpu/preferred
0-599 <<< Implies 0-599 are preferred for workloads and 600-719
should be avoided at this moment.
cat /sys/devices/system/cpu/preferred
0-719 <<< All CPUs are usable. There is no preference.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
Documentation/ABI/testing/sysfs-devices-system-cpu | 14 ++++++++++++++
drivers/base/cpu.c | 12 ++++++++++++
2 files changed, 26 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu
index 82d10d556cc8..676412c46b2a 100644
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -806,3 +806,17 @@ Date: Nov 2022
Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org>
Description:
(RO) the list of CPUs that can be brought online.
+
+What: /sys/devices/system/cpu/preferred
+Date: July 2026
+Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org>
+Description:
+ (RO) the list of preferred CPUs applicable in
+ paravirtualized environments.
+
+ The steal governor driver dynamically adjusts this mask
+ based on observed steal time. Scheduling tasks on
+ CPUs outside of this list may lead to performance
+ degradations due to underlying physical CPU contention.
+
+ See Documentation/scheduler/sched-arch.rst for more details.
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 19d288a3c80c..5da2a96fb37e 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -391,6 +391,15 @@ static int cpu_uevent(const struct device *dev, struct kobj_uevent_env *env)
}
#endif
+#ifdef CONFIG_PREFERRED_CPU
+static ssize_t preferred_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ return sysfs_emit(buf, "%*pbl\n", cpumask_pr_args(cpu_preferred_mask));
+}
+static DEVICE_ATTR_RO(preferred);
+#endif
+
const struct bus_type cpu_subsys = {
.name = "cpu",
.dev_name = "cpu",
@@ -531,6 +540,9 @@ static struct attribute *cpu_root_attrs[] = {
#endif
#ifdef CONFIG_GENERIC_CPU_AUTOPROBE
&dev_attr_modalias.attr,
+#endif
+#ifdef CONFIG_PREFERRED_CPU
+ &dev_attr_preferred.attr,
#endif
NULL
};
--
2.47.3
^ permalink raw reply related
* [PATCH v8 00/11] sched, steal_governor: Introduce cpu_preferred_mask and steal-driven vCPU backoff
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
This patch series represents the result of multiple iterations,
redesigns and community feedback. What started as an arch-specific RFC
has evolved into a scheduler mechanism paired with a virtualization
driver.
Special thanks to Yury Norov for the rigorous reviews that greatly
improved the series and to everyone who have provided their review
comments so far. Really appreciated! _/\_
I have put a detailed context around problem statement, design, best
practises and performance numbers below. This cover-letter is a good
starting point for anyone looking into this solution without the pain of
browsing through all the previous patches/videos.
Apologies in advance if any review comments are missed or any
missing implementation for the new driver. If so would be purely
accidental, not in any way intentional.
Background and Problem Statement
================================
As hardware scales, the density of physical CPUs (pCPUs) per server is
increasing across many architectures. On these massive systems, deploying
a single bare-metal OS for general workloads becomes increasingly difficult
to manage if not impossible. The natural shift is to deploy
Virtual Machines(VMs). For example, on IBM PowerPC architecture customers
frequently deploy Shared Processor LPARs (SPLPARs) to maximize hardware ROI.
Typical enterprise workloads are combination of bursty and long running;
their average CPU utilization is low, but they require high core counts
during peak transactions. To accommodate this, customers often
use CPU overcommit strategies i.e. configuring VMs with a large number
of virtual CPUs (vCPUs) while backing them with a smaller, shared pool
of physical CPUs (pCPUs). This achieves a high server consolidation
and excellent cost efficiency.
However, when multiple such VMs have high utilization simultaneously,
the shared pCPU pool becomes contended. The hypervisor is forced to preempt
one vCPU to run another to maintain fairness. It maybe schedule vCPU of same
VM or different VM. If a vCPU is preempted while holding a lock or
irq disabled section, overall forward progress collapses. There are some
mitigation strategies such as yielding the vCPU to lock-holder, but they
don't cover all the cases. In addition there are hidden costs such as cache,
tlb misses, cost of vCPU preemption, host scheduling overheads etc.
Under heavy contention, the most effective mitigation strategy is for
the guests/VMs to voluntarily fold its workload onto a smaller subset
of its vCPUs. By demanding fewer pCPUs, the VMs reduces overall host
contention, which decreases vCPU preemption and improves total throughput
for the system.
Limitations of Existing Approaches
==================================
CPU Hotplug, Isolated cpusets, cpuset:
- This is a heavy and administrative operation that requires topology rebuild.
Crucially, it breaks userspace CPU affinities.
Explicit task affinity:
- Very difficult to manage for the users, if not impossible.
We need a fast, co-operative backoff mechanism inside the kernel that can
dynamically react to contention without violating user/task affinity
contracts. Since reacting to the contention is agnostic to the user
it cannot violate user affinity contracts.
When there is high contention, fold the workload and use limited vCPUs
and when there is no contention, use all the vCPUs again. This natural
expansion/contraction gives the best possible performance to the users
based on the underlying contention.
Proposed Architecture
=====================
Current design is built on basis that contention is effectively
quantified by steal time as seen in guest kernel.
Steal time is already a well established construct today in
para-virtualization world. All major archs support this feature.
It is indication of the contention of physical CPU. It scales according
to the amount of contention. Today it is used by administrative users
for changing the VM configurations. During high contention the steal
time shows up in each guest based on its configuration. The proposed
solution works well when all VMs honor the hint and work in co-operative
manner. Note there is still no inter-guest communication to achieve this
co-operation. Read the section on best practises on how to get the
best out of this solution.
This series introduces a dynamic vCPU backoff mechanism.
It is separated into a core scheduler mechanism and a loadable
virtualization policy module.
Layer A: The Scheduler Mechanism (preferred CPUs)
=================================================
Series introduces a new CPU state called preferred. It indicates that
vCPU can be safely used and using that vCPU won't increase contention
for underlying physical CPUs. This state info is made available via
cpu_preferred_mask, which is strictly maintained as a subset of
cpu_active_mask.
The scheduler uses this mask as a hint to fold workloads onto preferred
CPUs using a few mechanisms.
1, Wakeup: is_cpu_allowed() checks if CPU is preferred. If not calls
select_fallback_rq, which selects a preferred CPU if tasks's affinity
permits.
2. The Tick (Push): During sched_tick(), if the current CPU is non-preferred,
the scheduler actively pushes the running task onto a preferred CPU
using a stopper thread.
3. Load Balance: sched_balance_rq restricts its domain span to
cpu_preferred_mask, preventing tasks from being pulled toward
non-preferred CPUs.
Design Constraint: The scheduler strictly respects user affinities.
If a task is pinned exclusively to non-preferred CPUs, it will remain there.
The kernel will not break user/task affinity contracts.
Layer B: The Policy Engine (virt/steal_governor)
================================================
The core scheduler should not dictate virtualization policy.
Therefore, the policy is isolated into a new driver: steal_governor.
(Can be selected by CONFIG_STEAL_GOVERNOR)
This module latches onto that concept that contention is quantified by
steal time. It periodically samples the steal time values across the
system and depending on high/low steal values, takes appropriate action.
When it sees high steal times, i.e. steal time exceeds high_threshold
(default 5%), driver reduces the preferred CPUs by 1 core.
When it sees Low Steal Times, i.e. steal time drops below low_threshold
(default 2%), driver increases the preferred CPUs by 1 core.
This creates a dynamic, self-maintained stepwise loop. The guest automatically
shrinks its pCPU footprint when the host is saturated, and expands it when
the noise clears while requiring zero cross-VM communication.
Policy Design Constraints:
- Ensure at least one core is kept as preferred.
- Ensure preferred is always subset of active.
Best Practises
==============
1. Ensure all the VM run kernel which has the patches.
2. Keep CONFIG_STEAL_GOVERNOR=m. Build it as module, but don't load it by
default. When the administrative user enables it in one VM, he/she
will likely enable it in all VMs. Also module parameters can
only be changed at module load. Having it as module also allows one
to disable it to remove additional overhead it brings.
3. Keep the interval_ms=500 to 5000. I.e. between 500ms to 5second.
Though parameters allows slightly higher range.
4. Fine tune low and high threshold depending on your platform for best
results. Even where is no contention, very small steal values
might show up. So it might be better to keep low threshold higher
than 0.
Baseline and Revision History
==============================
tip/sched/core at
commit: '04998aa54848 ("sched/eevdf: Delayed dequeue task can't preempt")'
For a detailed talk on the problem and discussion on this issue, one can also
refer to the OSPM26 talk[1].
[1]: https://youtu.be/adxUKFPlOp0
[2]: https://www.ibm.com/support/pages/ibm-power-virtualization-best-practices-guide
[3]: https://www.ibm.com/docs/en/linux-on-systems?topic=bad-daytrader
v7->v8:
- Rename to STEAL_GOVERNOR from STEAL_MONITOR.
- Remove additional defaults.c and move it to core.c (Yury Norov)
- Remove SM_DIR gating for direction control. (Yury Norov)
- Enforce design constraint and restore the state if not met (Yury
Norov)
- Drop nohz_full tick enable patch.
- Move Kconfig patch as the last patch for enablement. (Yury Norov)
- Use disable_delayed_work_sync to avoid race condition during
module unload. (Yury Norov)
- Add same kconfig dependency and fail to compile the driver (Yury Norov)
- Make low < high comparison during module init instead as they
are dependent parameters (Sashiko)
- Update sysfs file helper section (Yury Norov)
- Make preferred sysfs file available only with CONFIG_PREFERRED_CPU=y
(Yury Norov)
- A few documentation and comments fixes. (Randy Dunlap)
- Fix possible race in sched_push_current_non_preferred_cpu (Yury Norov)
- Move is_migration_disabled check just before actual migration.
- Make 100ms as minimal interval_ms from 10ms.
- Make helper functions static and remove from header file as there
are no other callers.
- Collapse helper functions and periodic work into one patch.
Short summary on previous versions:
v6->v7:
- Consolidate new driver code to 4-5 patches.
- deffer the arch specific interface.
- Use possible CPUs instead of active for steal value calculations.
- Simplify is_cpu_allowed.
- Make module parameters fixed at module load
- Define CONFIG_STEAL_MONITOR and Make it select CONFIG_PREFERRED_CPU
v5->v6:
- Drop the optimization of caching the preferred state
in select_fallback_rq
- Drop wakeup patch
v4->v5:
- Move the computation of steal time and decide on preferred CPU state
to a driver. i.e new driver called STEAL_MONITOR
v3->v4:
- Make preferred subset of active instead of online.
- Dropped RT patch and Defer sched_ext. Support only FAIR class.
v2->v3:
- Introduce a new config CONFIG_PREFERRED_CPU
v1->v2:
- A new name - Preferred CPUs and cpu_preferred_mask
- Arch independent code. Everything happens in scheduler.
- Steal time computation is gated with sched feature STEAL_MONITOR
RFC v3-> RFC v4:
- Introduced computation of steal time in arch/powerpc.
RFC PATCH v1:
- push task mechanism.
- No steal time computation. Manual sysfs hint for preferred CPUs
v1: https://lore.kernel.org/all/236f4925-dd3c-41ef-be04-47708c9ce129@linux.ibm.com/
v2: https://lore.kernel.org/all/20260407191950.643549-1-sshegde@linux.ibm.com/#t
v3: https://lore.kernel.org/all/20260514152204.481115-1-sshegde@linux.ibm.com/#r
v4: https://lore.kernel.org/all/20260617174139.155540-1-sshegde@linux.ibm.com/#t
v5: https://lore.kernel.org/all/20260625124648.802832-1-sshegde@linux.ibm.com/
v6: https://lore.kernel.org/all/20260701141654.500125-1-sshegde@linux.ibm.com/#t
v7: https://lore.kernel.org/all/20260709215648.1246821-1-sshegde@linux.ibm.com/
Even earlier version:
https://lore.kernel.org/all/236f4925-dd3c-41ef-be04-47708c9ce129@linux.ibm.com/
========================================
Performance Numbers (powerpc, x86, s390)
========================================
PowerPC:
===================
VM1: 60VP/30EC and VM2: 30VP/20EC
Shared physical CPU pool size: 50 Cores. Each core is SMT8.
(VP - Virtual Core, EC - Entitles Core) - PowerVM terminologies of SPLPAR[2]
Default parameter values: 1000ms, 200 low threshold, 500 high threshold
Both the VMs are running the same workload. Total throughput/time of VM1+VM2
is being mentioned in all cases.
Hackbench
baseline steal_governor steal_governor
disabled enabled
======================================================================
10 groups 5.20 | 5.40 (-3.85%) | 4.65 (+10.58%)
20 groups 11.39 | 12.01 (-5.44%) | 7.09 (+37.75%)
40 groups 20.32 | 19.80 (+2.56%) | 11.31 (+44.34%)
10 groups(-p) 2.37 | 2.26 (+4.64%) | 2.06 (+13.08%)
20 groups(-p) 3.34 | 3.28 (+1.80%) | 3.20 (+4.19%)
40 groups(-p) 4.46 | 4.83 (-8.30%) | 4.26 (+4.48%)
Remarks: Net improvement with steal_governor specially high load points.
schbench ( -L -n 0 -r 30 -s 0)
baseline steal_governor steal_governor
disabled enabled
======================================================================
-m 1 -t 128 2475162 | 2621246 (+5.90%) | 2527299 (+2.11%)
-m 1 -t 256 1467350 | 1470032 (+0.18%) | 1492372 (+1.71%)
-m 1 -t 512 1408813 | 1454687 (+3.26%) | 1437605 (+2.04%)
Remarks: Effectively means no-improvements or regressions
kernbench baseline steal_governor steal_governor
(elapsed time) disabled enabled
======================================================================
-j nr_cpus 231 | 235 (-1.7%) | 199 (+14%)
Remarks: Net improvement in elapsed time.
Daytrader - A real life work which is a proxy for trading based
on db2[3]
baseline steal_governor steal_governor
disabled enabled
======================================================================
Load@30% 1x | 0.96x | 1.53x
Load@60% 1x | 0.94x | 1.41x
Remarks: Good improvement seen at different load points.
When there is no steal time (such as dedicated LPAR, or only VM2
is running) throughput was same with steal_governor enabled/disabled
which indicates minimal overhead of steal_governor.
Data from x86,s390 KVM which Ilya Leoshkevich carried out during OSPM26
time. *This was based on v2*. Idea is still the name, numbers are
expected to be better in v8 as some of the overhead has been removed.
Note: Other variations of the benchmark shows no observable
difference.
x86:
====
cascade-lake: 32 threads = 16 cores
Benchmark #VMs #CPUs/VM ΔRPS (%std)
===============================================
hackbench 8 16 90.73% ± 9.97%
hackbench 4 24 52.67% ± 7.43%
hackbench 4 16 37.96% ± 11.19%
hackbench 4 32 37.82% ± 4.38%
hackbench 12 8 36.90% ± 4.74%
hackbench 8 8 35.30% ± 3.61%
pgbench 16 4 31.77% ± 2.44%
hackbench 2 24 25.85% ± 8.63%
hackbench 16 8 24.87% ± 3.46%
pgbench 16 8 21.83% ± 2.20%
pgbench 12 8 21.35% ± 2.15%
pgbench 8 8 18.46% ± 1.01%
hackbench 2 32 15.56% ± 4.53%
pgbench 12 4 14.28% ± 2.04%
hackbench 16 4 14.07% ± 2.90%
hackbench 12 4 9.60% ± 3.49%
[...]
pgbench 4 8 -1.16% ± 3.60%
hackbench 4 4 -1.80% ± 9.55%
sysbench 12 4 -2.19% ± 0.78%
pgbench 4 24 -2.43% ± 4.38%
pgbench 4 32 -3.21% ± 0.79%
sysbench 16 4 -3.22% ± 1.09%
S390:
=====
z16: 16 threads = 8 cores (SMT-2)
Benchmark #VMs #CPUs/VM ΔRPS (std%)
===============================================
pgbench 2 8 73.50% ± 35.91%
pgbench 16 4 61.30% ± 4.09%
hackbench 16 4 54.11% ± 4.38%
hackbench 12 4 36.34% ± 4.63%
pgbench 12 4 34.83% ± 2.57%
hackbench 8 4 29.75% ± 5.86%
hackbench 8 8 25.98% ± 5.09%
pgbench 2 4 23.31% ± 33.44%
pgbench 2 16 19.95% ± 17.12%
hackbench 4 8 19.43% ± 9.33%
pgbench 8 4 19.32% ± 4.50%
[...]
schbench 8 8 -0.79% ± 0.33%
sysbench 8 8 -0.81% ± 0.39%
hackbench 4 16 -1.11% ± 5.82%
sysbench 8 4 -1.62% ± 0.49%
sysbench 16 4 -2.70% ± 0.58%
schbench 16 4 -2.73% ± 0.91%
sysbench 12 4 -2.91% ± 0.61%
hackbench 2 24 -4.99% ± 3.31%
Summary:
- Many improvement across archs specially with real life workloads.
- No major regressions observed.
- Overhead of steal_governor looks minimal when there is no steal time.
- Overhead when STEAL_GOVERNOR=n is negligible.
Testing and Validation
======================
Apart from performance, To ensure the robustness of the preferred
CPU masking and push mechanisms, the following scenarios were tested:
- CPU Hotplug: bringing CPUs up/down change the preferred mask
accordingly under no-contention and contention.
- Housekeeping cores: Verified with different combinations of
nohz_full=<beginning, middle, end set of CPUs> to ensure that
policy engine restricts to first housekeeping core in extreme cases.
- User Affinity: Confirmed that tasks explicitly pinned to non-preferred
CPUs via taskset remain on their assigned CPUs.
- Affine Move: Confirmed the affinity move using "taskset -cp" happens
on all combinations of non-preferred, non-preferred under contention.
- Affinity and hotplug: It works as expected. I.e affinity gets
reset if all the CPUs of p->cpus_ptr go offline even if they are
non-preferred CPUs.
- Extreme load and running threads: for example 4800 stress-ng threads
on 480 CPU system and it still packs to preferred CPUs.
Known Limitations & Future Work
===============================
To keep this initial implementation clean and minimal, a few optimizations
have been deferred:
- Push all tasks on rq: Currently, the stopper thread only pushes the current
running task off a non-preferred CPU. Future optimizations may look into
migrating all queued tasks on that runqueue.
- Sched Classes: This feature currently only works for the FAIR
class. Real-time (RT) and sched_ext classes are deferred for now,
as there is no need for it.
- Arch specific hints from HW and framework for it as been deferred to
the future.
- NUMA Splicing: The steal_governor currently removes last active core
based on CPU number. It does not yet do complex NUMA-aware splicing,
expecting that CPUs are spread out uniformly across nodes in
most cases.
Shrikanth Hegde (11):
sched/docs: Document cpu_preferred_mask and Preferred CPU concept
cpumask: Introduce cpu_preferred_mask
sysfs: Add preferred CPU file
sched/core: Try to use a preferred CPU in is_cpu_allowed
sched/fair: Load balance only among preferred CPUs
sched/core: Push current task from non preferred CPU
sched/debug: Add migration stats due to non preferred CPUs
virt: Introduce steal governor driver
virt/steal_governor: Add control knobs for handling steal values
virt/steal_governor: Implement steal_governor policy loop
virt/steal_governor: Enable the driver
.../ABI/testing/sysfs-devices-system-cpu | 14 +
Documentation/driver-api/index.rst | 1 +
Documentation/driver-api/steal-governor.rst | 117 ++++++++
Documentation/scheduler/sched-arch.rst | 58 ++++
MAINTAINERS | 9 +
drivers/base/cpu.c | 12 +
drivers/virt/Kconfig | 2 +
drivers/virt/Makefile | 1 +
drivers/virt/steal_governor/Kconfig | 18 ++
drivers/virt/steal_governor/Makefile | 6 +
drivers/virt/steal_governor/core.c | 277 ++++++++++++++++++
drivers/virt/steal_governor/core.h | 30 ++
include/linux/cpumask.h | 24 ++
include/linux/sched.h | 1 +
kernel/Kconfig.preempt | 4 +
kernel/cpu.c | 6 +
kernel/sched/core.c | 100 ++++++-
kernel/sched/debug.c | 1 +
kernel/sched/fair.c | 11 +-
kernel/sched/sched.h | 20 ++
20 files changed, 704 insertions(+), 8 deletions(-)
create mode 100644 Documentation/driver-api/steal-governor.rst
create mode 100644 drivers/virt/steal_governor/Kconfig
create mode 100644 drivers/virt/steal_governor/Makefile
create mode 100644 drivers/virt/steal_governor/core.c
create mode 100644 drivers/virt/steal_governor/core.h
--
2.47.3
^ permalink raw reply
* [PATCH v8 02/11] cpumask: Introduce cpu_preferred_mask
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
Provide cpu_preferred_mask infrastructure. Define get/set macros
which could be used to get/set CPU state as preferred.
PREFERRED_CPU config will be selected by the driver which handles
steal time values. It is going to set/clear preferred CPU state.
This driver will be called steal_governor and it is introduced in
subsequent patches. It periodically samples the steal time and
decides on preferred CPU state.
A CPU is set to preferred when it becomes active. Later it may be
marked as non-preferred depending on steal time values with
steal_governor being enabled.
Always maintain design construct of preferred is subset of active.
i.e. preferred ⊆ active ⊆ online ⊆ present ⊆ possible
With PREFERRED_CPU=n, ensure set_cpu_preferred is a nop and get
method returns the active state in that case.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
include/linux/cpumask.h | 24 ++++++++++++++++++++++++
kernel/Kconfig.preempt | 4 ++++
kernel/cpu.c | 6 ++++++
kernel/sched/core.c | 5 +++++
4 files changed, 39 insertions(+)
diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h
index d3cda0544954..34d08a3d80e1 100644
--- a/include/linux/cpumask.h
+++ b/include/linux/cpumask.h
@@ -122,12 +122,20 @@ extern struct cpumask __cpu_enabled_mask;
extern struct cpumask __cpu_present_mask;
extern struct cpumask __cpu_active_mask;
extern struct cpumask __cpu_dying_mask;
+
+#ifdef CONFIG_PREFERRED_CPU
+extern struct cpumask __cpu_preferred_mask;
+#else
+#define __cpu_preferred_mask __cpu_active_mask
+#endif
+
#define cpu_possible_mask ((const struct cpumask *)&__cpu_possible_mask)
#define cpu_online_mask ((const struct cpumask *)&__cpu_online_mask)
#define cpu_enabled_mask ((const struct cpumask *)&__cpu_enabled_mask)
#define cpu_present_mask ((const struct cpumask *)&__cpu_present_mask)
#define cpu_active_mask ((const struct cpumask *)&__cpu_active_mask)
#define cpu_dying_mask ((const struct cpumask *)&__cpu_dying_mask)
+#define cpu_preferred_mask ((const struct cpumask *)&__cpu_preferred_mask)
extern atomic_t __num_online_cpus;
extern unsigned int __num_possible_cpus;
@@ -1164,6 +1172,12 @@ void init_cpu_possible(const struct cpumask *src);
#define set_cpu_active(cpu, active) assign_cpu((cpu), &__cpu_active_mask, (active))
#define set_cpu_dying(cpu, dying) assign_cpu((cpu), &__cpu_dying_mask, (dying))
+#ifdef CONFIG_PREFERRED_CPU
+#define set_cpu_preferred(cpu, preferred) assign_cpu((cpu), &__cpu_preferred_mask, (preferred))
+#else
+#define set_cpu_preferred(cpu, preferred) do { } while (0)
+#endif
+
void set_cpu_online(unsigned int cpu, bool online);
void set_cpu_possible(unsigned int cpu, bool possible);
@@ -1258,6 +1272,11 @@ static __always_inline bool cpu_dying(unsigned int cpu)
return cpumask_test_cpu(cpu, cpu_dying_mask);
}
+static __always_inline bool cpu_preferred(unsigned int cpu)
+{
+ return cpumask_test_cpu(cpu, cpu_preferred_mask);
+}
+
#else
#define num_online_cpus() 1U
@@ -1296,6 +1315,11 @@ static __always_inline bool cpu_dying(unsigned int cpu)
return false;
}
+static __always_inline bool cpu_preferred(unsigned int cpu)
+{
+ return cpu == 0;
+}
+
#endif /* NR_CPUS > 1 */
#define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
diff --git a/kernel/Kconfig.preempt b/kernel/Kconfig.preempt
index 88c594c6d7fc..de789b274ba3 100644
--- a/kernel/Kconfig.preempt
+++ b/kernel/Kconfig.preempt
@@ -192,3 +192,7 @@ config SCHED_CLASS_EXT
For more information:
Documentation/scheduler/sched-ext.rst
https://github.com/sched-ext/scx
+
+config PREFERRED_CPU
+ bool
+ depends on SMP && PARAVIRT
diff --git a/kernel/cpu.c b/kernel/cpu.c
index b3c8553d7bd6..376d297a6292 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -3103,6 +3103,11 @@ EXPORT_SYMBOL(__cpu_dying_mask);
atomic_t __num_online_cpus __read_mostly;
EXPORT_SYMBOL(__num_online_cpus);
+#ifdef CONFIG_PREFERRED_CPU
+struct cpumask __cpu_preferred_mask __read_mostly;
+EXPORT_SYMBOL_GPL(__cpu_preferred_mask);
+#endif
+
void init_cpu_present(const struct cpumask *src)
{
cpumask_copy(&__cpu_present_mask, src);
@@ -3160,6 +3165,7 @@ void __init boot_cpu_init(void)
/* Mark the boot cpu "present", "online" etc for SMP and UP case */
set_cpu_online(cpu, true);
set_cpu_active(cpu, true);
+ set_cpu_preferred(cpu, true);
set_cpu_present(cpu, true);
set_cpu_possible(cpu, true);
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2e7cde033a31..a45f7c308329 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -8690,6 +8690,9 @@ int sched_cpu_activate(unsigned int cpu)
*/
sched_set_rq_online(rq, cpu);
+ /* preferred is subset of active and follows its state */
+ set_cpu_preferred(cpu, true);
+
return 0;
}
@@ -8703,6 +8706,8 @@ int sched_cpu_deactivate(unsigned int cpu)
if (ret)
return ret;
+ set_cpu_preferred(cpu, false);
+
/*
* Remove CPU from nohz.idle_cpus_mask to prevent participating in
* load balancing when not active
--
2.47.3
^ permalink raw reply related
* [PATCH v8 01/11] sched/docs: Document cpu_preferred_mask and Preferred CPU concept
From: Shrikanth Hegde @ 2026-07-20 17:22 UTC (permalink / raw)
To: linux-kernel, mingo, peterz, juri.lelli, vincent.guittot,
yury.norov, kprateek.nayak, iii, corbet
Cc: sshegde, tglx, gregkh, pbonzini, seanjc, vschneid, huschle,
rostedt, dietmar.eggemann, maddy, srikar, hdanton, chleroy,
vineeth, frederic, arighi, pauld, christian.loehle, tj,
tommaso.cucinotta, maz, rafael, rdunlap, kernellwp, linux-doc,
jgross, virtualization, kernel test robot
In-Reply-To: <20260720172250.2257582-1-sshegde@linux.ibm.com>
Add documentation for new cpumask called cpu_preferred_mask. This could
help users in understanding what this mask is and the concept behind it.
Document how to enable it and implementation aspects of it.
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202606180717.yNM0yb41-lkp@intel.com/
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
---
Documentation/scheduler/sched-arch.rst | 58 ++++++++++++++++++++++++++
1 file changed, 58 insertions(+)
diff --git a/Documentation/scheduler/sched-arch.rst b/Documentation/scheduler/sched-arch.rst
index ed07efea7d02..1ce0f92bc3f2 100644
--- a/Documentation/scheduler/sched-arch.rst
+++ b/Documentation/scheduler/sched-arch.rst
@@ -62,6 +62,64 @@ Your cpu_idle routines need to obey the following rules:
arch/x86/kernel/process.c has examples of both polling and
sleeping idle functions.
+Preferred CPUs
+==============
+
+In virtualised environments it is possible to overcommit CPU resources. i.e.
+the sum of virtual CPUs (vCPUs) of all VMs is greater than number of physical
+CPUs (pCPUs). Under such conditions when all or many VMs have high utilization,
+hypervisor won't be able to satisfy the CPU requirement and has to context
+switch within or across VMs. The hypervisor needs to preempt one vCPU to run
+another. This is called vCPU preemption. This is more expensive compared to
+task context switch within a vCPU.
+
+In such cases it is better that combined vCPU ask from all VMs is reduced
+by not using some of the vCPUs in each VM. vCPUs where workload can be safely
+scheduled which won't increase any contention for pCPU are called as
+"Preferred CPUs".
+
+Main design construct is preferred CPUs are always a subset of active CPUs.
+In most cases preferred CPUs will be same as active CPUs, when there is pCPU
+contention, Preferred CPUs will reduce based on the amount of steal time.
+When the pCPU contention goes away as indicated by steal time, Preferred CPUs
+will become same as active CPUs again. This is done by loading the
+steal_governor driver available at drivers/virt/steal_governor.
+
+For scheduling decisions such as wakeup, pushing the task etc, needs this
+CPU state info. This is maintained in cpu_preferred_mask.
+vCPUs which are not in cpu_preferred_mask should be treated as vCPUs which
+should not be used at this moment provided it doesn't break user affinity.
+
+This is achieved by:
+
+1. Selecting a preferred CPU at wakeup using fallback mechanism.
+2. Push the task away from non-preferred CPU at tick.
+3. Only select preferred CPUs for load balance.
+
+/sys/devices/system/cpu/preferred prints the current cpu_preferred_mask in
+cpulist format.
+
+Notes:
+
+1. This feature is available under CONFIG_PREFERRED_CPU. It is selected
+ by steal_governor driver (CONFIG_STEAL_GOVERNOR). On enabling the
+ driver, CPU preferred state can change based on steal time. Without that
+ driver, preferred CPUs is same as active CPUs.
+
+2. This feature works for FAIR class only.
+
+3. A task pinned, which can't be moved to preferred CPUs will continue
+ to run based on its affinity. But no load balancing happens.
+
+4. Decision to use/not use is driven by kernel. Hence it shouldn't
+ break user affinities. One of the main reasons why CPU hotplug
+ or Isolated cpuset partitions was not a solution.
+
+5. This feature works best only when all the VMs enable the feature as
+ it is a co-operative scheme. If a specific VM doesn't enable this feature
+ it may end up with more CPUs than others, still should lead to better
+ performance when seen from system view.
+ Users who enable this driver must ensure it is enabled in all VMs.
Possible arch/ problems
=======================
--
2.47.3
^ permalink raw reply related
* Re: [External] [PATCH v5 4/8] riscv_cbqri: Add capacity controller probe and allocation device ops
From: Drew Fustini @ 2026-07-20 17:12 UTC (permalink / raw)
To: yunhui cui
Cc: Adrien Ricciardi, Alexandre Ghiti, Atish Kumar Patra, Atish Patra,
Babu Moger, Ben Horgan, Borislav Petkov, Chen Pei, Conor Dooley,
Conor Dooley, Dave Hansen, Dave Martin, Fenghua Yu, Gong Shuai,
Gong Shuai, guo.wenjia23, James Morse, Kornel Dulęba,
Krzysztof Kozlowski, liu.qingtao2, Liu Zhiwei, Palmer Dabbelt,
Paul Walmsley, Peter Newman, Radim Krčmář,
Reinette Chatre, Rob Herring, Samuel Holland,
Sebastian Andrzej Siewior, Tony Luck, Vasudevan Srinivasan,
Ved Shanbhogue, Weiwei Li, Zhanpeng Zhang, linux-kernel,
linux-riscv, x86, devicetree, linux-rt-devel, linux-doc
In-Reply-To: <CAEEQ3w=9JKswTz9BcxXRvLh-kyN_Rdu3=YUR5Rey2J9UNyd0AA@mail.gmail.com>
On Mon, Jul 20, 2026 at 08:12:28PM +0800, yunhui cui wrote:
> Hi Drew,
>
> On Wed, Jul 15, 2026 at 8:24 AM Drew Fustini <fustini@kernel.org> wrote:
> >
> > Add support for the RISC-V CBQRI capacity controller. A platform driver
> > passes a cbqri_controller_info descriptor together with the cache level
> > to riscv_cbqri_register_cc_dt(), which probes the controller and adds it
> > to the controller list.
> >
> > Assisted-by: Claude:claude-opus-4-8
> > Co-developed-by: Adrien Ricciardi <aricciardi@baylibre.com>
> > Signed-off-by: Adrien Ricciardi <aricciardi@baylibre.com>
> > Signed-off-by: Drew Fustini <fustini@kernel.org>
> > ---
> > MAINTAINERS | 3 +
> > drivers/resctrl/Kconfig | 13 +
> > drivers/resctrl/Makefile | 3 +
> > drivers/resctrl/cbqri_devices.c | 563 +++++++++++++++++++++++++++++++++++++++
> > drivers/resctrl/cbqri_internal.h | 122 +++++++++
> > include/linux/riscv_cbqri.h | 45 ++++
> > 6 files changed, 749 insertions(+)
[..]
> > +static int cbqri_probe_cc(struct cbqri_controller *ctrl)
> > +{
> > + int err, status;
> > + int ver_major, ver_minor;
> > + u64 reg;
> > +
> > + reg = cbqri_readq(ctrl->base + CBQRI_CC_CAPABILITIES_OFF);
> > + if (reg == 0)
> > + return -ENODEV;
> > +
> > + ver_minor = FIELD_GET(CBQRI_CC_CAPABILITIES_VER_MINOR_MASK, reg);
> > + ver_major = FIELD_GET(CBQRI_CC_CAPABILITIES_VER_MAJOR_MASK, reg);
> > + ctrl->cc.ncblks = FIELD_GET(CBQRI_CC_CAPABILITIES_NCBLKS_MASK, reg);
> > +
> > + pr_debug("version=%d.%d ncblks=%d cache_level=%d\n",
> > + ver_major, ver_minor,
> > + ctrl->cc.ncblks, ctrl->cache.cache_level);
> > +
> > + /*
> > + * NCBLKS == 0 would divide-by-zero in the schemata math while
> > + * ctrl->lock is held.
> > + */
> > + if (!ctrl->cc.ncblks) {
> > + pr_warn("CC at %pa has 0 capacity blocks, skipping\n",
> > + &ctrl->addr);
> > + return -ENODEV;
> > + }
> > +
> > + if (ctrl->cc.ncblks > 32) {
> > + pr_warn("CC at %pa has ncblks=%u > 32 (resctrl CBM is u32), skipping\n",
> > + &ctrl->addr, ctrl->cc.ncblks);
> > + return -ENODEV;
> > + }
>
> Could you add a short comment here, like MPAM does, to note that the
> NCBLKS <= 32 limit comes from resctrl using u32 bitmap configs?
Sure, I will add a comment that the limit comes from resctrl
representing the CBM as a u32.
Thanks,
Drew
^ permalink raw reply
* Re: [External] [PATCH v5 4/8] riscv_cbqri: Add capacity controller probe and allocation device ops
From: Drew Fustini @ 2026-07-20 17:10 UTC (permalink / raw)
To: yunhui cui
Cc: Adrien Ricciardi, Alexandre Ghiti, Atish Kumar Patra, Atish Patra,
Babu Moger, Ben Horgan, Borislav Petkov, Chen Pei, Conor Dooley,
Conor Dooley, Dave Hansen, Dave Martin, Fenghua Yu, Gong Shuai,
Gong Shuai, guo.wenjia23, James Morse, Kornel Dulęba,
Krzysztof Kozlowski, liu.qingtao2, Liu Zhiwei, Palmer Dabbelt,
Paul Walmsley, Peter Newman, Radim Krčmář,
Reinette Chatre, Rob Herring, Samuel Holland,
Sebastian Andrzej Siewior, Tony Luck, Vasudevan Srinivasan,
Ved Shanbhogue, Weiwei Li, Zhanpeng Zhang, linux-kernel,
linux-riscv, x86, devicetree, linux-rt-devel, linux-doc
In-Reply-To: <CAEEQ3w=kVgMnQ8mu=X+d3xYbANT6dNfv7w71BUwmYxP6qcUhgg@mail.gmail.com>
On Mon, Jul 20, 2026 at 05:56:29PM +0800, yunhui cui wrote:
> Hi Drew,
>
> On Wed, Jul 15, 2026 at 8:24 AM Drew Fustini <fustini@kernel.org> wrote:
> >
> > Add support for the RISC-V CBQRI capacity controller. A platform driver
> > passes a cbqri_controller_info descriptor together with the cache level
> > to riscv_cbqri_register_cc_dt(), which probes the controller and adds it
> > to the controller list.
> >
> > Assisted-by: Claude:claude-opus-4-8
> > Co-developed-by: Adrien Ricciardi <aricciardi@baylibre.com>
> > Signed-off-by: Adrien Ricciardi <aricciardi@baylibre.com>
> > Signed-off-by: Drew Fustini <fustini@kernel.org>
> > ---
> > MAINTAINERS | 3 +
> > drivers/resctrl/Kconfig | 13 +
> > drivers/resctrl/Makefile | 3 +
> > drivers/resctrl/cbqri_devices.c | 563 +++++++++++++++++++++++++++++++++++++++
> > drivers/resctrl/cbqri_internal.h | 122 +++++++++
> > include/linux/riscv_cbqri.h | 45 ++++
> > 6 files changed, 749 insertions(+)
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index a0a4b41f02c5..064a6ae2823e 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -23365,6 +23365,9 @@ L: linux-riscv@lists.infradead.org
> > S: Supported
> > F: arch/riscv/include/asm/qos.h
> > F: arch/riscv/kernel/qos.c
> > +F: drivers/resctrl/cbqri_devices.c
> > +F: drivers/resctrl/cbqri_internal.h
> > +F: include/linux/riscv_cbqri.h
> >
> > RISC-V RPMI AND MPXY DRIVERS
> > M: Rahul Pathak <rahul@summations.net>
> > diff --git a/drivers/resctrl/Kconfig b/drivers/resctrl/Kconfig
> > index 672abea3b03c..92b9c82cf9f3 100644
> > --- a/drivers/resctrl/Kconfig
> > +++ b/drivers/resctrl/Kconfig
> > @@ -29,3 +29,16 @@ config ARM64_MPAM_RESCTRL_FS
> > default y if ARM64_MPAM_DRIVER && RESCTRL_FS
> > select RESCTRL_RMID_DEPENDS_ON_CLOSID
> > select RESCTRL_ASSIGN_FIXED
> > +
> > +menuconfig RISCV_CBQRI
> > + bool "RISC-V CBQRI support"
> > + depends on RISCV && RISCV_ISA_SSQOSID
> > + help
> > + Capacity and Bandwidth QoS Register Interface (CBQRI) support for
> > + RISC-V cache QoS resources. CBQRI exposes cache capacity
> > + allocation through the resctrl filesystem at /sys/fs/resctrl when
> > + RESCTRL_FS is also enabled.
> > +
> > +if RISCV_CBQRI
> > +
> > +endif
>
> Should this empty if/endif block go in patch [8/8] instead?
Good point. I'll move it to patch 8.
Thanks,
Drew
^ permalink raw reply
* Re: [PATCH v5 00/20] vfio/pci: Base Live Update support for VFIO
From: Vipin Sharma @ 2026-07-20 17:08 UTC (permalink / raw)
To: Yanjun.Zhu
Cc: kexec, linux-kernel, linux-doc, kvm, linux-mm, linux-kselftest,
ajayachandra, alex, amastro, ankita, apopple, bhelgaas, chrisl,
christian.koenig, corbet, dmatlack, graf, jacob.pan, jgg, jgg,
jrhilke, julianr, kees, kevin.tian, leon, leonro, lukas, mattev,
michal.winiarski, parav, pasha.tatashin, praan, pratyush, rananta,
rientjes, rodrigo.vivi, rppt, saeedm, schnelle, skhan, skhawaja,
vivek.kasireddy, witu, yi.l.liu
In-Reply-To: <8fd94fad-e457-4839-9e41-ae257a138bee@linux.dev>
On Thu, Jul 16, 2026 at 06:22:07PM -0700, Yanjun.Zhu wrote:
>
> On 7/16/26 4:24 PM, Yanjun.Zhu wrote:
> >
> >
> > On 7/14/26 8:14 AM, Vipin Sharma wrote:
> > > Testing
> > > -------
> > >
> > > Tested using VFIO Live Update selftests in both QEMU and bare-metal
> > > environment (Intel DSA PCIe device).
> >
> > I want to make tests VFIO Live Update selftests in QEMU. But how to
> > select a PCIe device?
> >
> > what device do the PCI ID 0000:00:04.0 and 0000:6a:01.0 mean in the
> > following testcase?
Sorry, this was my copy-paste mistake when combining things from both
qemu and baremetal. You only need one device not two separate ones.
For testing using QEMU, you can use QEMU PCI test device.
-device pci-testdev
In VM, use `lspci` command to find the BDF of the PCI device. It will show
output like:
00:04.0 Unclassified device [00ff]: Red Hat, Inc. QEMU PCI Test Device
Then prefix the domain (generally, 0000) to the BDF and use the testing
commands.
`0000:6a:01.0` was the BDF of the Intel DSA device on my host. I
incorrectly mixed two devices during my copy-paste.
Limitation of qemu pci-testdev is that it does not have the DMA
capability. I used physical host with Intel DSA to verify that the
device performing DMA during live update gets reset.
> >
> In my qemu,
>
> 00:04.0 System peripheral: InnoTek Systemberatung GmbH VirtualBox Guest
> Service
> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
> Stepping- SERR- FastB2B- DisINTx-
> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
> <MAbort- >SERR- <PERR- INTx-
> Interrupt: pin A routed to IRQ 20
> Region 0: I/O ports at d040 [size=32]
> Region 1: Memory at f0400000 (32-bit, non-prefetchable) [size=4M]
> Region 2: Memory at f0800000 (32-bit, prefetchable) [size=16K]
> Kernel driver in use: vboxguest
> Kernel modules: vboxguest
>
> I am wondering if the above pci device can work well with this patch set or
> not.
I think you should be able to test Live Update with any PCI device which
can be attached vfio-pci driver.
`vfio_pci_liveupdate_kexec_test` will initiate DMA if the VFIO selftest
framework has the driver for the passed device (Intel DSA) otherwise it
will just skip over the DMA flow and do the remaining things.
Let me know if you still face some issue in running Live Update, I will
be happy to provide more details.
Thanks
Vipin
^ permalink raw reply
* Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot
From: Babu Moger @ 2026-07-20 17:00 UTC (permalink / raw)
To: Reinette Chatre, tony.luck, bp
Cc: x86, Dave.Martin, james.morse, corbet, skhan, tglx, mingo,
dave.hansen, hpa, linux-kernel, linux-doc, eranian, peternewman
In-Reply-To: <d337915a-ef4c-43b2-a915-e97381b7dca0@intel.com>
Hi Reinette,
Thanks for quick response.
On 7/17/26 17:56, Reinette Chatre wrote:
> Hi Babu,
>
> On 7/17/26 2:13 PM, Babu Moger wrote:
>> The kernel currently enables the ABMC-based "mbm_event" mode by default on
>> hardware that supports it. However, this can cause bandwidth monitoring
>> failures with existing userspace tools such as pqos.
>>
>> The pqos tool mounts the resctrl filesystem and creates 16 or more resctrl
>> groups by default. On systems with 32 or fewer ABMC counters, this default
>> configuration can consume all available counters, since each group requires
>> one counter for local MBM and another for total MBM. If additional
>> monitoring groups are created, counter resources are exhausted and pqos
>> tool reports memory bandwidth counters as zero for those groups.
>
> It is not obvious to me that this is a problem. If I understand correctly
> there are two scenarios possible with this pqos behavior:
>
> - ABMC is not in use ("mbm_assign_mode" is set to "default")
> - pqos can create 16 or more monitor groups
> - hardware still supports a limited number of counters with consequence that
> underlying counters reset at any time as the different monitoring groups
> need to be tracked.
> - pqos can read monitoring data of all 16 monitor groups, sometimes reading the
> events would return "Unavailable", sometimes reading the events return data.
> - *None* of the monitoring numbers returned are guaranteed to be accurate.
>
> - ABMC is in use ("mbm_assign_mode" is set to "mbm_event"):
> - pqos can create 16 or more monitor groups
> - only a subset of monitoring groups have counters assigned and these counters
> are guaranteed to only track the monitor groups/events they are assigned to
> - pqos can read monitoring data of all 16 monitor groups with two possibilities:
> - monitor group/event has counter assigned: monitoring numbers are guaranteed to be accurate
> - monitor group/event does not have counter assigned: monitoring numbers return 0
>
> If my understanding is correct then the preference is to rather have wrong data than
> see 0? This does not sound right. What am I missing?
This hardware can monitor up to 64 RMIDs without any counter resets.
As you know, the pqos tool creates COS1 through COS15 regardless of the
command-line options used, resulting in a total of 16 groups including
the default group. With ABMC enabled, this consumes all available ABMC
counters(32 counters, 2 counters for each group).
When pqos is invoked with the -m option, it creates additional
monitoring groups. For example:
pqos -m all:0 -> creates 1 monitoring group
pqos -m all:0,1 -> creates 2 monitoring groups
Since all ABMC counters have already been allocated to the default set
of groups, no counters remain for these additional monitoring groups. As
a result, the monitoring commands report zero values, effectively making
monitoring unusable.
In contrast, the default monitoring mode can still support up to 48
additional monitoring groups (64 total RMIDs minus the 16 default groups
created by pqos).
For this reason, I still believe keeping the default monitoring mode as
the default is the better option.
Thanks
Babu
>
> The changelog starts with "this can cause bandwidth monitoring failures with existing
> userspace tools such as pqos". How could returning accurate memory bandwidth data be
> considered a failure? How does this issue manifest itself?
>
>>
>> Avoid this compatibility issue by leaving mbm_assign_mode in the "default"
>> mode during initialization. Users who want to use ABMC can continue to
>> enable it explicitly:
>>
>> echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
>>
>> Update the resctrl documentation to reflect the new boot-time default and
>> adjust the mbm_assign_mode examples accordingly.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> There are plans to enable "mbm_event" by default once additional counters
>> are available. For now, keep the default mode to maintain compatibility
>> with existing tools.
>
> This is not ideal. resctrl should aim to provide a consistent user interface
> across kernel versions.
>
> Reinette
>
>
^ permalink raw reply
* Re: [PATCH v7 08/11] KVM: TDX: Get/put PAMT pages when (un)mapping private memory
From: Edgecombe, Rick P @ 2026-07-20 16:48 UTC (permalink / raw)
To: sashiko-reviews@lists.linux.dev
Cc: kvm@vger.kernel.org, linux-coco@lists.linux.dev, Huang, Kai,
Hansen, Dave, Zhao, Yan Y, tony.lindgren@linux.intel.com,
Wu, Binbin, kas@kernel.org, seanjc@google.com, mingo@redhat.com,
linux-kernel@vger.kernel.org, pbonzini@redhat.com,
nik.borisov@suse.com, linux-doc@vger.kernel.org, hpa@zytor.com,
tglx@kernel.org, Annapurve, Vishal, bp@alien8.de, Gao, Chao,
x86@kernel.org
In-Reply-To: <20260718061050.E17B01F000E9@smtp.kernel.org>
On Sat, 2026-07-18 at 06:10 +0000, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Infinite kernel loop in `kvm_tdp_mmu_map_private_pfn` due to permanent PAMT cache depletion on transient TDX module contention.
> --
Our internal Sashiko found this too. It's a false positive as a real bug.
Today kvm_tdp_mmu_map_private_pfn() is only called tdx_gmem_post_populate()
during TD setup. It holds the heavyweight tdx_vm_state_guard which grabs vm-
>lock, kvm->slots_lock, and all vcpu->mutex. So there should be no contention
possible.
Any potential confusion is not new either, because a similar thing could happen
with the external page tables.
But Yan and I were discussing that it would be a good cleanup to fix this anyway
because the reason it is not a functional issue is not clear from the code. For
improved readability (and quieter sashiko reports) the topup can happen inside
the retry loop. Either by moving the retry loop or moving the topup.
>
> --- Patch [8]: [PATCH v7 08/11] KVM: TDX: Get/put PAMT pages when (un)mapping private memory ---
> commit 3474807f1c1bc4dedc75e2a756b65254c986a41d
> Author: Kiryl Shutsemau <kas@kernel.org>
>
> KVM: TDX: Get/put PAMT pages when (un)mapping private memory
>
> Add Dynamic PAMT support to KVM's S-EPT MMU by "getting" a PAMT page when
> adding guest memory (PAGE.ADD or PAGE.AUG), and "putting" the page when
> removing guest memory (PAGE.REMOVE).
>
> > diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> > index 528ebb3cb0389..049da8559882b 100644
> > --- a/arch/x86/kvm/vmx/tdx.c
> > +++ b/arch/x86/kvm/vmx/tdx.c
> > @@ -1679,16 +1693,28 @@ static struct page *tdx_spte_to_sept_pt(struct kvm *kvm, gfn_t gfn,
> > static int tdx_sept_map_nonleaf_spte(struct kvm *kvm, gfn_t gfn,
> > enum pg_level level, u64 new_spte)
> > {
> > + struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
> > gpa_t gpa = gfn_to_gpa(gfn);
> > u64 err, entry, level_state;
> > struct page *sept_pt;
> > + int ret;
> > +
> > + if (KVM_BUG_ON(!vcpu, kvm))
> > + return -EIO;
> >
> > sept_pt = tdx_spte_to_sept_pt(kvm, gfn, new_spte, level);
> > if (!sept_pt)
> > return -EIO;
> >
> > + ret = tdx_pamt_get(page_to_pfn(sept_pt), &to_tdx(vcpu)->pamt_cache);
> > + if (ret)
> > + return ret;
> > +
> > err = tdh_mem_sept_add(&to_kvm_tdx(kvm)->td, gpa, level, sept_pt,
> > &entry, &level_state);
> > + if (err)
> > + tdx_pamt_put(page_to_pfn(sept_pt));
> > +
> > if (unlikely(tdx_operand_busy(err)))
> > return -EBUSY;
>
> [Severity: High]
> Does this error path risk causing an infinite loop if the TDX module
> contention leads to TDX_OPERAND_BUSY?
>
> When tdh_mem_sept_add() returns TDX_OPERAND_BUSY, tdx_pamt_put() frees the
> PAMT pages directly back to the buddy allocator, not back into the vCPU's
> PAMT cache.
>
> If this happens during VM initialization in a retry loop like
> kvm_tdp_mmu_map_private_pfn(), the vCPU memory caches are topped up outside
> of the loop:
>
> r = mmu_topup_memory_caches(vcpu, false);
> if (r)
> return r;
>
> do {
> ...
> r = kvm_tdp_mmu_map(vcpu, &fault);
> } while (r == RET_PF_RETRY);
>
> Since the PAMT pages are discarded rather than returned to the cache, a few
> busy retries will completely empty the vCPU's PAMT cache.
>
> On the next retry, tdx_pamt_get() will return -ENOMEM because the cache is
> empty and it does not fall back to alloc_page() when a cache is provided.
>
> The TDP MMU handles -ENOMEM by returning RET_PF_RETRY, but because the cache
> is never replenished inside the loop, the loop will infinitely fail and retry.
>
> Could the PAMT pages be returned to the vCPU cache instead of the buddy
> allocator during transient failures, or should the cache top-up be moved
> inside the retry loop?
>
^ permalink raw reply
* Re: [PATCH 11/21] KVM: arm64: Context swap Partitioned PMU guest registers
From: James Clark @ 2026-07-20 16:46 UTC (permalink / raw)
To: Colton Lewis, kvm
Cc: Alexandru Elisei, Paolo Bonzini, Jonathan Corbet, Russell King,
Catalin Marinas, Will Deacon, Marc Zyngier, Oliver Upton,
Mingwei Zhang, Joey Gouly, Suzuki K Poulose, Zenghui Yu,
Mark Rutland, Shuah Khan, Ganapatrao Kulkarni, linux-doc,
linux-kernel, linux-arm-kernel, kvmarm, linux-perf-users,
linux-kselftest
In-Reply-To: <20260612192909.1153907-12-coltonlewis@google.com>
On 12/06/2026 20:28, Colton Lewis wrote:
> Save and restore newly untrapped registers that can be directly
> accessed by the guest when the PMU is partitioned.
>
> * PMEVCNTRn_EL0
> * PMCCNTR_EL0
> * PMSELR_EL0
> * PMCR_EL0
> * PMCNTEN_EL0
> * PMINTEN_EL1
>
> If we know we are not partitioned (that is, using the emulated vPMU),
> then return immediately. A later patch will make this lazy so the
> context swaps don't happen unless the guest has accessed the PMU.
>
> PMEVTYPER is handled in a following patch since we must apply the KVM
> event filter before writing values to hardware.
>
> PMOVS guest counters are cleared to avoid the possibility of
> generating spurious interrupts when PMINTEN is written. This is fine
> because the virtual register for PMOVS is always the canonical value.
>
> Signed-off-by: Colton Lewis <coltonlewis@google.com>
> ---
> arch/arm/include/asm/arm_pmuv3.h | 4 +
> arch/arm64/kvm/arm.c | 2 +
> arch/arm64/kvm/pmu-direct.c | 183 +++++++++++++++++++++++++++++++
> include/kvm/arm_pmu.h | 16 +++
> 4 files changed, 205 insertions(+)
>
[...]
> +
> +/**
> + * kvm_pmu_put() - Put untrapped PMU registers
> + * @vcpu: Pointer to struct kvm_vcpu
> + *
> + * Put all untrapped PMU registers from the VCPU into the PCPU. Mask
> + * to only bits belonging to guest-reserved counters and leave
> + * host-reserved counters alone in bitmask registers.
> + */
> +void kvm_pmu_put(struct kvm_vcpu *vcpu)
> +{
> + struct arm_pmu *pmu;
> + unsigned long guest_counters;
> + unsigned long flags;
> + u64 mask;
> + u8 i;
> + u64 val;
> +
> + /*
> + * If we aren't guest-owned then we know the guest is not
> + * accessing the PMU anyway, so no need to bother with the
> + * swap.
> + */
> + if (!kvm_pmu_is_partitioned(vcpu->kvm))
> + return;
> +
> + preempt_disable();
> +
> + pmu = vcpu->kvm->arch.arm_pmu;
> + guest_counters = kvm_pmu_guest_counter_mask(pmu);
> +
> + for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) {
> + if (i == ARMV8_PMU_CYCLE_IDX)
> + val = read_pmccntr();
> + else
> + val = read_pmevcntrn(i);
> +
> + __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + i, val);
> + }
> +
> + val = read_sysreg(pmselr_el0);
> + __vcpu_assign_sys_reg(vcpu, PMSELR_EL0, val);
> +
> + val = read_sysreg(pmcr_el0);
> + __vcpu_assign_sys_reg(vcpu, PMCR_EL0, val);
> +
> + /* Mask these to only save the guest relevant bits. */
> + mask = kvm_pmu_guest_counter_mask(pmu);
> +
> + val = read_sysreg(pmcntenset_el0);
> + __vcpu_assign_sys_reg(vcpu, PMCNTENSET_EL0, val & mask);
> +
> + val = read_sysreg(pmintenset_el1);
> + __vcpu_assign_sys_reg(vcpu, PMINTENSET_EL1, val & mask);
> +
> + /* Save pending guest hardware overflows. */
> + local_irq_save(flags);
> + val = read_sysreg(pmovsset_el0);
> + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val & mask);
> + write_sysreg(val & mask, pmovsclr_el0);
> + local_irq_restore(flags);
> +
> + /* Stop guest counters and disable interrupts in hardware. */
> + write_sysreg(mask, pmcntenclr_el0);
> + write_sysreg(mask, pmintenclr_el1);
> +
> + kvm_pmu_set_guest_counters(pmu, 0);
Hi Colton,
This function doesn't get added until "KVM: arm64: Apply dynamic guest
counter reservations" a few commits later.
kvm_pmu_guest_counter_mask() is also used in a commit before it's added.
^ permalink raw reply
* Re: [PATCH v8 00/21] ARM64 PMU Partitioning
From: James Clark @ 2026-07-20 16:46 UTC (permalink / raw)
To: Colton Lewis, kvm
Cc: Alexandru Elisei, Paolo Bonzini, Jonathan Corbet, Russell King,
Catalin Marinas, Will Deacon, Marc Zyngier, Oliver Upton,
Mingwei Zhang, Joey Gouly, Suzuki K Poulose, Zenghui Yu,
Mark Rutland, Shuah Khan, Ganapatrao Kulkarni, linux-doc,
linux-kernel, linux-arm-kernel, kvmarm, linux-perf-users,
linux-kselftest
In-Reply-To: <20260612192909.1153907-1-coltonlewis@google.com>
On 12/06/2026 8:28 pm, Colton Lewis wrote:
> This series creates a new PMU scheme on ARM, a partitioned PMU that
> allows reserving a subset of counters for more direct guest access,
> significantly reducing overhead. More details, including performance
> benchmarks, can be read in the v1 cover letter linked below.
>
> An overview of what this series accomplishes was presented at KVM
> Forum 2025. Slides [1] and video [2] are linked below.
>
> The kernel command line parameter for the driver still exists, but now
> only defines an upper limit of counters the guest might use rather
> than taking those counters from the host permanently.
>
> I would appreciate any discussion on whether that parameter should
> still exist as it's an inconvenient enabling gate on the feature that
> is no longer required. The question comes down to what, if any, guards
> we want against a guest monopolizing all counters on a system.
>
Hi Colton,
The existence of the parameter makes sense, but can't the default be
arm_pmuv3.reserved_host_counters=0 instead of -1 (partition disabled)?
It's still a bit fiddly having to do two things to make it work, and
there's no documentation about what the defaults or other prerequisites
are. IMO it's not even trivial to work out that you need to prefix it
with "arm_pmuv3.", which documentation would improve.
Testing the whole set I ran into a few issues:
This warn is hit when there is some kind of interaction with sleeping. I
tried to bisect it but it only appears on the last commit when the
option to enable partitioning is added.
/*
* ARM pmu always has to reprogram the period, so ignore
* PERF_EF_RELOAD, see the comment below.
*/
if (flags & PERF_EF_RELOAD)
WARN_ON_ONCE(!(hwc->state & PERF_HES_UPTODATE));
Steps to reproduce:
Host (arm_pmuv3.reserved_host_counters=0):
$ sudo perf stat -C 2 -e \
'branches,branches,branches,branches,branches,branches'
$ sudo taskset --cpu-list 2 ./lkvm run --kernel \
/boot/vmlinux-7.1.0-rc7+ -m 1024 -c 1 --pmu
Guest:
$ sleep 1
WARNING: drivers/perf/arm_pmu.c:302 at cpu_pm_pmu_notify+0x278/0x2c0,
CPU#2: swapper/2/0
Call trace:
cpu_pm_pmu_notify+0x278/0x2c0 (P)
notifier_call_chain+0x84/0x1d0
raw_notifier_call_chain+0x24/0x38
cpu_pm_exit+0x34/0x68
acpi_processor_ffh_lpi_enter+0x40/0x78
acpi_idle_lpi_enter+0x54/0x78
cpuidle_enter_state+0xb4/0x248
cpuidle_enter+0x44/0x68
do_idle+0x21c/0x300
cpu_startup_entry+0x40/0x50
secondary_start_kernel+0x120/0x150
__secondary_switched+0xc0/0xc8
When running the guest on a single CPU I get different counts for the
same event for a single process, although this never happens on a host.
I think there might even be some Perf tests which expect them to be the
same, and this doesn't depend on whether any events are running on the
host or not. Not sure if you ran all the Perf selftests in a guest or not?
I'm not sure if the exception level filtering isn't working or there is
something wrong with freezing. Doesn't the host PMU driver need to
freeze with HPME? I see it's still doing it with PMCR_EL0.E which
freezes the guest's counters now doesn't it?
Host (arm_pmuv3.reserved_host_counters=0):
$ sudo taskset --cpu-list 2 ./lkvm run --kernel \
/boot/vmlinux-7.1.0-rc7+ -m 1024 -c 1 --pmu
Guest:
$ perf stat -e branches,branches true
Performance counter stats for 'true':
167963 branches
160925 branches
$ perf stat -e branches,branches,branches,branches,branches true
Performance counter stats for 'true':
164425 branches
164425 branches
164425 branches
157743 branches
157743 branches
When running the guest on two CPUs I just get zeros. Do the kvm
selftests not catch this? Or is it something to do with my setup:
Host (arm_pmuv3.reserved_host_counters=0):
$ sudo taskset --cpu-list 2-3 ./lkvm run --kernel \
/boot/vmlinux-7.1.0-rc7+ -m 1024 -c 1 --pmu
Guest:
$ perf stat -e branches,branches true
Performance counter stats for 'true':
0 branches
0 branches
I also noticed I get the "squeezed" warning printed after launching the
guest but not using Perf. This comment implies that not using counters
makes it a nop:
/*
* If we aren't guest-owned then we know the guest isn't using
* the PMU anyway, so no need to bother with the swap.
*/
if (vcpu->arch.pmu.access != VCPU_PMU_ACCESS_GUEST_OWNED)
return;
But maybe linux is touching them in a way that makes them guest owned,
even on probe? Or is the guest/host ownership tracking not working, I
didn't look too hard.
Finally I think there are 3 critical Sashiko comments on this version.
If they're false positives, then maybe some comments in the code or
commit messages could reassure it.
Thanks
James
> v8:
>
> * Rebase on top of v7.1-rc7.
>
> * Implement Oliver Upton's accessor proposal to centralize PMU
> register access and simplify trap handlers. Instead of one singular
> accessor, implement as two because the read and write paths are
> always different anyway.
>
> * Introduce the partitioning flag along with the
> kvm_pmu_is_partitioned predicate
>
> * Don't use ifdef for partitioning predicates as that can be handled
> by has_vhe
>
> * Clean up MDCR_EL2 handling by open-coding use_fgt and hpmn and
> unconditionally setting RES0 bits.
>
> * Use {read,write}_pmcrcntrn in context swaps
>
> * Put operators on preceeding lines
>
> * Rename hw_cntr_mask to hw_cntr_impl to clarify it tracks the number
> of counters implemented by hardware
>
> * Use GENMASK_ULL in mask functions returning u64
>
> * warn_once when host events are squeezed out by guest counter
> allocations.
>
> * Address Sashiko AI Review findings:
>
> - Critical fixes for lazy PMU context swaps (ensuring guest state is
> loaded on transition to GUEST_OWNED), PMSELR_EL0 trapping to
> prevent stale selector index, and masking guest PMCR_EL0 writes to
> prevent host reset.
>
> - High priority fixes for lock safety (disabling IRQs when acquiring
> perf context lock), disabling guest counters on vCPU put,
> preserving VHE host profiling in MDCR_EL2, waking halted vCPUs on
> guest PMU interrupts, masking host configuration leaks, preemption
> safety in per-CPU accesses, emulating PMCR.N reads, and preventing
> data races in PMOVSSET_EL0 accesses.
>
> - Medium/Low fixes for user-access fallback safety, VM-wide state
> modification restrictions, selftests type safety, and cleanup of
> unused fields and typos.
>
> v7:
> https://lore.kernel.org/kvmarm/20260504211813.1804997-1-coltonlewis@google.com/
>
> v6:
> https://lore.kernel.org/kvmarm/20260209221414.2169465-1-coltonlewis@google.com/
>
> v5:
> https://lore.kernel.org/kvmarm/20251209205121.1871534-1-coltonlewis@google.com/
>
> v4:
> https://lore.kernel.org/kvmarm/20250714225917.1396543-1-coltonlewis@google.com/
>
> v3:
> https://lore.kernel.org/kvm/20250626200459.1153955-1-coltonlewis@google.com/
>
> v2:
> https://lore.kernel.org/kvm/20250620221326.1261128-1-coltonlewis@google.com/
>
> v1:
> https://lore.kernel.org/kvm/20250602192702.2125115-1-coltonlewis@google.com/
>
> [1] https://gitlab.com/qemu-project/kvm-forum/-/raw/main/_attachments/2025/Optimizing__itvHkhc.pdf
> [2] https://www.youtube.com/watch?v=YRzZ8jMIA6M&list=PLW3ep1uCIRfxwmllXTOA2txfDWN6vUOHp&index=9
>
> Colton Lewis (20):
> arm64: cpufeature: Add cpucap for HPMN0
> KVM: arm64: Reorganize PMU functions
> perf: arm_pmuv3: Generalize counter bitmasks
> perf: arm_pmuv3: Check cntr_mask before using pmccntr
> perf: arm_pmuv3: Allocate counter indices from high to low
> perf: arm_pmuv3: Add method to partition the PMU
> KVM: arm64: Set up FGT for Partitioned PMU
> KVM: arm64: Add Partitioned PMU register trap handlers
> KVM: arm64: Set up MDCR_EL2 to handle a Partitioned PMU
> KVM: arm64: Context swap Partitioned PMU guest registers
> KVM: arm64: Enforce PMU event filter at vcpu_load()
> perf: Add perf_pmu_resched_update()
> KVM: arm64: Apply dynamic guest counter reservations
> KVM: arm64: Implement lazy PMU context swaps
> perf: arm_pmuv3: Handle IRQs for Partitioned PMU guest counters
> KVM: arm64: Detect overflows for the Partitioned PMU
> KVM: arm64: Add vCPU device attr to partition the PMU
> KVM: selftests: Add find_bit to KVM library
> KVM: arm64: selftests: Add test case for Partitioned PMU
> KVM: arm64: selftests: Relax testing for exceptions when partitioned
>
> Marc Zyngier (1):
> KVM: arm64: Reorganize PMU includes
>
> arch/arm/include/asm/arm_pmuv3.h | 18 +
> arch/arm64/include/asm/arm_pmuv3.h | 12 +-
> arch/arm64/include/asm/kvm_host.h | 17 +-
> arch/arm64/include/asm/kvm_types.h | 6 +-
> arch/arm64/include/uapi/asm/kvm.h | 2 +
> arch/arm64/kernel/cpufeature.c | 10 +-
> arch/arm64/kvm/Makefile | 2 +-
> arch/arm64/kvm/arm.c | 2 +
> arch/arm64/kvm/config.c | 41 +-
> arch/arm64/kvm/debug.c | 30 +-
> arch/arm64/kvm/pmu-direct.c | 507 ++++++++++++
> arch/arm64/kvm/pmu-emul.c | 684 +----------------
> arch/arm64/kvm/pmu.c | 720 ++++++++++++++++++
> arch/arm64/kvm/sys_regs.c | 271 +++++--
> arch/arm64/tools/cpucaps | 1 +
> arch/arm64/tools/sysreg | 6 +-
> drivers/perf/arm_pmuv3.c | 136 +++-
> include/kvm/arm_pmu.h | 93 ++-
> include/linux/perf/arm_pmu.h | 8 +
> include/linux/perf/arm_pmuv3.h | 14 +-
> include/linux/perf_event.h | 3 +
> kernel/events/core.c | 31 +-
> tools/include/perf/arm_pmuv3.h | 12 +-
> tools/testing/selftests/kvm/Makefile.kvm | 1 +
> .../selftests/kvm/arm64/vpmu_counter_access.c | 112 ++-
> tools/testing/selftests/kvm/lib/find_bit.c | 2 +
> 26 files changed, 1918 insertions(+), 823 deletions(-)
> create mode 100644 arch/arm64/kvm/pmu-direct.c
> create mode 100644 tools/testing/selftests/kvm/lib/find_bit.c
>
>
> base-commit: 4549871118cf616eecdd2d939f78e3b9e1dddc48
^ permalink raw reply
* [PATCH] docs/core-api: memory-allocation: adopt type aware kmalloc_obj
From: Manuel Ebner @ 2026-07-20 16:29 UTC (permalink / raw)
To: Kees Cook, Jonathan Corbet, Shuah Khan
Cc: Linux MM, Manuel Ebner, linux-doc, linux-kernel
Update memory-allocation.rst to reflect new type-aware kmalloc-family
as suggested in commit 2932ba8d9c99 ("slab: Introduce kmalloc_obj()
and family").
Add 'prt = ' to example because allocating without having the pointer
is nonsensical.
Replace *alloc() with *alloc_obj() or *alloc_objs().
Signed-off-by: Manuel Ebner <manuelebnerli@mailbox.org>
---
I have no deep technical knowledge of memory allocation. I tried to update
memmory-allocation.rst with the help of deprecated.rst and some research.
Therefore @Kees Cock: Can you review my patch?
I couldn't find any clue in the kernel doc of the deprecated k[mzc]alloc
functions of their deprecation status. Is this deliberate?
---
Documentation/core-api/memory-allocation.rst | 21 ++++++++++----------
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst
index 0f19dd524323..d3d8b39c95eb 100644
--- a/Documentation/core-api/memory-allocation.rst
+++ b/Documentation/core-api/memory-allocation.rst
@@ -21,10 +21,11 @@ answer, although very likely you should use
::
- kzalloc(<size>, GFP_KERNEL);
+ ptr = kzalloc_obj(*ptr, GFP_KERNEL);
-Of course there are cases when other allocation APIs and different GFP
-flags must be used.
+The ``GFP_KERNEL`` flag can be omitted, because GFP_KERNEL is the
+default value. It is left here deliberately. Of course there are
+cases when other allocation APIs and different GFP flags must be used.
Get Free Page flags
===================
@@ -133,24 +134,24 @@ Selecting memory allocator
==========================
The most straightforward way to allocate memory is to use a function
-from the kmalloc() family. And, to be on the safe side it's best to use
-routines that set memory to zero, like kzalloc(). If you need to
-allocate memory for an array, there are kmalloc_array() and kcalloc()
+from the kmalloc_obj() family. And, to be on the safe side it's best to use
+routines that set memory to zero, like kzalloc_obj(). If you need to
+allocate memory for an array, there are kmalloc_objs() and kzalloc_objs()
helpers. The helpers struct_size(), array_size() and array3_size() can
be used to safely calculate object sizes without overflowing.
-The maximal size of a chunk that can be allocated with `kmalloc` is
+The maximal size of a chunk that can be allocated with `kmalloc_obj` is
limited. The actual limit depends on the hardware and the kernel
-configuration, but it is a good practice to use `kmalloc` for objects
+configuration, but it is a good practice to use `kmalloc_obj` for objects
smaller than page size.
-The address of a chunk allocated with `kmalloc` is aligned to at least
+The address of a chunk allocated with `kmalloc_obj` is aligned to at least
ARCH_KMALLOC_MINALIGN bytes. For sizes which are a power of two, the
alignment is also guaranteed to be at least the respective size. For other
sizes, the alignment is guaranteed to be at least the largest power-of-two
divisor of the size.
-Chunks allocated with kmalloc() can be resized with krealloc(). Similarly
+Chunks allocated with kmalloc_obj() can be resized with krealloc(). Similarly
to kmalloc_array(): a helper for resizing arrays is provided in the form of
krealloc_array().
--
2.54.0
^ permalink raw reply related
* Re: [PATCH net-next v5 00/15][pull request] Introduce iXD driver
From: Larysa Zaremba @ 2026-07-20 16:24 UTC (permalink / raw)
To: Tony Nguyen
Cc: davem, kuba, pabeni, edumazet, andrew+netdev, netdev,
przemyslaw.kitszel, aleksander.lobakin, sridhar.samudrala,
michal.swiatkowski, maciej.fijalkowski, emil.s.tantilov,
madhu.chittim, joshua.a.hay, jacob.e.keller,
jayaprakash.shanmugam, jiri, horms, corbet, richardcochran, skhan,
linux-doc
In-Reply-To: <20260715180042.1972010-1-anthony.l.nguyen@intel.com>
I have looked though Sashiko feedback for this version. A lot medium+ comments
were things that I have addressed as false-positives in v4, so those are left
out, same as many of low-severity subjective issues. Unfortunately, patch 4 does
need a fixup (see Re: for the patch itself), but hopefully this does not warrant
another PR.
^ permalink raw reply
* Re: [PATCH net-next v5 04/15] libie: add control queue support
From: Larysa Zaremba @ 2026-07-20 16:17 UTC (permalink / raw)
To: Tony Nguyen
Cc: davem, kuba, pabeni, edumazet, andrew+netdev, netdev,
Phani R Burra, przemyslaw.kitszel, aleksander.lobakin,
sridhar.samudrala, michal.swiatkowski, maciej.fijalkowski,
emil.s.tantilov, madhu.chittim, joshua.a.hay, jacob.e.keller,
jayaprakash.shanmugam, jiri, horms, corbet, richardcochran, skhan,
linux-doc, Samuel Salin, Bharath R
In-Reply-To: <20260715180042.1972010-5-anthony.l.nguyen@intel.com>
Sashiko has found one pretty valid issue and one low-hanging-fruit in this
patch. Just in case, here is diff that addresses those:
Author: Larysa Zaremba <larysa.zaremba@intel.com>
Date: Mon Jul 20 18:00:25 2026 +0200
fixup! libie: add control queue support
diff --git a/drivers/net/ethernet/intel/libie/controlq.c b/drivers/net/ethernet/intel/libie/controlq.c
index 0a179957bfef..1eee507dd484 100644
--- a/drivers/net/ethernet/intel/libie/controlq.c
+++ b/drivers/net/ethernet/intel/libie/controlq.c
@@ -127,6 +127,7 @@ int libie_ctlq_post_rx_buffs(struct libie_ctlq_info *ctlq)
if (likely(ctlq->next_to_post != ntp)) {
ctlq->next_to_post = ntp;
+ dma_wmb();
writel(ntp, ctlq->reg.tail);
}
@@ -331,11 +332,6 @@ libie_ctlq_add(struct libie_ctlq_ctx *ctx,
qinfo->len > FIELD_MAX(LIBIE_CTLQ_MBX_ATQ_LEN))
return ERR_PTR(-EOPNOTSUPP);
- /* libie_ctlq_init was not called */
- scoped_guard(spinlock, &ctx->ctlqs_lock)
- if (!ctx->ctlqs.next)
- return ERR_PTR(-EINVAL);
-
ctlq = kvzalloc_obj(*ctlq);
if (!ctlq)
return ERR_PTR(-ENOMEM);
^ permalink raw reply related
* Re: [PATCH net-next v5 14/15] ixd: add the core initialization
From: Larysa Zaremba @ 2026-07-20 16:14 UTC (permalink / raw)
To: Tony Nguyen
Cc: davem, kuba, pabeni, edumazet, andrew+netdev, netdev,
przemyslaw.kitszel, aleksander.lobakin, sridhar.samudrala,
michal.swiatkowski, maciej.fijalkowski, emil.s.tantilov,
madhu.chittim, joshua.a.hay, jacob.e.keller,
jayaprakash.shanmugam, jiri, horms, corbet, richardcochran, skhan,
linux-doc, Bharath R
In-Reply-To: <20260715180042.1972010-15-anthony.l.nguyen@intel.com>
Sashiko says:
> --- Patch [14]: [PATCH net-next v5 14/15] ixd: add the core initialization ---
> commit 43763d63c75ee5bcdd9189c20cf8b272997bdfec
> Author: Larysa Zaremba <larysa.zaremba@intel.com>
>
> ixd: add the core initialization
>
> This patch initializes the core when the mailbox is set up. The driver
> confirms the virtchnl version, requests necessary capabilities, and sets up
> the message handling framework.
>
> > diff --git a/drivers/net/ethernet/intel/ixd/ixd_ctlq.c b/drivers/net/ethernet/intel/ixd/ixd_ctlq.c
> > index 0000000000000..6d658be0695d4
> > --- /dev/null
> > +++ b/drivers/net/ethernet/intel/ixd/ixd_ctlq.c
>
> [ ... ]
>
> > @@ -0,0 +1,146 @@
>
> [ ... ]
>
> > +void ixd_ctlq_rx_task(struct work_struct *work)
> > +{
> > + struct ixd_adapter *adapter;
> > +
> > + adapter = container_of(work, struct ixd_adapter, mbx_task.work);
> > +
> > + queue_delayed_work(system_unbound_wq, &adapter->mbx_task,
> > + msecs_to_jiffies(300));
> > +
> > + ixd_ctlq_recv_mb_msg(adapter);
> > +}
>
> Does queueing the delayed work before executing the main logic create a race
> condition? If cancel_delayed_work_sync() is called (for example, during
> teardown in ixd_deinit_dflt_mbx()), try_to_grab_pending() could successfully
> delete the newly queued timer and return immediately, assuming the work is
> fully cancelled.
>
> This would bypass flush_work() and allow the teardown path to free
> adapter->xnm and set adapter->arq to NULL while ixd_ctlq_recv_mb_msg() is
> still running, potentially causing a use-after-free or NULL pointer
> dereference. Should the re-arming be moved to the end of the task?
>
I assume the process of cancelling delayed work is more robust.
> > diff --git a/drivers/net/ethernet/intel/ixd/ixd_lib.c b/drivers/net/ethernet/intel/ixd/ixd_lib.c
> > index afc413d3650fb..ce870aba143d4 100644
> > --- a/drivers/net/ethernet/intel/ixd/ixd_lib.c
> > +++ b/drivers/net/ethernet/intel/ixd/ixd_lib.c
>
> [ ... ]
>
> > @@ -136,8 +145,27 @@ void ixd_init_task(struct work_struct *work)
>
> [ ... ]
>
> > + libie_ctlq_xn_shutdown(adapter->xnm);/
> > + ixd_trigger_reset(adapter);
> > + ixd_deinit_dflt_mbx(adapter);
>
> Is it safe to trigger a hardware reset before cancelling the mailbox polling
> task? ixd_deinit_dflt_mbx() is responsible for cancelling the mbx_task, but
> calling ixd_trigger_reset() first means the polling task could run while the
> hardware is resetting.
>
> During reset, MMIO reads or DMA ring memory might temporarily contain invalid
> states, and the still-running mbx_task might process corrupted descriptors.
> Should the polling task be cancelled before asserting the hardware reset here,
> as well as in ixd_remove() in ixd_main.c?
>
Yes, this is safe, we just won't receive anything at this point.
^ permalink raw reply
* Re: [PATCH net-next v5 09/15] idpf: refactor idpf to use libie control queues
From: Larysa Zaremba @ 2026-07-20 16:11 UTC (permalink / raw)
To: Tony Nguyen
Cc: davem, kuba, pabeni, edumazet, andrew+netdev, netdev,
Pavan Kumar Linga, przemyslaw.kitszel, aleksander.lobakin,
sridhar.samudrala, michal.swiatkowski, maciej.fijalkowski,
emil.s.tantilov, joshua.a.hay, jacob.e.keller,
jayaprakash.shanmugam, jiri, horms, corbet, richardcochran, skhan,
linux-doc, Aleksandr Loktionov, Samuel Salin
In-Reply-To: <20260715180042.1972010-10-anthony.l.nguyen@intel.com>
Sashiko says:
> > +void idpf_send_vf_reset_msg(struct idpf_adapter *adapter)
> > +{
> > + struct libie_ctlq_info *ctlq = adapter->asq;
> > +
> > + /* Forcefully claim send queue slot */
> > + idpf_mb_clean(adapter, ctlq, true);
> > +
> > + scoped_guard(spinlock, &ctlq->lock) {
> > + *ctlq->tx_msg[ctlq->next_to_use] = (struct libie_ctlq_msg) {
> > + .opcode = LIBIE_CTLQ_SEND_MSG_TO_CP,
> > + .chnl_opcode = VIRTCHNL2_OP_RESET_VF,
> > + };
> > +
> > + libie_ctlq_send(adapter->asq, 1);
> > + }
> > +}
> Compared to the old call, which routed through idpf_send_mb_msg() and
> first ran idpf_is_reset_detected() (via the arq registers) to bail out
> early if the mailbox was gone, this new helper unconditionally
> dereferences adapter->asq and immediately calls idpf_mb_clean() which
> takes &ctlq->lock.
> Can adapter->asq be NULL here?
> idpf_deinit_dflt_mbx() sets asq = NULL after libie_ctlq_xn_deinit(),
> and idpf_init_dflt_mbx() can fail with asq/arq still NULL if
> libie_find_ctlq() returns NULL. If a hard reset flow is entered on
> such an error path, ctlq->lock would fault.
> Also, is bypassing the xn manager and reaching directly into libie
> internals (ctlq->tx_msg[ctlq->next_to_use], ctlq->lock, and
> libie_ctlq_send()) intentional? This is the only send path in the
> refactored driver that pokes libie state directly, and it is fragile
> against future changes to how tx_msg[] slots are allocated or
> lifecycled after a forced idpf_mb_clean().
> Finally, is it safe to writel() to the mailbox tail via
> libie_ctlq_send() when hardware is already in reset? The old
> idpf_is_reset_detected() gate specifically avoided that.
VF reset is done only in one flow, where it is performed only
if no other reset is going on.
[...]
>
> > +int idpf_send_mb_msg_kfree(struct idpf_adapter *adapter,
> > + struct libie_ctlq_xn_send_params *xn_params,
> > + void *send_buf, size_t send_buf_size)
> > +{
> > + int err = idpf_send_mb_msg(adapter, xn_params, send_buf, send_buf_size);
> > +
> > + if (libie_cp_can_send_onstack(send_buf_size))
> > + kfree(send_buf);
> > +
> > + return err;
> > +}
> Is the stack-vs-heap discrimination here safe as a long-term invariant?
> The reset-detected branch decides to kfree(send_buf) based purely on
> libie_cp_can_send_onstack(send_buf_size), and rel_tx_buf is set to
> kfree unconditionally so libie_ctlq_xn_send() will also invoke kfree
> on the send buffer when the send fails and the size crosses
> LIBIE_CP_TX_COPYBREAK (128 bytes).
> Many call sites pass the address of a stack-allocated struct:
> struct virtchnl2_get_capabilities caps = {};
> ... idpf_send_mb_msg(adapter, &xn_params, &caps, sizeof(caps));
> struct virtchnl2_vport_stats stats_msg = {};
> ... idpf_send_mb_msg(np->adapter, &xn_params, &stats_msg,
> sizeof(stats_msg));
> virtchnl2_vport_stats is exactly 128 bytes today, so it sits right at
> the LIBIE_CP_TX_COPYBREAK boundary. Adding one field to it, or any
> similar stack-passed structure growing past 128 bytes, would silently
> cause kfree() to be called on a stack address in the reset-detected
> or send-failure path, corrupting the slab allocator.
> Would a BUILD_BUG_ON at each stack-based call site, or a typed helper
> that only accepts on-stack buffers up to LIBIE_CP_TX_COPYBREAK, be
> worth adding so this constraint is enforced at compile time rather
> than by convention?
I would very much prefer to go the callback route right away, like in ixd,
but the diff would be even more unmanageable then.
[...]
>
> > @@ -1586,33 +1168,29 @@ int idpf_send_create_vport_msg(struct idpf_adapter *adapter,
> > }
> >
> > if (!adapter->vport_params_recvd[idx]) {
> > - adapter->vport_params_recvd[idx] = kzalloc(IDPF_CTLQ_MAX_BUF_LEN,
> > - GFP_KERNEL);
> > + adapter->vport_params_recvd[idx] =
> > + kzalloc(LIBIE_CTLQ_MAX_BUF_LEN, GFP_KERNEL);
> > if (!adapter->vport_params_recvd[idx]) {
> > err = -ENOMEM;
> > goto rel_buf;
> > }
> > }
> >
> > - xn_params.vc_op = VIRTCHNL2_OP_CREATE_VPORT;
> > - xn_params.send_buf.iov_base = vport_msg;
> > - xn_params.send_buf.iov_len = buf_size;
> > - xn_params.recv_buf.iov_base = adapter->vport_params_recvd[idx];
> > - xn_params.recv_buf.iov_len = IDPF_CTLQ_MAX_BUF_LEN;
> > - xn_params.timeout_ms = IDPF_VC_XN_DEFAULT_TIMEOUT_MSEC;
> > - reply_sz = idpf_vc_xn_exec(adapter, &xn_params);
> > - if (reply_sz < 0) {
> > - err = reply_sz;
> > - goto free_vport_params;
> > + err = idpf_send_mb_msg_kfree(adapter, &xn_params, vport_msg,
> > + sizeof(*vport_msg));
> > + if (err) {
> > + kfree(adapter->vport_params_recvd[idx]);
> > + adapter->vport_params_recvd[idx] = NULL;
> > + return err;
> > }
> >
> > - kfree(vport_msg);
> > + memcpy(adapter->vport_params_recvd[idx], xn_params.recv_mem.iov_base,
> > + xn_params.recv_mem.iov_len);
> Is the memcpy() length here guaranteed to be bounded?
> adapter->vport_params_recvd[idx] is a fixed LIBIE_CTLQ_MAX_BUF_LEN
> allocation, but the copy length is taken directly from
> xn_params.recv_mem.iov_len without a defensive cap. This function
libie_cp does cap the iov_len at LIBIE_CTLQ_MAX_BUF_LEN in
libie_ctlq_fill_rx_msg(), so this is fine.
> relies on libie's Rx path always delivering iov_len <=
> LIBIE_CTLQ_MAX_BUF_LEN. If that invariant ever changes (larger RX
> buffers, multi-descriptor delivery, or a missing bounds check against
> the descriptor's data_len in libie), this becomes an unchecked heap
> overflow driven by CP firmware.
> Other call sites in the same patch use bounded copies, for example
> idpf_send_get_caps_msg():
> memcpy(&adapter->caps, xn_params.recv_mem.iov_base,
> sizeof(adapter->caps));
> Would using min_t(size_t, LIBIE_CTLQ_MAX_BUF_LEN,
> xn_params.recv_mem.iov_len) match the pattern used elsewhere in this
> file?
>
^ permalink raw reply
* Re: [PATCH v4 3/3] watchdog: npcm: add bootstatus support
From: Guenter Roeck @ 2026-07-20 16:11 UTC (permalink / raw)
To: Tomer Maimon
Cc: andrew, wim, robh, krzk+dt, conor+dt, openbmc, linux-watchdog,
linux-doc, devicetree, linux-kernel, avifishman70, tali.perry1,
venture, yuenn, benjaminfair, corbet, skhan, joel
In-Reply-To: <20260706144828.3517631-4-tmaimon77@gmail.com>
On Mon, Jul 06, 2026 at 05:48:28PM +0300, Tomer Maimon wrote:
> The NPCM750 uses RESSR and the NPCM845 uses INTCR2 to latch reset
> indications. Read those bits during probe and map them into watchdog
> bootstatus flags.
>
> For NPCM845, cache the sampled INTCR2 state in SCRPAD10 after the reset
> status bits are cleared so later probes can report the same boot-time
> state. Also report WDIOF_CARDRESET for the watchdog instance whose reset
> bit is latched, while leaving WPCM450 behavior unchanged.
>
> Signed-off-by: Tomer Maimon <tmaimon77@gmail.com>
Applied.
Thanks,
Guenter
^ permalink raw reply
* Re: [PATCH v4 2/3] docs: watchdog: npcm: Add reset status description
From: Guenter Roeck @ 2026-07-20 16:10 UTC (permalink / raw)
To: Tomer Maimon
Cc: andrew, wim, robh, krzk+dt, conor+dt, openbmc, linux-watchdog,
linux-doc, devicetree, linux-kernel, avifishman70, tali.perry1,
venture, yuenn, benjaminfair, corbet, skhan, joel
In-Reply-To: <20260706144828.3517631-3-tmaimon77@gmail.com>
On Mon, Jul 06, 2026 at 05:48:27PM +0300, Tomer Maimon wrote:
> Add documentation describing how the NPCM watchdog driver reports reset
> causes through bootstatus on NPCM750 and NPCM845 systems.
>
> Document the reset flag mapping, the watchdog instance mapping for
> WDIOF_CARDRESET, and the NPCM750/NPCM845 latch handling. Also mention
> sysfs bootstatus reporting when watchdog sysfs support is enabled.
>
> Signed-off-by: Tomer Maimon <tmaimon77@gmail.com>
Applied.
Thanks,
Guenter
^ permalink raw reply
* Re: [PATCH v4 1/3] dt-bindings: watchdog: npcm: add GCR syscon property
From: Guenter Roeck @ 2026-07-20 16:09 UTC (permalink / raw)
To: Tomer Maimon
Cc: andrew, wim, robh, krzk+dt, conor+dt, openbmc, linux-watchdog,
linux-doc, devicetree, linux-kernel, avifishman70, tali.perry1,
venture, yuenn, benjaminfair, corbet, skhan, joel
In-Reply-To: <20260706144828.3517631-2-tmaimon77@gmail.com>
On Mon, Jul 06, 2026 at 05:48:26PM +0300, Tomer Maimon wrote:
> NPCM750 and NPCM845 latch watchdog reset indications in the SoC
> GCR block rather than in the watchdog block itself.
>
> Add the optional nuvoton,sysgcr phandle so watchdog nodes can
> reference the shared GCR reset-status registers that hold those
> latched watchdog reset indications.
>
> This is needed by the following reset-status support, which reads
> those latches and reports watchdog-caused resets through bootstatus.
>
> Signed-off-by: Tomer Maimon <tmaimon77@gmail.com>
> Acked-by: Conor Dooley <conor.dooley@microchip.com>
Applied.
Thanks,
Guenter
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox