Re: [RESEND PATCH v4 11/15] fs/resctrl: Program kernel-mode binding when CPU comes online

[Babu Moger <babu.moger@amd.com>]

On 7/7/26 16:50, Babu Moger wrote:
> Kernel-mode resctrl associations are programmed per CPU. When a
> global-assign kernel-mode policy is selected, all CPUs that are online at
> that time are configured; however, CPUs that come online later are not
> automatically updated.
> 
> As a result, hot-added CPUs, or CPUs that were offline when the policy was
> selected, can run without the active kernel-mode association even though
> the resctrl policy is still in effect.
> 
> Add resctrl_kmode_online_cpu() and call it from resctrl_online_cpu(). When
> a global-assign policy is active, add the newly online CPU to the bound
> group's kmode_cpu_mask and program the kernel-mode association on that CPU.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> v4: New patch in the series. Patch taken from:
>      https://lore.kernel.org/lkml/20260611111706.1981788-5-qinyuntan@linux.alibaba.com/
>      Updated the code to enable kernel-mode by default whenever a CPU comes online when
>      one of global-assign mode is enabled.
> ---
>   fs/resctrl/rdtgroup.c | 29 +++++++++++++++++++++++++++++
>   1 file changed, 29 insertions(+)
> 
> diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
> index 21659fd75850..413d3ff14546 100644
> --- a/fs/resctrl/rdtgroup.c
> +++ b/fs/resctrl/rdtgroup.c
> @@ -4722,11 +4722,40 @@ int resctrl_online_mon_domain(struct rdt_resource *r, struct rdt_domain_hdr *hdr
>   	return err;
>   }
>   
> +/*
> + * resctrl_kmode_online_cpu() - Configure kernel-mode binding for a CPU that
> + *				comes online
> + * @cpu: CPU that has just been brought online.
> + *
> + * Kernel-mode associations are maintained per CPU. When a CPU becomes
> + * online after a global-assign policy has been applied, it is updated
> + * with the current kernel-mode configuration. The CPU is then added to
> + * the bound group's kmode_cpu_mask, and the appropriate association is
> + * programmed for it.
> + */
> +static void resctrl_kmode_online_cpu(unsigned int cpu)
> +{
> +	struct rdtgroup *rdtgrp = resctrl_kcfg.k_rdtgrp;
> +	bool assign_mon = false;
> +
> +	if (resctrl_kcfg.kmode_cur == INHERIT_CTRL_AND_MON || !rdtgrp)
> +		return;
> +
> +	assign_mon = (resctrl_kcfg.kmode_cur == GLOBAL_ASSIGN_CTRL_ASSIGN_MON_PER_CPU);
> +
> +	cpumask_set_cpu(cpu, &rdtgrp->kmode_cpu_mask);
> +
> +	resctrl_arch_configure_kmode(cpumask_of(cpu), rdtgrp->closid, rdtgrp->mon.rmid,
> +				     assign_mon, true);
> +}
> +
>   void resctrl_online_cpu(unsigned int cpu)
>   {
>   	mutex_lock(&rdtgroup_mutex);
>   	/* The CPU is set in default rdtgroup after online. */
>   	cpumask_set_cpu(cpu, &rdtgroup_default.cpu_mask);
> +	/* Program any active kernel-mode binding on this CPU. */
> +	resctrl_kmode_online_cpu(cpu);
>   	mutex_unlock(&rdtgroup_mutex);
>   }
>   

Sashiko reported:

https://sashiko.dev/#/patchset/cover.1783461016.git.babu.moger%40amd.com?part=11

--- Patch [11]: [RESEND PATCH v4 11/15] fs/resctrl: Program kernel-mode 
binding when CPU comes online ---
commit ab44f1490e3dbe9a24a0baa6c7b98b7456c62c03
Author: Babu Moger <babu.moger@amd.com>
fs/resctrl: Program kernel-mode binding when CPU comes online
Kernel-mode resctrl associations are programmed per CPU. When a 
global-assign
kernel-mode policy is selected, all CPUs that are online at that time are
configured; however, CPUs that come online later are not automatically
updated.
Link: 
https://lore.kernel.org/lkml/20260611111706.1981788-5-qinyuntan@linux.alibaba.com/
diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c
index 21659fd75850e..413d3ff14546b 100644
--- a/fs/resctrl/rdtgroup.c
+++ b/fs/resctrl/rdtgroup.c
[ ... ]
 > @@ -4722,11 +4722,40 @@ int resctrl_online_mon_domain(struct 
rdt_resource *r, struct rdt_domain_hdr *hdr
 >  	return err;
 >  }
 >
 > +/*
 > + * resctrl_kmode_online_cpu() - Configure kernel-mode binding for a 
CPU that
 > + *				comes online
 > + * @cpu: CPU that has just been brought online.
 > + *
 > + * Kernel-mode associations are maintained per CPU. When a CPU becomes
 > + * online after a global-assign policy has been applied, it is updated
 > + * with the current kernel-mode configuration. The CPU is then added to
 > + * the bound group's kmode_cpu_mask, and the appropriate association is
 > + * programmed for it.
 > + */
 > +static void resctrl_kmode_online_cpu(unsigned int cpu)
 > +{
 > +	struct rdtgroup *rdtgrp = resctrl_kcfg.k_rdtgrp;
 > +	bool assign_mon = false;
 > +
 > +	if (resctrl_kcfg.kmode_cur == INHERIT_CTRL_AND_MON || !rdtgrp)
 > +		return;

If a CPU goes offline while a kernel-mode policy is active, and the 
policy is subsequently disabled (reverting to INHERIT_CTRL_AND_MON), 
does returning early here leave the newly onlined CPU with a stale 
kernel-mode resctrl association?

Looking at clear_closid_rmid(), it does not reset 
MSR_IA32_PQR_PLZA_ASSOC during CPU online initialization. Could this 
lead to the CPU executing CPL0 traffic with an invalid or malicious 
cache allocation if the stale CLOSID/RMID is reallocated?

--------------------------------------------------------------------

Yes. This is an issue.
Need to clear the kmode_cpu_list if the cpu being offlined is part of
the current kmode. Also, need to disable PLZA on the CPU that is offlined.

Will take care of this in resctrl_offline_cpu().

-------------------------------------------------------------------
 > +
 > +	assign_mon = (resctrl_kcfg.kmode_cur == 
GLOBAL_ASSIGN_CTRL_ASSIGN_MON_PER_CPU);
 > +
 > +	cpumask_set_cpu(cpu, &rdtgrp->kmode_cpu_mask);

Since the CPU is added to kmode_cpu_mask here when it comes online, 
shouldn't it be removed when it goes offline?
Looking at resctrl_offline_cpu(), it clears the CPU from rdtgrp->cpu_mask:
	if (cpumask_test_and_clear_cpu(cpu, &rdtgrp->cpu_mask)) {
		clear_childcpus(rdtgrp, cpu);
		break;
	}
But there is no corresponding logic to clear it from kmode_cpu_mask. 
Does this cause stale state to be exposed to userspace via the 
kmode_cpus file?

----------------------------------------------------------------------

Ditto: Will take care of this in resctrl_offline_cpu().

----------------------------------------------------------------------

Is it possible for a MON group that is actively bound to a kernel-mode 
policy to be reparented?

Looking at rdtgroup_rename(), it allows reparenting if cpu_mask is empty
without checking kmode_cpu_mask:
	if (!cpumask_empty(&rdtgrp->cpu_mask)) {
		ret = -EINVAL;
		goto out;
	}
Then mongrp_reparent() updates the closid but fails to reprogram PLZA 
MSRs for CPUs in kmode_cpu_mask:
	rdtgrp->mon.parent = new_prdtgrp;
	rdtgrp->closid = new_prdtgrp->closid;
	/* Propagate updated closid to all tasks in this group. */
	rdt_move_group_tasks(rdtgrp, rdtgrp, cpus);
	update_closid_rmid(cpus, NULL);

Does this leave CPUs in kmode_cpu_mask running with the old parent's 
CLOSID?

-----------------------------------------------------------------------

I don't see a reason to reparent the active kernel-mode group.

I'll add a check for:

if (!cpumask_empty(&rdtgrp->kmode_cpu_mask))

Thanks
Babu