Re: [PATCH v3 00/12] [PATCH v3 00/12] x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 6/12/26 8:37 AM, Moger, Babu wrote:
> 
> 
> On 6/11/2026 4:53 PM, Reinette Chatre wrote:
>> Hi Babu,
>>
>> On 4/30/26 4:24 PM, Babu Moger wrote:
>>> Design
>>> ======
>>>
>>> A new sysfs file, info/kernel_mode, holds a single global policy that
>>> selects what kernel work is steered and which rdtgroup it is steered
>>
>> How should "selects *what* kernel work is steered" be interpreted? Do these
>> modes not all apply to *all* kernel work?
> 
> How about?
> 
> A new sysfs file, info/kernel_mode, holds a single global policy for
> kernel contexts and the rdtgroup associated with the policy.

What does "kernel contexts" mean here? 
Also, since rdtgroup refers more to "struct rdtgroup" that is internal to resctrl
I would suggest "resource group" be used instead. 
Consider, for example (this is just something to get started, please do not just
copy&paste):

   A new resctrl file, info/kernel_mode, holds the global policy for
   resource allocation and monitoring of kernel work and the resource group
   (when applicable) associated with the policy.


>>> to.  Reads describe the supported modes and the currently-active
>>> binding; writes change the policy or rebind to a different group.
>>> Look at the thread below for design discussion.
>>> https://lore.kernel.org/lkml/14a8ad0a-e842-4268-871a-0762f1169e03@intel.com/
>>>
>>
>> ...
>>
>>> Examples
>>> ========
>>>
>>> (See Documentation/filesystems/resctrl.rst, "kernel_mode" and
>>> "kmode_cpus" sections, for the full UAPI.)
>>>
>>>    # Mount resctrl
>>>    # mount -t resctrl resctrl /sys/fs/resctrl
>>>    # cd /sys/fs/resctrl
>>>
>>>    # Read the supported modes.  The active mode is bracketed and reports
>>>    # the bound "<ctrl>/<mon>/" group; other supported modes report
>>>    # ":group=none" because nothing is bound to them.
>>>    # cat info/kernel_mode
>>>    [inherit_ctrl_and_mon:group=//]
>>
>> This is unexpected since associating a group to this mode implies that this
>> group is used to manage allocations and monitoring of kernel work but this
>> is not true, right? From what I understand there should be no group associated with
>> this default "inherit_ctrl_and_mon" mode.
> 
> The default mode is "inherit_ctrl_and_mon", where both user mode and kernel mode share the same CLOSID and RMID. This is current mode (without this series).
> 
> I thought we are going to set the default mode with the default group when system boots up. No?

As I have it, the end of our discussion on this topic is at:
https://lore.kernel.org/lkml/6709398b-269d-47b5-9b41-084f410bb1a6@amd.com/

Based on that discussion there is no resource group associated with the default
inherit_ctrl_and_mon.
I find the above output confusing to user space since adding the default group as the only
group to this mode implies that kernel work inherits resource allocation and monitoring
from the default group but that is not correct.

Your answer seems to refer to other discussions about what group should be used for
a mode when switching to a new mode and user space has not set the resource group. If not,
could you please point me to which discussion you are referring to?

> 
> 
>>
>>>    global_assign_ctrl_inherit_mon_per_cpu:group=none
>>>    global_assign_ctrl_assign_mon_per_cpu:group=none
>>
>> nit: "none" does not reflect state as clearly as "unset"/"uninitialized"/"NA"
> 
> Lets go with "uninitialized".

ok

...
>>>    # echo 0-3 > ctrl1/kmode_cpus_list
>>>    # cat ctrl1/kmode_cpus
>>>    f
>>>    # cat ctrl1/kmode_cpus_list
>>>    0-3
>>>
>>>    # Empty masks are rejected; use info/kernel_mode to reset to
>>>    # "every online CPU".
>>>    # echo "" > ctrl1/kmode_cpus_list
>>>    bash: echo: write error: Invalid argument
>>>    # cat info/last_cmd_status
>>>    Empty mask not allowed; use info/kernel_mode to unbind
>>
>> Why are empty masks rejected/not allowed?
> 
> No specific reason.
> 
> When the mode is switched, we discussed earlier to globally apply the mode to all the online CPUs.

Right. I did not see this being done in this implementation though. As I mentioned in
my response to patch #8 it appears that it uses old data from the resource
group's kmode_cpu_mask. I do think that applying it to all the online CPUs
matches the intention and would make the code much simpler.

> 
> At this point reading "kmode_cpus_list" will still report empty.

I do not think resctrl should do this. This is not accurate and conflicts with the
existing cpus resctrl files. It should be simple to just present the actual and
accurate data to user space, especially after incorporating Qinyun Tan's contributions.

> 
> Users can change it to selectively apply the mode by writing to "kmode_cpus_list".
> 
> I was not sure what was the action when empty masks are written.
> 
> Should the empty mask apply the mode to all the online CPUs?

Users are used to being able to use an empty write to remove all CPUs from a
resource group. It thus seems intuitive that an empty write to the kmode_cpus file
behave similarly. Sounds like this could mean that if user space sets the
kmode to global_assign_ctrl_inherit_mon_per_cpu or global_assign_ctrl_assign_mon_per_cpu
and then writes an empty mask to kmode_cpus then it would essentially be setting
inherit_ctrl_and_mon mode? This still seems ok since if disabling one of the "global"
modes on a CPU results in that CPU inheriting from PQR_ASSOC then it seems
reasonable to extend to when that mode is disabled for all CPUs.

>>>    # Disable kernel-mode steering (back to inherit, default group).
>>
>> This sounds like kernel work is steered to default group which I
>> do not think is accurate for the "inherit_ctrl_and_mon" mode.
> 
> How about ?
> 
> Drop the kernel-mode binding and restore inherit_ctrl_and_mon on the default group.

No. There is no "inherit_ctrl_and_mon on the default group". There is nothing special
about the default group when it comes to inherit_ctrl_and_mon mode ... or am I missing
something?
This could be something like: "Activate inherit_ctrl_and_mon mode to let
kernel work inherit the resource allocation and monitoring from the user space task."
(and drop the default group from the output associated with inherit_ctrl_and_mon)

Reinette