Re: [PATCH v3 00/12] [PATCH v3 00/12] x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Babu Moger <babu.moger@amd.com>]

Hi Reinette,

On 6/16/26 23:34, Reinette Chatre wrote:
> Hi Babu,
> 
> On 6/12/26 8:37 AM, Moger, Babu wrote:
>>
>>
>> On 6/11/2026 4:53 PM, Reinette Chatre wrote:
>>> Hi Babu,
>>>
>>> On 4/30/26 4:24 PM, Babu Moger wrote:
>>>> Design
>>>> ======
>>>>
>>>> A new sysfs file, info/kernel_mode, holds a single global policy that
>>>> selects what kernel work is steered and which rdtgroup it is steered
>>>
>>> How should "selects *what* kernel work is steered" be interpreted? Do these
>>> modes not all apply to *all* kernel work?
>>
>> How about?
>>
>> A new sysfs file, info/kernel_mode, holds a single global policy for
>> kernel contexts and the rdtgroup associated with the policy.
> 
> What does "kernel contexts" mean here?
> Also, since rdtgroup refers more to "struct rdtgroup" that is internal to resctrl
> I would suggest "resource group" be used instead.
> Consider, for example (this is just something to get started, please do not just
> copy&paste):
> 
>     A new resctrl file, info/kernel_mode, holds the global policy for
>     resource allocation and monitoring of kernel work and the resource group
>     (when applicable) associated with the policy.

Sounds good.

> 
> 
>>>> to.  Reads describe the supported modes and the currently-active
>>>> binding; writes change the policy or rebind to a different group.
>>>> Look at the thread below for design discussion.
>>>> https://lore.kernel.org/lkml/14a8ad0a-e842-4268-871a-0762f1169e03@intel.com/
>>>>
>>>
>>> ...
>>>
>>>> Examples
>>>> ========
>>>>
>>>> (See Documentation/filesystems/resctrl.rst, "kernel_mode" and
>>>> "kmode_cpus" sections, for the full UAPI.)
>>>>
>>>>     # Mount resctrl
>>>>     # mount -t resctrl resctrl /sys/fs/resctrl
>>>>     # cd /sys/fs/resctrl
>>>>
>>>>     # Read the supported modes.  The active mode is bracketed and reports
>>>>     # the bound "<ctrl>/<mon>/" group; other supported modes report
>>>>     # ":group=none" because nothing is bound to them.
>>>>     # cat info/kernel_mode
>>>>     [inherit_ctrl_and_mon:group=//]
>>>
>>> This is unexpected since associating a group to this mode implies that this
>>> group is used to manage allocations and monitoring of kernel work but this
>>> is not true, right? From what I understand there should be no group associated with
>>> this default "inherit_ctrl_and_mon" mode.
>>
>> The default mode is "inherit_ctrl_and_mon", where both user mode and kernel mode share the same CLOSID and RMID. This is current mode (without this series).
>>
>> I thought we are going to set the default mode with the default group when system boots up. No?
> 
> As I have it, the end of our discussion on this topic is at:
> https://lore.kernel.org/lkml/6709398b-269d-47b5-9b41-084f410bb1a6@amd.com/
> 
> Based on that discussion there is no resource group associated with the default
> inherit_ctrl_and_mon.
> I find the above output confusing to user space since adding the default group as the only
> group to this mode implies that kernel work inherits resource allocation and monitoring
> from the default group but that is not correct.

Ah, I misunderstood earlier.

The display will look like this when the system boots up.

# cat info/kernel_mode
   inherit_ctrl_and_mon:
   global_assign_ctrl_assign_mon_per_cpu:group=uninitialized
   global_assign_ctrl_assign_mon_per_cpu:group=uninitialized

There will not be any group associated with "inherit_ctrl_and_mon".
It is only used to switch from other two modes.

> 
> Your answer seems to refer to other discussions about what group should be used for
> a mode when switching to a new mode and user space has not set the resource group. If not,
> could you please point me to which discussion you are referring to?
> 
>>
>>
>>>
>>>>     global_assign_ctrl_inherit_mon_per_cpu:group=none
>>>>     global_assign_ctrl_assign_mon_per_cpu:group=none
>>>
>>> nit: "none" does not reflect state as clearly as "unset"/"uninitialized"/"NA"
>>
>> Lets go with "uninitialized".
> 
> ok
> 
> ...
>>>>     # echo 0-3 > ctrl1/kmode_cpus_list
>>>>     # cat ctrl1/kmode_cpus
>>>>     f
>>>>     # cat ctrl1/kmode_cpus_list
>>>>     0-3
>>>>
>>>>     # Empty masks are rejected; use info/kernel_mode to reset to
>>>>     # "every online CPU".
>>>>     # echo "" > ctrl1/kmode_cpus_list
>>>>     bash: echo: write error: Invalid argument
>>>>     # cat info/last_cmd_status
>>>>     Empty mask not allowed; use info/kernel_mode to unbind
>>>
>>> Why are empty masks rejected/not allowed?
>>
>> No specific reason.
>>
>> When the mode is switched, we discussed earlier to globally apply the mode to all the online CPUs.
> 
> Right. I did not see this being done in this implementation though. As I mentioned in
> my response to patch #8 it appears that it uses old data from the resource
> group's kmode_cpu_mask. I do think that applying it to all the online CPUs
> matches the intention and would make the code much simpler.

Yes. Sure.

> 
>>
>> At this point reading "kmode_cpus_list" will still report empty.
> 
> I do not think resctrl should do this. This is not accurate and conflicts with the
> existing cpus resctrl files. It should be simple to just present the actual and
> accurate data to user space, especially after incorporating Qinyun Tan's contributions.

Sure. Will do

> 
>>
>> Users can change it to selectively apply the mode by writing to "kmode_cpus_list".
>>
>> I was not sure what was the action when empty masks are written.
>>
>> Should the empty mask apply the mode to all the online CPUs?
> 
> Users are used to being able to use an empty write to remove all CPUs from a
> resource group. It thus seems intuitive that an empty write to the kmode_cpus file

Ok, That seems reasonable. Lets keep the same behavior as we update 
other cpu_lists.

> behave similarly. Sounds like this could mean that if user space sets the
> kmode to global_assign_ctrl_inherit_mon_per_cpu or global_assign_ctrl_assign_mon_per_cpu
> and then writes an empty mask to kmode_cpus then it would essentially be setting
> inherit_ctrl_and_mon mode? This still seems ok since if disabling one of the "global"
> modes on a CPU results in that CPU inheriting from PQR_ASSOC then it seems
> reasonable to extend to when that mode is disabled for all CPUs.

Yes. That is correct.

> 
>>>>     # Disable kernel-mode steering (back to inherit, default group).
>>>
>>> This sounds like kernel work is steered to default group which I
>>> do not think is accurate for the "inherit_ctrl_and_mon" mode.
>>
>> How about ?
>>
>> Drop the kernel-mode binding and restore inherit_ctrl_and_mon on the default group.
> 
> No. There is no "inherit_ctrl_and_mon on the default group". There is nothing special
> about the default group when it comes to inherit_ctrl_and_mon mode ... or am I missing
> something?

You are right. There is nothing special about inherit_ctrl_and_mon on 
the default group.

> This could be something like: "Activate inherit_ctrl_and_mon mode to let
> kernel work inherit the resource allocation and monitoring from the user space task."
> (and drop the default group from the output associated with inherit_ctrl_and_mon)
> 

Sure.

Thanks
Babu