Re: [PATCH v2 00/16] fs,x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Babu Moger <babu.moger@amd.com>]

Hi Reinette,

On 4/21/26 11:15, Reinette Chatre wrote:
> Hi Babu,
> 
> On 4/21/26 8:08 AM, Babu Moger wrote:
>> Hi Reinette,
>>
>> On 4/20/26 22:17, Reinette Chatre wrote:
>>> Hi Babu,
>>>
>>> On 4/20/26 5:40 PM, Moger, Babu wrote:
>>>>
>>>> We already discussed moving back to the default group on every mode
>>>> switch. Doing so here would once again cause extra MSR writes on
>>>> each mode transition, which is undesirable.
>>>>
>>>
>>> Needing to avoid extra MSR writes in resctrl is not so absolute. Consider, for
>>> example, how resctrl initializes default allocations when a new resource group is
>>> created. resctrl aims to initialize with sane defaults and the user is expected to
>>> follow with desired allocations.
>>>
>>> I am not against optimizing, I just want to be careful with such general statements.
>>>
>>> Considering your proposal in https://lore.kernel.org/lkml/39e0c786-cc35-4555-bfb9-ff7cd758c423@amd.com/:
>>>
>>> I do not think we should make info/kernel_mode read-only. If I understand correctly
>>> doing so would accommodate AMD PLZA but it ignores the discussions on how resctrl could
>>> support MPAM ... or do you perhaps have proposal on how MPAM can be supported when considering
>>> your proposal? Even if you do not want to consider MPAM - what if the PLZA_PQR register's
>>> scope becomes per-CPU in the next version of AMD PLZA?
>>>
>>> The idea behind info/kernel_mode is that the active mode it identifies indicates which
>>> configuration files exist to configure the active mode. Since the mode may not always
>>> depend on global configuration, for which info/kernel_mode_assignment was created, but instead
>>> rely on per-resource group files, I do not see how resctrl can build on a read-only
>>> info/kernel_mode backed by a mode and group change via info/kernel_mode_assignment.
>>> Specifically, MPAM support may not use info/kernel_mode_assignment at all.
>>> Instead, MPAM may use something like described in https://lore.kernel.org/lkml/aYyxAPdTFejzsE42@e134344.arm.com/
>>>
>>> Could we perhaps consider dropping info/kernel_mode_assignment entirely for
>>> AMD PLZA's global allocations? Similar to what you suggest, the mode and
>>> group assignment could be done via the info/kernel_mode file instead?
>>>
>>> Thinking about this more since the CPUs allocation is global, these could *theoretically*
>>> be included also (but see later).
>>> This could mean that "kernel_mode_cpus" and "kernel_mode_cpus_list" could be dropped?
>>> Although, this may complicate the interface since user space may want a convenient way
>>> to modify just CPUs independently from needing to repeat the mode and group every time.
>>>
>>> Consider, for example:
>>>
>>> # echo "global_assign_ctrl_assign_mon_per_cpu:group=ctrl1/mon1/;cpus_list=5-8" > info/kernel_mode
>>
>> This looks reasonable.
>>
>>>
>>> Having named fields (a) makes this extensible, (b) output does not need to be split among files,
>>> and (c) "inherit_ctrl_and_mon" can continue to be supported.
>>>
>>> The named fields could be made optional, if group is omitted then it will become the
>>> default resource group, and if cpus/cpus_list is omitted then it will default to all CPUs.
>>> This may not be intuitive since a user may expect that not mentioning a field means
>>> that the field is left untouched. Have you considered this scenario in your proposal?
>>>
>>> As an alternative the group could be made a required field and "kernel_mode_cpus"/"kernel_mode_cpuslist"
>>> can stay? This may be the simplest approach.
>>
>> How about keeping a single option to update the CPUs using
>> kernel_mode_cpus / kernel_mode_cpuslist within the group?
>>
>> Should we consider removing the per‑CPU extension altogether? By
>> default, the mode already applies to all online CPUs, and any
>> per‑CPU requirements can be handled within the group using
>> kernel_mode_cpus / kernel_mode_cpuslist.
> 
> It sounds like we are saying the same thing?
> When considering all the sharp corners I agree that keeping kernel_mode_cpus/kernel_mode_cpuslist
> seems most user friendly. When doing so there is no need to include CPU assignment in the global
> files.

Actually, I was talking about removing _per_cpu extension also as the 
per-CPU requirement is handled inside the group using 
kernel_mode_cpus/kernel_mode_cpuslist. It can be documented.

global_assign_ctrl_assign_mon_per_cpu -> global_assign_ctrl_assign_mon
global_assign_ctrl_inherit_mon_per_cpu -> global_assign_ctrl_inherit_mon


>>
>> # echo "global_assign_ctrl_assign_mon_per_cpu:group=ctrl1/mon1/
>>
>> Why do we still need to keep the "inherit_ctrl_and_mon"?  By default all the groups in the system falls in this category it is not plza enabled group.
>>
>>
>> System boots up with following options if PLZA is supported.
>>
>> # cat info/kernel_mode
>>        global_assign_ctrl_assign_mon_per_cpu
>>        global_assign_ctrl_inherit_mon_per_cpu
>>
>> No groups are associated with kernel mode at this point.
> 
> To me it seems useful to be clear to user space on what the current mode is. If I understand correctly
> above default scenario essentially means "inherit_ctrl_and_mon" but instead of adding it to this file
> we will need to add documentation that describes to user space how this file should be interpreted.
> It seems easier to me to just be clear via info/kernel_mode itself on what the current active mode is?
> 
> I think something like below will be more intuitive and not need much additional
> documentation to understand (I am just adding the "uninitialized" as an example to match text
> printed in schemata file during pseudo-locking ... even if there is a group named "uninitialized"
> the lack of "/" could be used to make it clear what this means?):
> 
> 	# cat info/kernel_mode
> 	[inherit_ctrl_and_mon]
> 	global_assign_ctrl_assign_mon_per_cpu:group=uninitialized
> 	global_assign_ctrl_inherit_mon_per_cpu:group=uninitialized
> 

Sounds ok to me.


> I also think an interface like this would be simpler for user space to use as it (user space) switches
> between PLZA capable and non-PLZA capable systems since user space need not associate existence of
> the file with some kernel mode state in addition to actual content of the file when it does exist.
> 
> I assumed that info/kernel_mode can just always be made visible and not depend on PLZA
> capable hardware. This means that on Intel and Arm this file can show:
> 
> 	# cat info/kernel_mode
> 	[inherit_ctrl_and_mon]
> 

Yes. Sure.


> For Intel this is accurate and also for Arm if I interpret the Arm implementation correctly
> (see mpam_thread_switch()) in  https://lore.kernel.org/lkml/20260313144617.3420416-7-ben.horgan@arm.com/
> 
>>
>> # echo "global_assign_ctrl_assign_mon_per_cpu:group=ctrl1/mon1/" > info/kernel_mode
>>
>> # cat info/kernel_mode
>>    global_assign_ctrl_assign_mon_per_cpu:group=ctrl1/mon1/
>>    global_assign_ctrl_inherit_mon_per_cpu
>>
>>
>> # echo "global_assign_ctrl_inherit_mon_per_cpu:group=//" > info/kernel_mode
>>
>>
>> # cat info/kernel_mode
>>    global_assign_ctrl_assign_mon_per_cpu
>>    global_assign_ctrl_inherit_mon_per_cpu:group=//
>>
>>
>> How does this look?
> 
> In addition to above I think it will be helpful to add a clear indication to user
> space on what the current active mode is, for example, via the [] characters.

# echo "global_assign_ctrl_assign_mon_per_cpu:group=ctrl1/mon1/" > 
info/kernel_mode

# cat info/kernel_mode
    inherit_ctrl_and_mon
    global_assign_ctrl_assign_mon_per_cpu:group=uninitialized
    [global_assign_ctrl_assign_mon_per_cpu]:group=ctrl1/mon1/

Something like this?

There is one problem here. The mode "inherit_ctrl_and_mon" listing not 
consistent with others.

Thanks
Babu