Re: [PATCH v2 00/16] fs,x86/resctrl: Add kernel-mode (e.g., PLZA) support to the resctrl subsystem

[Babu Moger <babu.moger@amd.com>]

Hi Reinette,

On 4/9/26 22:41, Reinette Chatre wrote:
> Hi Babu,
> 
> On 4/9/26 4:42 PM, Moger, Babu wrote:
>> Hi Reinette,
>>
>> On 4/9/2026 3:50 PM, Reinette Chatre wrote:
>>> Hi Babu,
>>>
>>> On 4/9/26 11:05 AM, Moger, Babu wrote:
>>>> On 4/9/2026 12:26 PM, Reinette Chatre wrote:
>>>>> On 4/9/26 10:19 AM, Moger, Babu wrote:
>>>>>> On 4/8/2026 6:41 PM, Reinette Chatre wrote:
>>>>>
>>>>>>> When the user switches to either "global_assign_ctrl_inherit_mon_per_cpu" or
>>>>>>> 'global_assign_ctrl_assign_mon_per_cpu" then "info/kernel_mode_assignment" is created
>>>>>>> (or made visible to user space) and is expected to point to default group.
>>>>>>> User can change the group using "info/kernel_mode_assignment" at this point.
>>>>>>>
>>>>>>> If the current scenario is below ...
>>>>>>>        # cat info/kernel_mode
>>>>>>>        [global_assign_ctrl_inherit_mon_per_cpu]
>>>>>>>        inherit_ctrl_and_mon
>>>>>>>        global_assign_ctrl_assign_mon_per_cpu
>>>>>>>
>>>>>>> ... then "info/kernel_mode_assignment" will exist but what it should contain if
>>>>>>> user switches mode at this point may be up for discussion.
>>>>>>>
>>>>>>> option 1)
>>>>>>> When user switches mode to "global_assign_ctrl_assign_mon_per_cpu" then
>>>>>>> the resource group in "info/kernel_mode_assignment" is reset to the
>>>>>>> default group and all CPUs PLZA state reset to match. The kernel_mode_cpus
>>>>>>> and kernel_mode_cpuslist files become visible in default resource group
>>>>>>> and they contain "all online CPUs".
>>>>>>>
>>>>>>> option 2)
>>>>>>> When user switches mode to "global_assign_ctrl_assign_mon_per_cpu" then
>>>>>>> the resource group in "info/kernel_mode_assignment" is kept and all
>>>>>>> CPUs PLZA state set to match it while also keeping the current
>>>>>>> values of that resource group's kernel_mode_cpus and kernel_mode_cpuslist
>>>>>>> files.
>>>>>>>
>>>>>>> I am leaning towards "option 1" to keep it consistent with a switch from
>>>>>>> "inherit_ctrl_and_mon" and being deterministic about how a mode is started with
>>>>>>
>>>>>> Yes. The "option 1" seems appropriate.
>>>>>>
>>>>>>> a clean slate. What are your thoughts? What would be use case where a user would
>>>>>>> want to switch between "global_assign_ctrl_inherit_mon_per_cpu" and
>>>>>>> "global_assign_ctrl_assign_mon_per_cpu" to just switch rmid_en on and off?
>>>>>>
>>>>>>
>>>>>> This is a bit tricky.
>>>>>>
>>>>>> Currently, our requirement is to have a CTRL_MON group for
>>>>>> global_assign_ctrl_inherit_mon_per_cpu. In this scenario, we use the
>>>>>> group’s CLOSID for PLZA configuration, and RMID is not used (rmid_en
>>>>>> = 0) when setting up PLZA.
>>>>>>
>>>>>> Our requirement is also to have a CTRL_MON/MON group for
>>>>>> global_assign_ctrl_assign_mon_per_cpu. In this case as well, the
>>>>>> group’s CLOSID and RMID (rmid_en = 1)  both are used configure PLZA.
>>>>>
>>>>> ah, right. Good catch.
>>>>>
>>>>>>
>>>>>> Actually, we should not allow these changes from
>>>>>> global_assign_ctrl_inherit_mon_per_cpu  to
>>>>>> global_assign_ctrl_assign_mon_per_cpu or visa versa.
>>>>>
>>>>> resctrl could allow it but as part of the switch it resets the "kernel mode group" to
>>>>> be the default group every time? This would be the "option 1" above.
>>>>
>>>> Other options.
>>>>
>>>> Allow global_assign_ctrl_inherit_mon_per_cpu -> global_assign_ctrl_assign_mon_per_cpu. As part of the switch, reset the "kernel mode group" to the default group.
>>>>
>>>> Allow global_assign_ctrl_assign_mon_per_cpu -> global_assign_ctrl_inherit_mon_per_cpu. In this case switch
>>>> to CTRL_MON/MON -> CTRL_MON.
>>>>
>>>
>>> ok. Could you please return the courtesy of providing feedback on the
>>> suggestion you are responding to and also include the motivation why your
>>> suggestion is the better option?
>>
>> Yea. Sure.
>>
>> We need to allow the switch between the modes. Otherwise only way to reset is to remount the resctrl filesystem. That is not a good option.
>>
>> Allow global_assign_ctrl_inherit_mon_per_cpu -> global_assign_ctrl_assign_mon_per_cpu. As part of the switch, reset the "kernel mode group" to the default group.
>>
>> This option is same as you suggested.
>>
>> Allow global_assign_ctrl_assign_mon_per_cpu -> global_assign_ctrl_inherit_mon_per_cpu. In this case switch
>> to CTRL_MON/MON -> CTRL_MON. This option basically disables monitor (rmid_en=0). It is less disruptive. Move is between child group to parent group.
> 
> ok. I am concerned that this creates an inconsistent interface. Specifically, sometimes
> when switching the mode the kernel group will reset and sometimes it won't. This inconsistency
> may be more apparent when writing the user documentation as part of this work. If you are
> able to clearly explain how this resctrl fs interface behaves (this cannot be about PLZA
> internals as above) then this could work.
Started working on these changes. May be it is better to discuss this 
before to avoid one more revision.


The current mode change behavior is very restrictive.

For example:

# cat info/kernel_mode
       inherit_ctrl_and_mon
       [global_assign_ctrl_assign_mon_per_cpu]
        global_assign_ctrl_inherit_mon_per_cpu


# cat info/kernel_mode_assignment
      ctrl1/mon1/

In this state, we cannot change kernel_mode to inherit_ctrl_and_mon. The 
expectation, however, is that inherit_ctrl_and_mon should always map to 
the RDTCTRL_GROUP.


A similar issue exists when switching between
global_assign_ctrl_inherit_mon_per_cpu and
global_assign_ctrl_assign_mon_per_cpu (in either direction).

The same problem also occurs when modifying the kernel_mode_assignment 
group. If the current group is an RDTMON_GROUP, we can't assign another
RDTCTRL_GROUP without changing both mode and group together.


To address this, I propose changing the mode and the group together.

System boots up with following defaults:

# cat info/kernel_mode
       [inherit_ctrl_and_mon]
       global_assign_ctrl_assign_mon_per_cpu
       global_assign_ctrl_inherit_mon_per_cpu

# cat info/kernel_mode_assignment
      inherit_ctrl_and_mon://


# echo "global_assign_ctrl_assign_mon_per_cpu:ctrl1/mon1/" > 
info/kernel_mode_assignment

# cat info/kernel_mode_assignment
      global_assign_ctrl_assign_mon_per_cpu:ctrl1/mon1/

# cat info/kernel_mode
       inherit_ctrl_and_mon
       [global_assign_ctrl_assign_mon_per_cpu]
       global_assign_ctrl_inherit_mon_per_cpu


# echo "inherit_ctrl_and_mon://" > info/kernel_mode_assignment

# cat info/kernel_mode_assignment
    inherit_ctrl_and_mon://


# cat info/kernel_mode
       [inherit_ctrl_and_mon]
       global_assign_ctrl_assign_mon_per_cpu
        global_assign_ctrl_inherit_mon_per_cpu


# echo "global_assign_ctrl_inherit_mon_per_cpu:ctrl1//"

# cat info/kernel_mode_assignment
    global_assign_ctrl_inherit_mon_per_cpu:ctrl1//

# cat info/kernel_mode
       inherit_ctrl_and_mon
       global_assign_ctrl_assign_mon_per_cpu
       [global_assign_ctrl_inherit_mon_per_cpu]


The interface "info/kernel_mode" becomes read-only,

The mode change and group change will be done with 
"info/kernel_mode_assignment"


I’m also planning to rename the kernel modes as follows:

inherit_ctrl_and_mon → shared_alloc_mon
global_assign_ctrl_inherit_mon_per_cpu → global_alloc_per_cpu
global_assign_ctrl_assign_mon_per_cpu → global_alloc_mon_per_cpu

What do you think?

Thanks
Babu