Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

[Fenghua Yu <fenghuay@nvidia.com>]

Hi, Reinette,

On 7/23/26 09:18, Reinette Chatre wrote:
> Hi Fenghua,
> 
> On 7/22/26 5:17 PM, Fenghua Yu wrote:
>> On 7/14/26 15:06, Reinette Chatre wrote:
>>> On 7/10/26 1:59 PM, Fenghua Yu wrote:
>>>> On 6/25/26 08:43, Reinette Chatre wrote:
>>>>> On 6/24/26 6:26 PM, Fenghua Yu wrote:
>>>>>> On 6/24/26 15:22, Reinette Chatre wrote:
>>>>>>> On 6/24/26 12:08 PM, Fenghua Yu wrote:
>>>>>>>> On 5/29/26 11:06, Reinette Chatre wrote:
> 
> ...
> 
>>>>>>>> 2. /sys/fs/resctrl/info/MB/max_lim
>>>>>>>> It shows number 0-3 for MPAM MBW max limit behaviors: 0 for supporting both softlimit and hardlimit, etc.
>>>>>>>
>>>>>>> Again this adds another *global* property to the MB resource but then above you
>>>>>>> describe the new "MB_HLIM" schemata file entry that implies that it is a new control
>>>>>>> for the MB resource. Having it be a new control for the MB resource matches earlier
>>>>>>> discussions. To support this I thus expect it to be exposed as a new control with
>>>>>>> potentially a new type if any of the existing planned types do not suffice.
>>>>>>>
>>>>>>
>>>>>> How about adding these MB_HLIM dir and files in info?
>>>>>>
>>>>>> /sys/fs/resctrl/info/MB_HLIM/resource_schemata/MB_HLIM/type: boolean
>>>>>> /sys/fs/resctrl/info/MB_HLIM/resource_schemata/MB_HLIM/max_lim: 0
>>>>>
>>>>> This presents "MB_HLIM" as a *resource* to user space. It is not a resource
>>>>> but a *control* of a resource, no? I thus expect it to instead look something like
>>>>> below that makes it clear that MB_HARDMAX is a control of the MB resource.
>>>>>
>>>>> info
>>>>> └── MB
>>>>>        └── resource_schemata
>>>>>            ├── MB
>>>>>            └── MB_HARDMAX
>>>>
>>>> Yes, this makes sense. I have changed to this hierarchy.
>>>
>>> Thank you very much for considering this approach.
>>
>> [ MB_MAXHLIM: I use this name for MBW_MAX hard limit feature as Dave Martin suggested before. He also suggested MB_HARDMAX. Either name is good for me. I use MB_MAXHLIM to explain MBW_MAX hard limit for now.]
>>
>> Some implementation thoughts:
>>
>> MBW_MAX hard limit itself is not a MB control. Rather, it configures
>> MB control, i.e. turn on MB control's hard limit or turn off its
>> hard limit. So MBW_MAX hard limit doesn't have properties like
>> bandwidth_gran, delay_linear, etc. MBW_MAX hard limit's property is
>> only a boolean type.
> 
> "MB" is becoming more and more a software concept that represents memory
> bandwidth allocation and in that sense I agree that the "hard limit" is not
> a "MB" control. Even so, since "hard limit" needs to be exposed via
> schemata file it is *a* control. This is because as part of exposing it via
> the schemata file it requires the following:
> - a "scope" that specifies what the "domain ID" associated with this control in schemata file means
> - "domains" to be the list of domains, one per "domain ID" from "scope", to contain the
>    staged values that user space provides when making changes to this control
> - a "name" that is presented in schemata file
> - resctrl needs a way to pass the user provided values to the architecture for
>    programming, the API for this is to pass struct resctrl_ctrl.
> 
> You are correct that none of the current control types are a good fit for this
> new boolean type. resctrl would need to support a new boolean control type.
> Ben and I recently exchanged a few ideas around this. Please see the thread
> that starts with the message below (search for HARDLIM in the message):
> https://lore.kernel.org/lkml/34b95afb-8b60-4680-9ad1-90c5b24e8fb7@arm.com/

Yes, I had some input in the thread.

> 
> 
>>
>> So I would think it maybe a configuration inside a control.
>>
>> Similar configurations could be hard limit for cache capacity in MPAM.
>>
>> Maybe can add "configs" inside resctrl_ctrl. Schemata and info/MB/
>> resource_schemata/MB will show/write the configurations per control?
>>
>> For this configuration or future configurations, add "configs" list in:
>> struct resctrl_ctrl {
>>          struct list_head        entry;
>>          enum resctrl_scope      scope;
>>          struct list_head        domains;
>>          enum resctrl_ctrl_type  type;
>>          enum resctrl_ctrl_name  name;
>>          struct resctrl_ctrl     *emulated_by;
>>          struct list_head        configs; <--- Add configs for this control
>>          union {
>>                  struct resctrl_cache    cache;
>>                  struct resctrl_membw    membw;
>>          };
>> };
> 
> As I understand the idea is that configs is a list of struct resctrl_ctrl?

Yes, it's a list. A control may have a few configurations. Currently 
there is only MB_MAXHLIM configuration in the list in MB control. A list 
may be expanded to future multiple configurations in one control.

> 
> At first glance it is not clear to me why the additional layer of abstraction is
> needed. What do you think of Ben's suggestion in thread I mentioned earlier to
> instead name the control:
> "<resource_type>_<scope>_<control>_<control_param> where <control_param> is HARDLIM"
> (nit: While I understand HARDLIM matches more closely to MPAM spec I do find
>   "HARDMAX" easier to understand)

Yes, add the name HARDLIM is fine too.

But the arguments for adding this configurations in a control are:

A "configurations" is different from a "control" in that:
1. The configuration configures the control, e.g. toggle hard limit on
MB control.
2. The configuration doesn't have the control's properties (e.g. gran).
3. The configuration is similar to "event_configs" in MB_MON, that can
also configure MB_MON's event_filter.

So distinguish "configuration" and "control" might be more hierarchy? 
Adding the config name in control name doesn't show the hierarchy of the 
MB and its configs?

> 
> The additional layer of abstraction would require both resctrl fs and
> the architecture to dig through two lists when handling these controls and it is
> not clear to me that this is necessary.
> 
>   
>> A resctrl control can have one or multiple configurations. Currently
>> MBW_MAX hard limit is the only one. But the infrastrucutre supports
>> multiple configurations per control.
>>
>> schemata:
>>          MB:1=100   <-- MBW_MAX on L3 id 1
>> MB_MAXHLIM:1=0     <-- turn on/off MBW_MAX hardlimit on L3 id 1
>>          L3:1=fff
>>
>> info/
>> ├── MB
>> │   ├── bandwidth_gran
>> │   ├── delay_linear
>> │   ├── min_bandwidth
>> │   ├── num_closids
>> │   └── resource_schemata
>> │       ├── MB
>> │       │   ├── configs
>> │       │   │   └── MB_MAXHLIM
>> │       │   │       └── type   <--- bool
>> │       │   ├── max
>> │       │   ├── min
>> │       │   ├── resolution
>> │       │   ├── scale
>> │       │   ├── scope
>> │       │   ├── status
>> │       │   ├── tolerance
>> │       │   ├── type
>> │       │   └── unit
>> │       └── mode
>>
>>
>> Is this a valid way to handle MB_MAX hard limit (and future more configurations per control)?
> It is possible but it does look very complicated to me when compared to using the control name to
> express relationship between control and configuration.

This configuration is similar to "event_configs" in "MB_MON":
└── MB_MON
     ├── available_mbm_cntrs
     ├── event_configs
     │   └── mbm_total_bytes
     │       └── event_filter
     ├── mbm_assign_mode
     ├── mbm_assign_on_mkdir
     ├── mon_features
     ├── num_mbm_cntrs
     └── num_rmids

So it's not a brand new info hierarchy. Both of configs change a 
resource contorl/monitor. It's just for "MB" not for "MB_MON".

What do you think?

Thanks.

-Fenghua