Re: [RFC] mpam,x86,fs/resctrl: Generic schema description Proof of Concept

["Moger, Babu" <bmoger@amd.com>]

Hi Reinette,

On 5/29/2026 1:06 PM, Reinette Chatre wrote:
> Hi Everybody,
> 
> It has been a while since we discussed the resctrl changes required to support
> hardware that has controls with fine granularity or hardware that has multiple
> controls per resource. For reference, the most recent email discussion can
> be found at [1] with a summary of discussions in last year's plumbers slides [2].
> 
> I created a PoC that I believe supports what folks have agreed to so far. I
> hope this can help us to restart the discussion with the goal that resctrl gains
> support for upcoming hardware that require these features.
> 
> Request regarding this PoC
> ==========================
> 
> Please consider this PoC as a "direction check" on the schema description and multiple
> control discussions held thus far.
> 
> Could folks working on enabling new hardware requiring this capability please consider
> if this is something you can build on and how it should be improved to support these
> upcoming capabilities?
> 
> Opens
> =====
> 
> While the PoC aims to support what folks agreed on some opens remain:
> - I attempted to make some MPAM supporting changes but these are all just compile
>    tested. While MPAM should benefit from the new control properties I did not
>    initialize them on MPAM and did not attempt refactor to separate out
>    the architecture specific control properties (more on what this means later).
>    I did attempt some MPAM refactoring that duplicates the MPAM domain to the
>    control domain and monitoring domain lists in support of there being multiple
>    controls each with its own list of control domains but it is definitely not good
>    design.
> - No support for emulated controls (yet). The PoC is quite large already
>    but I think it can be used as a base for emulated controls for which the software
>    controller could be a potential first customer. In this PoC mounting with
>    software controller will still display the original controller's properties.
> - One open that needs to be addressed as part of support for emulated controls is
>    how best to display emulation relationship via resctrl hierarchy.
> - No support for "read-modify-write" usage of schemata file. This is where we
>    discussed (without agreement) on possibly introducing the "#" prefix to schemata
>    file entries. This PoC does not support this prefix and the current assumption/expectation
>    is that when user space changes a configuration only the new control values are
>    written to schemata file. I thus do not have a plan to support this so please
>    share opinions in this regard if you have some.
> - Controls are independent for now. This means that, for example, if a resource
>    supports a "MIN" and "MAX" control then this implementation would allow user to
>    set the "maximum" control values to be less than the "minimum" control values.
> - PoC supports the "bitmap" control but does not (yet) expose properties of a bitmap
>    control to the new info/<resource>/resource_schemata directory.
> 
> Accessing PoC
> =============
> 
> Please consider the PoC as a rough draft. It has only been compile tested for Arm
> and known to be incomplete in Arm support. To help with experimenting I only
> fully adapted the Intel MBA resource to demo two dummy additional MBA controls.
> All architectures should immediately benefit from the new schema descriptions
> and new info/MB/resource_schemata hierarchy.
> 
> I considered the patches self too many for email. Instead, the PoC can be found at:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1
> 
> The work is based on v7.1-rc2 that also includes the following series (two of which has
> since been queued) included:
> 
> "selftests/resctrl: Fixes and improvements focused on Intel platforms"
> https://lore.kernel.org/lkml/cover.1775266384.git.reinette.chatre@intel.com/
> 
> "x86,fs/resctrl: Improve resctrl quality and consistency"
> https://lore.kernel.org/lkml/cover.1777419024.git.reinette.chatre@intel.com/
> 
> "x86,fs/resctrl: Pave the way for MPAM counter assignment"
> https://lore.kernel.org/lkml/20260506082855.3694761-1-ben.horgan@arm.com/
> 
> 
> Primary resctrl fs data structure changes
> =========================================
> 
> Introduces a control represented by struct resctrl_ctrl that looks as below. To make
> the changes easier to follow I kept some of the original names to help communicate
> where familiar data structures land.
> 
> What to notice about a control is that it has some common properties required
> from all controls (scope, type, etc.) and then depending on the type of control
> (RESCTRL_CTRL_BITMAP or RESCTRL_CTRL_SCALAR) there are type specific properties.
> 
> /**
>   * struct resctrl_ctrl - A resource control
>   * @entry:	List entry of rdt_resource::controls
>   * @scope:	Scope of the resource that this control allocates
>   * @domains:	RCU list of all control domains
>   * @type:	The control type that determines the properties of the control,
>   *		format string for displaying control values to user space, and
>   *		parser of control values provided by user space.
>   * @name:	Name of the control. Appended to final resource name
>   *		(rdt_resource_final::name) to create final schema entry.
>   *		Specifically, "rdt_resource_final::name"_"resctrl_ctrl::name".
>   *		For example, with resource name "MB" and control name "MAX" the
>   *		schema entry will be "MB_MAX".
>   * @cache:	Cache allocation control properties.
>   * @membw:	Bandwidth control properties.
>   */
> struct resctrl_ctrl {
> 	struct list_head	entry;
> 	enum resctrl_scope	scope;
> 	struct list_head	domains;
> 	enum resctrl_ctrl_type	type;
> 	enum resctrl_ctrl_name	name;
> 	union {
> 		struct resctrl_cache	cache;
> 		struct resctrl_membw	membw;
> 	};
> };
> 
> Two members summarize how this new structure fits into the rest of resctrl:
> a) resctrl_ctrl::entry
>     Since a resource can support multiple controls there is a new list
>     in struct rdt_resource named "controls" that contains the list of all
>     controls supported by the resource.
> b) resctrl_ctrl::domains
>     Instead of the list of control domains belonging to a resource they
>     now belong to the control self. By doing so resctrl can support resource
>     controls at different scope for the same resource. This is intended to
>     support some upcoming MPAM and RISC-V usages.
> 	
> Example architectural data structure changes
> ============================================
> 
> An architecture can use the new control by following a similar pattern to
> resource and domain use by architectures. Consider the following for x86
> where a new architecture specific struct resctrl_hw_ctrl includes
> struct resctrl_ctrl and any architecture private data needed to support
> the control:
> 
> /*
>   * struct resctrl_hw_ctrl - Arch private properties of a resource control
>   * @r_ctrl:	Control properties exposed to resctrl file system
>   * @msr_base:	Base MSR address where control values should be programmed
>   * @msr_update:	Function pointer to update control values
>   */
> struct resctrl_hw_ctrl {
> 	struct resctrl_ctrl	r_ctrl;
> 	unsigned int		msr_base;
> 	void			(*msr_update)(struct msr_param *m);
> };
> 
> Structure of patch series
> =========================
> 
> As a PoC the series is not perfectly structured but to help navigate this work
> on a high level the changes can be categorized as follows:
> 
> Patch 1 to 11:
> With a vision of what a "control" is, remove unused/unnecessary
> members, make clear what is a *resource* property vs a *control*
> property, do some renaming to help with the PoC.
> 
> Patch 12:
> Introduce struct resctrl_ctrl and re-arrange existing struct rdt_resource
> members to form part of new rdt_resource::ctrl
> 
> Patch 13 to 44:
> A lot of wrangling to introduce struct resctrl_ctrl to all code that needs
> to work with a control and/or domain without assuming that the control is
> the one and only control embedded in the resource it belongs to. Essentially,
> a lot of changes passing the control around in addition to the resource/domain.
> 
> Patch 45:
> Switch the single struct resctrl_ctrl member of struct rdt_resource to be
> a list of struct resctrl_ctrl.
> 
> Patch 47 to 49:
> Introduce new info/<resource>/resource_schemata hierarchy to first only
> consist of properties already known to resctrl fs.
> 
> Patch 50 to 52:
> Introduce the new control properties per [1], initialize them for x86,
> and expose them via info/<resource>/resource_schemata
> 
> Patch 53:
> Let the new struct resctrl_hw_ctrl contain architecture's control properties.
> 
> Patch 54:
> Teach resctrl fs about "MIN" and "MAX" controls.
> 
> Patch 55:
> Sample of "MIN" and "MAX" memory bandwidth controls for x86.
> 
> Example interactions
> ====================
> 
> This series can be used on an x86 system where it will show two new dummy controls
> where it is possible to interact with the new controls.
> For example:
> 
> # cat schemata
>          MB_MAX:0=100;1=100
>          MB_MIN:0=100;1=100
>          MB:0=100;1=100
>          L3:0=fff;1=fff
> # echo 'MB_MIN:0=50' > schemata
> # cat schemata
>          MB_MAX:0=100;1=100
>          MB_MIN:0=50;1=100
>          MB:0=100;1=100
>          L3:0=fff;1=fff
> 
> Writing to the dummy control will call a dummy callback that just prints to the
> kernel log:
> "resctrl: Updata temporary MIN control on domain 0 with user value 50"
> 
> 
> Example output of info/MB/:
> /sys/fs/resctrl/info/MB/thread_throttle_mode:max
> /sys/fs/resctrl/info/MB/num_closids:15
> /sys/fs/resctrl/info/MB/delay_linear:1
> /sys/fs/resctrl/info/MB/min_bandwidth:10
> /sys/fs/resctrl/info/MB/resource_schemata/MB/resolution:100
> /sys/fs/resctrl/info/MB/resource_schemata/MB/tolerance:5
> /sys/fs/resctrl/info/MB/resource_schemata/MB/type:scalar
> /sys/fs/resctrl/info/MB/resource_schemata/MB/min:10
> /sys/fs/resctrl/info/MB/resource_schemata/MB/scale:1
> /sys/fs/resctrl/info/MB/resource_schemata/MB/scope:L3
> /sys/fs/resctrl/info/MB/resource_schemata/MB/unit:all
> /sys/fs/resctrl/info/MB/resource_schemata/MB/max:100
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/resolution:100
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/tolerance:5
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/type:scalar
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/min:10
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/scale:1
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/scope:L3
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/unit:all
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MIN/max:100
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/resolution:100
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/tolerance:5
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/type:scalar
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/min:10
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/scale:1
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/scope:L3
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/unit:all
> /sys/fs/resctrl/info/MB/resource_schemata/MB_MAX/max:100
> /sys/fs/resctrl/info/MB/bandwidth_gran:10
> 
> Any feedback is appreciated.
> 
> Reinette
> 
> [1] https://lore.kernel.org/lkml/aPtfMFfLV1l%2FRB0L@e133380.arm.com/
> [2] https://lpc.events/event/19/contributions/2093/attachments/1958/4172/resctrl%20Microconference%20LPC%202025%20Tokyo.pdf
> 

Re-based GMBA patches[3] on top of this series.

[3] https://lore.kernel.org/lkml/cover.1776980182.git.babu.moger@amd.com/

#sudo mount -t resctrl resctrl /sys/fs/resctrl

#cat /sys/fs/resctrl/schemata
SMBA_GBL:0=4096;1=4096
      SMBA:0=8192;1=8192;2=8192;3=8192
    MB_GBL:0=4096;1=4096
        MB:0=8192;1=8192;2=8192;3=8192
        L3:0=ffff;1=ffff;2=ffff;3=ffff


Needed this change to get the schemata alignment correctly.

Note: Renamed GMB to MB_GBL and GSMBA to SMBA_GBL.

===========================================================
diff --git a/fs/resctrl/ctrlmondata.c b/fs/resctrl/ctrlmondata.c
index 979176b054e8..6123689e1250 100644
--- a/fs/resctrl/ctrlmondata.c
+++ b/fs/resctrl/ctrlmondata.c
@@ -512,17 +512,22 @@ static void show_doms(struct seq_file *s, struct 
rdt_resource_final *f,
  {
         struct rdt_resource *r = f->res;
         struct rdt_ctrl_domain *dom;
+       char res_name[16];
         bool sep = false;
         u32 ctrl_val;

         /* Walking r->domains, ensure it can't race with cpuhp */
         lockdep_assert_cpus_held();

-       if (print_ctrl)
-               seq_printf(s, "%*s%s%s:", max_name_width, f->name,
-                               resctrl_ctrl_is_default(ctrl) ? "" : "_",
-                               resctrl_ctrl_is_default(ctrl) ?
-                                "" : resctrl_ctrl_name_str(ctrl->name));
+       if (print_ctrl) {
+               if (resctrl_ctrl_is_default(ctrl))
+                       snprintf(res_name, sizeof(res_name), "%s", f->name);
+               else
+                       snprintf(res_name, sizeof(res_name), "%s_%s",
+                                       f->name, 
resctrl_ctrl_name_str(ctrl->name));
+               seq_printf(s, "%*s:", max_name_width, res_name);
+       }
+
         list_for_each_entry(dom, &ctrl->domains, hdr.list) {
                 if (sep)
                         seq_puts(s, ";");

====================================================================

$ tree /sys/fs/resctrl/info/MB/
/sys/fs/resctrl/info/MB/
├── bandwidth_gran
├── delay_linear
├── min_bandwidth
├── num_closids
└── resource_schemata
     ├── MB
     │   ├── max
     │   ├── min
     │   ├── resolution
     │   ├── scale
     │   ├── scope
     │   ├── tolerance
     │   ├── type
     │   └── unit
     └── MB_GBL
         ├── max
         ├── min
         ├── resolution
         ├── scale
         ├── scope
         ├── tolerance
         ├── type
         └── unit

4 directories, 20 files


$ tree /sys/fs/resctrl/info/SMBA/
/sys/fs/resctrl/info/SMBA/
├── bandwidth_gran
├── delay_linear
├── min_bandwidth
├── num_closids
└── resource_schemata
     ├── SMBA
     │   ├── max
     │   ├── min
     │   ├── resolution
     │   ├── scale
     │   ├── scope
     │   ├── tolerance
     │   ├── type
     │   └── unit
     └── SMBA_GBL
         ├── max
         ├── min
         ├── resolution
         ├── scale
         ├── scope
         ├── tolerance
         ├── type
         └── unit

4 directories, 20 files

Does the info directory hierarchy look right?

After going through the comments, it seems like it should cover most of 
the upcoming features. At least the Global MBA/SMBA stuff looks like it 
fits in well with the RFC.

What is your expectation for the RFC going forward?

Thanks
Babu