[RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre]

Hi Everybody,                                                                   
                                                                                
I understand that folks are eager for this work to settle so that features that
depend on it can make progress.
                                                                                
I do have a request if you have a feature that depends on this PoC:
                                                                                
	Please do not fork this PoC and keep changes to the PoC code in your fork
        to meet the requirements of the feature you are working on. This is resulting
        in duplicate (even triplicate) changes to PoC floating around. Instead,
        please collaborate on this PoC by responding to the patches that need changes/fixes
        to support your feature. If the patches do not exist yet, please participate in
        the discussions that plan these changes so the work adding the discussed features
	is not duplicated. If you already have the code, please feel free to send as fixups.
        Doing so will help this PoC to settle faster and help meet the requirements of the
        feature(s) a couple of folks are working on.
                                                                                
RFC v1: https://lore.kernel.org/lkml/aab804b9-e8b5-40ad-a85b-af7033391243@intel.com/
	git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1

RFC v2 of this PoC can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v2

The list of changes is long. For an overview of the new interfaces, please see the new
documentation patch:
https://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git/patch/?id=2e2d0140e857eae2ba3e71e8d9c595897649599a

Remaining Opens:
---------------
- MPAM only compile tested.
- Only "SAMPLE" code provided for x86 to demonstrate emulated controls.
- Architecture should take care to restore control value of legacy control when
  switching mode from "native" to "legacy". This may be complicated by a
  legacy control backed by multiple native controls and thus best handled by
  architecture. This adds a complication that resctrl cannot promise that
  switching between control modes will not be destructive since a reverse mapping
  may not exist.
- When switching control mode to native the top level control info files
  stop returning information. This could be considered "work as intended"?
- Add relationships between controls. For example, cannot set "MIN" control
  value to be larger than "MAX. Unclear how to do this in resctrl fs when
  considering that the relationship may span between enabled and disabled
  controls. This is potentially something that needs to be managed by architecture
  and thus potentially without a consistent failure to user space.
                                                                                
Changes since RFC v1:
--------------------
There are many changes since RFC v1 since I aimed to address all feedback
(but please see the deferred/dropped list below). If I missed anything it
was not intentional. Please feel free to point it out to me.

User visible changes:
--------------------
- Expose bitmap control properties to user space.
    -- Only expose two properties:
       - resctrl_ctrl_bitmap::min_cbm_bits exposed via new file "min_bits".
         This drops the familiar "cbm" term to not make the control specific
         to capacity.
       - resctrl_ctrl_bitmap::cbm_len exposed via "max" file
    -- While "shareable_bits" is technically a bitmap control property this does
       seem an opportunity to deprecate it since the IO alloc support proved
       that it does not accurately represent shared allocations, "bit_usage"
       is the best for that.
- Add support for control flags with "linear" as first flag of the "scalar"
  control and "sparse" the first flag of the "bitmap" control.
  Include flags in output of the "type" file.
  Suggested by Chenyu: https://lore.kernel.org/lkml/90ae82da-02b2-4058-8c1e-f16df3226d8a@intel.com/
- Add a "default" field to a control using the "reset_val" name suggested
  by Drew and return that instead of always using "max" for the scalar control.
  The "MIN" control can return its minimum value as default.
  Suggested by Ben in https://lore.kernel.org/lkml/29c95b69-e1a4-46b1-ab8b-45c09308b924@arm.com/
  Use case from Drew: https://lore.kernel.org/lkml/aiCBratZchVFVhws@gen8/
  Drew suggested name as "reset_val" https://lore.kernel.org/lkml/aiOrznnZjTGywMna@thelio>
- Change "resource_schemata" to "schemata".
  Fenghua: https://lore.kernel.org/lkml/d0f84cef-58f5-4b84-8171-a0dfe8fa6829@nvidia.com/
- Fix the control alignment. (Chenyu and Tony)
- Drop the "per control" scope. Move the "scope" back to being per-resource and
  do not expose a per control "scope" file.  This helps to support new resources
  like node scoped MBA that cannot emulate the legacy MB control. Also
  supports emulated controls by ensuring that all controls associated with
  a resource have the same scope.
- Support emulating legacy control.
  - Support multiple backing controls for a legacy control.
  - Expose new per-resource "control_mode" file that user space can use to
    switch between legacy and native controls. Only the enabled controls
    are shown to user in schemata file.
  - The new "native" control mode is only supported for the MBA resource.
    The solution is generic but there are implications for, for example, the
    resctrl features that rely on the legacy cache control (cache pseudo-locking
    and IO alloc). Defer support for other resource since there are no immediate
    plans to have, for example, the legacy cache control backed by something else.
  - Architecture differences imply a lot of architecture flexibility in
    support for emulating controls.
    Architecture can expect resctrl fs to only stage control values in
    controls that are enabled. Within architecture there could be rules
    that determine what to program based on the control's emulated controls
    or it could dynamically program update callbacks when the user changes
    the control mode. RFC includes sample code identified by "SAMPLE" for the
    latter that demonstrates one way to solve the x86 variant where the hardware
    self supports both legacy and finer grained controls.
  - The top level legacy control property files stop returning data when
    the mode switches to "native".
- Draft of documentation that describes multiple controls and emulated
  controls.                                                                     
- Fix Intel MBA tolerance to be 10 to match the "round-up" implementation.

Changes not user visible:
-------------------------
- Rebase on tip x86/cache with following applied on top:
  "x86,fs/resctrl: Improve resctrl quality and consistency"
    https://lore.kernel.org/lkml/cover.1782857711.git.reinette.chatre@intel.com/
  "x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch"
    https://lore.kernel.org/lkml/20260709093111.367851-1-ben.horgan@arm.com/
- Fix initialization of controls list in MPAM. (Ben)
- Dan Carpenter reported an issue with missing unlocks in
  rdt_bit_usage_show() that was resolved as part of rebase on top of the
  new info_kn_lock()/info_kn_unlock() helpers.
- Drop "mpam,x86,fs/resctrl: Make memory bandwidth delay a resource
  property" so that memory bandwidth delay linear flag can be a control flag
  instead.
- Separate adding the control to struct msr_param from the change that
  makes hardware update functions part of control.
  ("x86/resctrl: Make control update properties part of the control") to
  create new patch that is moved earlier in series to support bandwidth
  linear flag property being part of control.
  New patch: "x86/resctrl: Include control with information controlling hardware change"
- Rename the controls to match their type to reduce confusion.
  Suggested by Chenyu: https://lore.kernel.org/lkml/773353e8-b2a7-4c20-b6fa-195b0b301107@intel.com/
  New names:
  struct resctrl_membw -> struct resctrl_ctrl_scalar
  struct resctrl_cache -> struct resctrl_ctrl_bitmap
  resctrl_ctrl::membw -> resctrl_ctrl::scalar
  resctrl_ctrl::cache -> resctrl_ctrl::bitmap
- Switch subject prefix to be "arm,x86,fs/resctrl: ..." when changing both
  architectures and resctrl fs code.
- Switch to MPAM subject prefix of "arm_mpam: resctrl:" when only changing
  MPAM code.

Changes deferred/dropped:
------------------------
- Let "scope" form part of the control name.
  Drop this since the current plan is to have the scope be part of the resource
  name for new resources.
- Introduce new per-control "status" that can be "enabled" or "disabled".
  Drop this. While controls now have an "enabled" or "disabled" state internally,
  this is not exposed to user space via a file. Instead, the "enabled" or
  "disabled" state is exposed to user space via the entries appearing in schemata
  file. This simplifies the subtleties when the hardware self supports both
  legacy and finer grained interfaces (aka RDT's region-aware).
- Defer emulated control support for cache controls.
- Supporting different monitoring scope is not part of this PoC.
                                                                                
Any feedback is appreciated.

Reinette

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Ben Horgan]

Hi Reinette,

On 8/4/26 06:26, Reinette Chatre wrote:
> Hi Everybody,                                                                   
>                                                                                 
> I understand that folks are eager for this work to settle so that features that
> depend on it can make progress.
>                                                                                 
> I do have a request if you have a feature that depends on this PoC:
>                                                                                 
> 	Please do not fork this PoC and keep changes to the PoC code in your fork
>         to meet the requirements of the feature you are working on. This is resulting
>         in duplicate (even triplicate) changes to PoC floating around. Instead,
>         please collaborate on this PoC by responding to the patches that need changes/fixes
>         to support your feature. If the patches do not exist yet, please participate in
>         the discussions that plan these changes so the work adding the discussed features
> 	is not duplicated. If you already have the code, please feel free to send as fixups.
>         Doing so will help this PoC to settle faster and help meet the requirements of the
>         feature(s) a couple of folks are working on.
>                                                                                 
> RFC v1: https://lore.kernel.org/lkml/aab804b9-e8b5-40ad-a85b-af7033391243@intel.com/
> 	git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1
> 
> RFC v2 of this PoC can be found at:
> git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v2
> 
> The list of changes is long. For an overview of the new interfaces, please see the new
> documentation patch:
> https://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git/patch/?id=2e2d0140e857eae2ba3e71e8d9c595897649599a
> 
> Remaining Opens:
> ---------------
> - MPAM only compile tested.
> - Only "SAMPLE" code provided for x86 to demonstrate emulated controls.
> - Architecture should take care to restore control value of legacy control when
>   switching mode from "native" to "legacy". This may be complicated by a
>   legacy control backed by multiple native controls and thus best handled by
>   architecture. This adds a complication that resctrl cannot promise that
>   switching between control modes will not be destructive since a reverse mapping
>   may not exist.
> - When switching control mode to native the top level control info files
>   stop returning information. This could be considered "work as intended"?
> - Add relationships between controls. For example, cannot set "MIN" control
>   value to be larger than "MAX. Unclear how to do this in resctrl fs when
>   considering that the relationship may span between enabled and disabled
>   controls. This is potentially something that needs to be managed by architecture
>   and thus potentially without a consistent failure to user space.
>                                                                                 
> Changes since RFC v1:
> --------------------
> There are many changes since RFC v1 since I aimed to address all feedback
> (but please see the deferred/dropped list below). If I missed anything it
> was not intentional. Please feel free to point it out to me.
> 
> User visible changes:
> --------------------
> - Expose bitmap control properties to user space.
>     -- Only expose two properties:
>        - resctrl_ctrl_bitmap::min_cbm_bits exposed via new file "min_bits".
>          This drops the familiar "cbm" term to not make the control specific
>          to capacity.
>        - resctrl_ctrl_bitmap::cbm_len exposed via "max" file
>     -- While "shareable_bits" is technically a bitmap control property this does
>        seem an opportunity to deprecate it since the IO alloc support proved
>        that it does not accurately represent shared allocations, "bit_usage"
>        is the best for that.
> - Add support for control flags with "linear" as first flag of the "scalar"
>   control and "sparse" the first flag of the "bitmap" control.
>   Include flags in output of the "type" file.
>   Suggested by Chenyu: https://lore.kernel.org/lkml/90ae82da-02b2-4058-8c1e-f16df3226d8a@intel.com/
> - Add a "default" field to a control using the "reset_val" name suggested
>   by Drew and return that instead of always using "max" for the scalar control.
>   The "MIN" control can return its minimum value as default.
>   Suggested by Ben in https://lore.kernel.org/lkml/29c95b69-e1a4-46b1-ab8b-45c09308b924@arm.com/
>   Use case from Drew: https://lore.kernel.org/lkml/aiCBratZchVFVhws@gen8/
>   Drew suggested name as "reset_val" https://lore.kernel.org/lkml/aiOrznnZjTGywMna@thelio>
> - Change "resource_schemata" to "schemata".
>   Fenghua: https://lore.kernel.org/lkml/d0f84cef-58f5-4b84-8171-a0dfe8fa6829@nvidia.com/
> - Fix the control alignment. (Chenyu and Tony)
> - Drop the "per control" scope. Move the "scope" back to being per-resource and
>   do not expose a per control "scope" file.  This helps to support new resources
>   like node scoped MBA that cannot emulate the legacy MB control. Also
>   supports emulated controls by ensuring that all controls associated with
>   a resource have the same scope.
> - Support emulating legacy control.
>   - Support multiple backing controls for a legacy control.
>   - Expose new per-resource "control_mode" file that user space can use to
>     switch between legacy and native controls. Only the enabled controls
>     are shown to user in schemata file.
>   - The new "native" control mode is only supported for the MBA resource.
>     The solution is generic but there are implications for, for example, the
>     resctrl features that rely on the legacy cache control (cache pseudo-locking
>     and IO alloc). Defer support for other resource since there are no immediate
>     plans to have, for example, the legacy cache control backed by something else.
>   - Architecture differences imply a lot of architecture flexibility in
>     support for emulating controls.
>     Architecture can expect resctrl fs to only stage control values in
>     controls that are enabled. Within architecture there could be rules
>     that determine what to program based on the control's emulated controls
>     or it could dynamically program update callbacks when the user changes
>     the control mode. RFC includes sample code identified by "SAMPLE" for the
>     latter that demonstrates one way to solve the x86 variant where the hardware
>     self supports both legacy and finer grained controls.
>   - The top level legacy control property files stop returning data when
>     the mode switches to "native".
> - Draft of documentation that describes multiple controls and emulated
>   controls.                                                                     
> - Fix Intel MBA tolerance to be 10 to match the "round-up" implementation.
> 
> Changes not user visible:
> -------------------------
> - Rebase on tip x86/cache with following applied on top:
>   "x86,fs/resctrl: Improve resctrl quality and consistency"
>     https://lore.kernel.org/lkml/cover.1782857711.git.reinette.chatre@intel.com/
>   "x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch"
>     https://lore.kernel.org/lkml/20260709093111.367851-1-ben.horgan@arm.com/
> - Fix initialization of controls list in MPAM. (Ben)
> - Dan Carpenter reported an issue with missing unlocks in
>   rdt_bit_usage_show() that was resolved as part of rebase on top of the
>   new info_kn_lock()/info_kn_unlock() helpers.
> - Drop "mpam,x86,fs/resctrl: Make memory bandwidth delay a resource
>   property" so that memory bandwidth delay linear flag can be a control flag
>   instead.
> - Separate adding the control to struct msr_param from the change that
>   makes hardware update functions part of control.
>   ("x86/resctrl: Make control update properties part of the control") to
>   create new patch that is moved earlier in series to support bandwidth
>   linear flag property being part of control.
>   New patch: "x86/resctrl: Include control with information controlling hardware change"
> - Rename the controls to match their type to reduce confusion.
>   Suggested by Chenyu: https://lore.kernel.org/lkml/773353e8-b2a7-4c20-b6fa-195b0b301107@intel.com/
>   New names:
>   struct resctrl_membw -> struct resctrl_ctrl_scalar
>   struct resctrl_cache -> struct resctrl_ctrl_bitmap
>   resctrl_ctrl::membw -> resctrl_ctrl::scalar
>   resctrl_ctrl::cache -> resctrl_ctrl::bitmap
> - Switch subject prefix to be "arm,x86,fs/resctrl: ..." when changing both
>   architectures and resctrl fs code.
> - Switch to MPAM subject prefix of "arm_mpam: resctrl:" when only changing
>   MPAM code.
> 
> Changes deferred/dropped:
> ------------------------
> - Let "scope" form part of the control name.
>   Drop this since the current plan is to have the scope be part of the resource
>   name for new resources.
> - Introduce new per-control "status" that can be "enabled" or "disabled".
>   Drop this. While controls now have an "enabled" or "disabled" state internally,
>   this is not exposed to user space via a file. Instead, the "enabled" or
>   "disabled" state is exposed to user space via the entries appearing in schemata
>   file. This simplifies the subtleties when the hardware self supports both
>   legacy and finer grained interfaces (aka RDT's region-aware).
> - Defer emulated control support for cache controls.
> - Supporting different monitoring scope is not part of this PoC.
>                                                                                 
> Any feedback is appreciated.

Just given a go at running this on a model with MPAM and I can mount resctrl with this
small patch to initialise the emulated_by lists so that list_empty() behaves.

diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
index 21dcbf5764cc..638095649151 100644
--- a/drivers/resctrl/mpam_resctrl.c
+++ b/drivers/resctrl/mpam_resctrl.c
@@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
        case RDT_RESOURCE_L3:
                mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
                mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
+               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
                INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
                __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
                mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
@@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
                r->ctrl_scope = RESCTRL_L3_CACHE;
                mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
                mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
+               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
                INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);

                r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;

I haven't a good look at the code yet but from the list of changes you have above it seems to be
heading in the right direction. Thank you for all the work on this and for always considering MPAM
in the design. I'm out of office for a couple of weeks but I'll give this a better look when I'm back.

One little thing that seems odd is that control_mode shows 'native' as an option even if there are
no emulated controls.
$ cat info/L3/control_mode
[legacy] native

Thanks,

Ben

> 
> Reinette

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre]

Hi Ben,

On 8/5/26 9:59 AM, Ben Horgan wrote:
> Just given a go at running this on a model with MPAM and I can mount resctrl with this
> small patch to initialise the emulated_by lists so that list_empty() behaves.
> 
> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
> index 21dcbf5764cc..638095649151 100644
> --- a/drivers/resctrl/mpam_resctrl.c
> +++ b/drivers/resctrl/mpam_resctrl.c
> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>         case RDT_RESOURCE_L3:
>                 mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
>                 mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>                 INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>                 __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
>                 mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>                 r->ctrl_scope = RESCTRL_L3_CACHE;
>                 mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
>                 mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>                 INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
> 
>                 r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;

Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1

> I haven't a good look at the code yet but from the list of changes you have above it seems to be
> heading in the right direction. Thank you for all the work on this and for always considering MPAM
> in the design. I'm out of office for a couple of weeks but I'll give this a better look when I'm back.
> 
> One little thing that seems odd is that control_mode shows 'native' as an option even if there are
> no emulated controls.
> $ cat info/L3/control_mode
> [legacy] native
> 
Yes, that is odd. Especially for the L3 resource that, on top of not having no emulated controls, does not
even support switching of the control mode. I'll fix this. Thank you very much.

Reinette

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Luck, Tony]

On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote:
> Hi Ben,
> 
> On 8/5/26 9:59 AM, Ben Horgan wrote:
> > Just given a go at running this on a model with MPAM and I can mount resctrl with this
> > small patch to initialise the emulated_by lists so that list_empty() behaves.
> > 
> > diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
> > index 21dcbf5764cc..638095649151 100644
> > --- a/drivers/resctrl/mpam_resctrl.c
> > +++ b/drivers/resctrl/mpam_resctrl.c
> > @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
> >         case RDT_RESOURCE_L3:
> >                 mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
> >                 mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
> > +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
> >                 INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
> >                 __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
> >                 mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
> > @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
> >                 r->ctrl_scope = RESCTRL_L3_CACHE;
> >                 mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
> >                 mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
> > +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
> >                 INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
> > 
> >                 r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;
> 
> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1

Is same needed for x86? I don't see any initialization of the
"r_ctrl.emulated_by" lists in similar initialization functions.

-Tony

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre]

Hi Tony,

On 8/6/26 10:04 AM, Luck, Tony wrote:
> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote:
>> Hi Ben,
>>
>> On 8/5/26 9:59 AM, Ben Horgan wrote:
>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this
>>> small patch to initialise the emulated_by lists so that list_empty() behaves.
>>>
>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
>>> index 21dcbf5764cc..638095649151 100644
>>> --- a/drivers/resctrl/mpam_resctrl.c
>>> +++ b/drivers/resctrl/mpam_resctrl.c
>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>         case RDT_RESOURCE_L3:
>>>                 mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
>>>                 mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>                 INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>                 __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
>>>                 mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>                 r->ctrl_scope = RESCTRL_L3_CACHE;
>>>                 mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
>>>                 mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>                 INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>
>>>                 r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;
>>
>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1
> 
> Is same needed for x86? I don't see any initialization of the
> "r_ctrl.emulated_by" lists in similar initialization functions.
x86 "emulated_by" list initialization should be in both branches.

Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization
(for x86 and MPAM) into commit:
3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")

In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can
be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls")

Reinette

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Moger, Babu]

Hi Reinette,

Thanks for the patches. I am yet to look at all the patches.

On 8/6/2026 12:11 PM, Reinette Chatre wrote:
> Hi Tony,
> 
> On 8/6/26 10:04 AM, Luck, Tony wrote:
>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote:
>>> Hi Ben,
>>>
>>> On 8/5/26 9:59 AM, Ben Horgan wrote:
>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this
>>>> small patch to initialise the emulated_by lists so that list_empty() behaves.
>>>>
>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
>>>> index 21dcbf5764cc..638095649151 100644
>>>> --- a/drivers/resctrl/mpam_resctrl.c
>>>> +++ b/drivers/resctrl/mpam_resctrl.c
>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>          case RDT_RESOURCE_L3:
>>>>                  mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
>>>>                  mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>                  INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>                  __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
>>>>                  mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>                  r->ctrl_scope = RESCTRL_L3_CACHE;
>>>>                  mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
>>>>                  mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>                  INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>
>>>>                  r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;
>>>
>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1
>>
>> Is same needed for x86? I don't see any initialization of the
>> "r_ctrl.emulated_by" lists in similar initialization functions.
> x86 "emulated_by" list initialization should be in both branches.
> 
> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization
> (for x86 and MPAM) into commit:
> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
> 
> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can
> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls")
> 

Looking at the commit:

commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and 
control mode")

Based on the patch description, emulated controls are intended to be 
used only when there is a difference between the native and legacy 
controls. If no such difference exists, both modes should operate 
identically.

For MBA(AMD) and GMBA, there does not appear to be any difference 
between the two modes. Is that understanding correct?

Thanks
Babu

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre]

Hi Babu,

On 8/7/26 7:14 AM, Moger, Babu wrote:
> On 8/6/2026 12:11 PM, Reinette Chatre wrote:
>> On 8/6/26 10:04 AM, Luck, Tony wrote:
>>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote:
>>>> On 8/5/26 9:59 AM, Ben Horgan wrote:
>>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this
>>>>> small patch to initialise the emulated_by lists so that list_empty() behaves.
>>>>>
>>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
>>>>> index 21dcbf5764cc..638095649151 100644
>>>>> --- a/drivers/resctrl/mpam_resctrl.c
>>>>> +++ b/drivers/resctrl/mpam_resctrl.c
>>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>>          case RDT_RESOURCE_L3:
>>>>>                  mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
>>>>>                  mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>>                  INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>>                  __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
>>>>>                  mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
>>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>>                  r->ctrl_scope = RESCTRL_L3_CACHE;
>>>>>                  mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
>>>>>                  mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>>                  INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>>
>>>>>                  r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;
>>>>
>>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1
>>>
>>> Is same needed for x86? I don't see any initialization of the
>>> "r_ctrl.emulated_by" lists in similar initialization functions.
>> x86 "emulated_by" list initialization should be in both branches.
>>
>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization
>> (for x86 and MPAM) into commit:
>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
>>
>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can
>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls")
>>
> 
> Looking at the commit:
> 
> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
> 
> Based on the patch description, emulated controls are intended to be
> used only when there is a difference between the native and legacy
> controls. If no such difference exists, both modes should operate
> identically.

The original AMD MBA enabling did not follow the original percentage based MBA
control so resctrl essentially has *two* "legacy" MB controls today: one for AMD
and one for Intel, MPAM, and RISC-V (planned afaik). That cannot be changed now.
AMD would continue to expose the MB control that is not the percentage based "legacy"
control but actually AMD's native control. In short, yes, on AMD's "MB" control
the "legacy" and "native" control modes should operate identically.
At least now users could use the files in info/MB/schemata/MB/* to learn the
properties of the control.
> For MBA(AMD) and GMBA, there does not appear to be any difference
> between the two modes. Is that understanding correct?
It is not clear to me how resctrl should support AMD's GLBE. Note I am intentionally
not using GMBA since that already makes an assumption on how resctrl will support this.
Do you perhaps have an answer for Ben's question in 
https://lore.kernel.org/lkml/1444b515-752e-4167-87f5-30ace189e05c@arm.com/ ?

GLBE claims to enable users to allocate memory bandwidth at node scope but the memory
is managed at L3. This results in scenarios where, for example, a NUMA node can be
online and used but resctrl cannot expose it for bandwidth allocation when all the CPUs
at that NUMA node scope are offline. GLBE is thus not actually allocating memory bandwidth
at the NUMA node.

Should resctrl prepare for some future where an AMD system may support memory bandwidth
allocation at the NUMA node while also supporting GLBE (memory bandwidth allocation at L3)? 

Reinette

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Chen, Yu C]

Hi Babu,

On 8/7/2026 10:14 PM, Moger, Babu wrote:

[ ... ]

>>> Is same needed for x86? I don't see any initialization of the
>>> "r_ctrl.emulated_by" lists in similar initialization functions.
>> x86 "emulated_by" list initialization should be in both branches.
>>
>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list 
>> initialization
>> (for x86 and MPAM) into commit:
>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
>>
>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list 
>> initialization can
>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated 
>> controls")
>>
> 
> Looking at the commit:
> 
> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and 
> control mode")
> 
> Based on the patch description, emulated controls are intended to be 
> used only when there is a difference between the native and legacy 
> controls. If no such difference exists, both modes should operate 
> identically.
> 
> For MBA(AMD) and GMBA, there does not appear to be any difference 
> between the two modes. Is that understanding correct?
> 

Yes I think so. __rdt_get_mem_config_amd() does
INIT_LIST_HEAD(&hw_ctrl->r_ctrl.emulated_by), so the list is
a valid empty one and resctrl_enabled_ctrl_iter_next() short-circuits
on list_empty() regardless of ctrl_mode, meaning MBA
expose the same single control in either mode.

thanks,
Chenyu

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Moger, Babu]

Hi Chenyu,

On 8/7/2026 10:35 AM, Chen, Yu C wrote:
> Hi Babu,
> 
> On 8/7/2026 10:14 PM, Moger, Babu wrote:
> 
> [ ... ]
> 
>>>> Is same needed for x86? I don't see any initialization of the
>>>> "r_ctrl.emulated_by" lists in similar initialization functions.
>>> x86 "emulated_by" list initialization should be in both branches.
>>>
>>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list 
>>> initialization
>>> (for x86 and MPAM) into commit:
>>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control 
>>> mode")
>>>
>>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list 
>>> initialization can
>>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated 
>>> controls")
>>>
>>
>> Looking at the commit:
>>
>> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and 
>> control mode")
>>
>> Based on the patch description, emulated controls are intended to be 
>> used only when there is a difference between the native and legacy 
>> controls. If no such difference exists, both modes should operate 
>> identically.
>>
>> For MBA(AMD) and GMBA, there does not appear to be any difference 
>> between the two modes. Is that understanding correct?
>>
> 
> Yes I think so. __rdt_get_mem_config_amd() does
> INIT_LIST_HEAD(&hw_ctrl->r_ctrl.emulated_by), so the list is
> a valid empty one and resctrl_enabled_ctrl_iter_next() short-circuits
> on list_empty() regardless of ctrl_mode, meaning MBA
> expose the same single control in either mode.
> 

Got it,
Thank you
Babu

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Moger, Babu]

Hi Reinette/Ben,

On 8/7/2026 10:35 AM, Reinette Chatre wrote:
> Hi Babu,
> 
> On 8/7/26 7:14 AM, Moger, Babu wrote:
>> On 8/6/2026 12:11 PM, Reinette Chatre wrote:
>>> On 8/6/26 10:04 AM, Luck, Tony wrote:
>>>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote:
>>>>> On 8/5/26 9:59 AM, Ben Horgan wrote:
>>>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this
>>>>>> small patch to initialise the emulated_by lists so that list_empty() behaves.
>>>>>>
>>>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
>>>>>> index 21dcbf5764cc..638095649151 100644
>>>>>> --- a/drivers/resctrl/mpam_resctrl.c
>>>>>> +++ b/drivers/resctrl/mpam_resctrl.c
>>>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>>>           case RDT_RESOURCE_L3:
>>>>>>                   mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
>>>>>>                   mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>>>                   INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>>>                   __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
>>>>>>                   mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
>>>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>>>                   r->ctrl_scope = RESCTRL_L3_CACHE;
>>>>>>                   mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
>>>>>>                   mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>>>                   INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>>>
>>>>>>                   r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;
>>>>>
>>>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1
>>>>
>>>> Is same needed for x86? I don't see any initialization of the
>>>> "r_ctrl.emulated_by" lists in similar initialization functions.
>>> x86 "emulated_by" list initialization should be in both branches.
>>>
>>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization
>>> (for x86 and MPAM) into commit:
>>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
>>>
>>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can
>>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls")
>>>
>>
>> Looking at the commit:
>>
>> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
>>
>> Based on the patch description, emulated controls are intended to be
>> used only when there is a difference between the native and legacy
>> controls. If no such difference exists, both modes should operate
>> identically.
> 
> The original AMD MBA enabling did not follow the original percentage based MBA
> control so resctrl essentially has *two* "legacy" MB controls today: one for AMD
> and one for Intel, MPAM, and RISC-V (planned afaik). That cannot be changed now.
> AMD would continue to expose the MB control that is not the percentage based "legacy"
> control but actually AMD's native control. In short, yes, on AMD's "MB" control
> the "legacy" and "native" control modes should operate identically.
> At least now users could use the files in info/MB/schemata/MB/* to learn the
> properties of the control.
>> For MBA(AMD) and GMBA, there does not appear to be any difference
>> between the two modes. Is that understanding correct?
> It is not clear to me how resctrl should support AMD's GLBE. Note I am intentionally
> not using GMBA since that already makes an assumption on how resctrl will support this.
> Do you perhaps have an answer for Ben's question in
> https://lore.kernel.org/lkml/1444b515-752e-4167-87f5-30ace189e05c@arm.com/ ?

Yes. We are already at RFC v2. Let me respond here. Please see my 
response below.

> 
> GLBE claims to enable users to allocate memory bandwidth at node scope but the memory
> is managed at L3. This results in scenarios where, for example, a NUMA node can be
> online and used but resctrl cannot expose it for bandwidth allocation when all the CPUs
> at that NUMA node scope are offline. GLBE is thus not actually allocating memory bandwidth
> at the NUMA node.

Yea. That is correct.

There are a couple of key differences when compared to the pure NUMA scope.

1. In some cases, a NUMA node is treated as the entire system.

  
https://lore.kernel.org/lkml/8f77f498b1c77fa8fd8f5d5687f03ae598068544.1776980182.git.babu.moger@amd.com/

2. When a user updates the settings on a GLBE for a specific node, we 
need to update the MSRs within that node at the L3 scope:

https://lore.kernel.org/lkml/a2a06bd290e68f902be9e7cc3ad35f0a2211b950.1776980182.git.babu.moger@amd.com/


Considering these differences, I think we should probably treat GLBE 
separate from MB_NODE scope. What do you think?


> 
> Should resctrl prepare for some future where an AMD system may support memory bandwidth
> allocation at the NUMA node while also supporting GLBE (memory bandwidth allocation at L3)?

No, I don't believe that's the case. At least, I haven't come across any 
information suggesting that.

Thanks,
Babu

Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept

[Reinette Chatre]

Hi Babu and Ben,

On 8/7/26 11:33 AM, Moger, Babu wrote:
> Hi Reinette/Ben,
> 
> On 8/7/2026 10:35 AM, Reinette Chatre wrote:
>> Hi Babu,
>>
>> On 8/7/26 7:14 AM, Moger, Babu wrote:
>>> On 8/6/2026 12:11 PM, Reinette Chatre wrote:
>>>> On 8/6/26 10:04 AM, Luck, Tony wrote:
>>>>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote:
>>>>>> On 8/5/26 9:59 AM, Ben Horgan wrote:
>>>>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this
>>>>>>> small patch to initialise the emulated_by lists so that list_empty() behaves.
>>>>>>>
>>>>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
>>>>>>> index 21dcbf5764cc..638095649151 100644
>>>>>>> --- a/drivers/resctrl/mpam_resctrl.c
>>>>>>> +++ b/drivers/resctrl/mpam_resctrl.c
>>>>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>>>>           case RDT_RESOURCE_L3:
>>>>>>>                   mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP;
>>>>>>>                   mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>>>>                   INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>>>>                   __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags);
>>>>>>>                   mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd;
>>>>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res)
>>>>>>>                   r->ctrl_scope = RESCTRL_L3_CACHE;
>>>>>>>                   mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR;
>>>>>>>                   mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF;
>>>>>>> +               INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by);
>>>>>>>                   INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains);
>>>>>>>
>>>>>>>                   r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED;
>>>>>>
>>>>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1
>>>>>
>>>>> Is same needed for x86? I don't see any initialization of the
>>>>> "r_ctrl.emulated_by" lists in similar initialization functions.
>>>> x86 "emulated_by" list initialization should be in both branches.
>>>>
>>>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization
>>>> (for x86 and MPAM) into commit:
>>>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
>>>>
>>>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can
>>>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls")
>>>>
>>>
>>> Looking at the commit:
>>>
>>> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode")
>>>
>>> Based on the patch description, emulated controls are intended to be
>>> used only when there is a difference between the native and legacy
>>> controls. If no such difference exists, both modes should operate
>>> identically.
>>
>> The original AMD MBA enabling did not follow the original percentage based MBA
>> control so resctrl essentially has *two* "legacy" MB controls today: one for AMD
>> and one for Intel, MPAM, and RISC-V (planned afaik). That cannot be changed now.
>> AMD would continue to expose the MB control that is not the percentage based "legacy"
>> control but actually AMD's native control. In short, yes, on AMD's "MB" control
>> the "legacy" and "native" control modes should operate identically.
>> At least now users could use the files in info/MB/schemata/MB/* to learn the
>> properties of the control.
>>> For MBA(AMD) and GMBA, there does not appear to be any difference
>>> between the two modes. Is that understanding correct?
>> It is not clear to me how resctrl should support AMD's GLBE. Note I am intentionally
>> not using GMBA since that already makes an assumption on how resctrl will support this.
>> Do you perhaps have an answer for Ben's question in
>> https://lore.kernel.org/lkml/1444b515-752e-4167-87f5-30ace189e05c@arm.com/ ?
> 
> Yes. We are already at RFC v2. Let me respond here. Please see my response below.
> 
>>
>> GLBE claims to enable users to allocate memory bandwidth at node scope but the memory
>> is managed at L3. This results in scenarios where, for example, a NUMA node can be
>> online and used but resctrl cannot expose it for bandwidth allocation when all the CPUs
>> at that NUMA node scope are offline. GLBE is thus not actually allocating memory bandwidth
>> at the NUMA node.
> 
> Yea. That is correct.
> 
> There are a couple of key differences when compared to the pure NUMA scope.
> 
> 1. In some cases, a NUMA node is treated as the entire system.
> 
>  
> https://lore.kernel.org/lkml/8f77f498b1c77fa8fd8f5d5687f03ae598068544.1776980182.git.babu.moger@amd.com/
> 
> 2. When a user updates the settings on a GLBE for a specific node, we need to update the MSRs within that node at the L3 scope:
> 
> https://lore.kernel.org/lkml/a2a06bd290e68f902be9e7cc3ad35f0a2211b950.1776980182.git.babu.moger@amd.com/
> 
> 
> Considering these differences, I think we should probably treat GLBE
> separate from MB_NODE scope. What do you think?

I think so too, but to Ben's point we need to be clear on terminology here. There are two usages
of "scope" to consider:
1) The scope of the *resource* being allocated.
   Here "scope" applies to the resource. For cache there is L2 and L3 that indicates the
   scope of the cache resource. There is also now two different scope to consider for
   memory bandwidth allocation, "L3 scope" for memory bandwidth at egress from L3 and "NUMA/node"
   scope for memory bandwidth at ingress to NUMA node.

2) The scope of the *control* used to allocate the resource.
   So far the scope of the resource has been assumed that to be the same as the scope of the control.
   That is, a resource at particular scope is allocated at that same scope. GLBE taught us this is not
   always the case - memory bandwidth at L3 scope can be allocated at NUMA(with some caveat as you point out)
   scope.

We have gone back and forth on this. Currently upstream resctrl has control scope as a property of
the resource with expectation that the resource and its controls have the same scope.  As a change
to this RFC v1 of this PoC had "scope" as a property of the control to support a control to have
different scope as resource. After deciding to treat MB_NODE as a new resource to support the CPU-less
nodes, RFC v2 changed this back to have scope a property of the resource.

First, it looks to me as though the RFC v1 separating control scope from resource scope needs to
return.

Second, how to name these resources/controls needs to be decided. This is what prompted my question
about future considerations to you because my original proposal in
https://lore.kernel.org/lkml/f5b6cec4-03d8-4a11-884d-d4579dab6b22@intel.com/
was able to convey the resource and allocation scope via info hierarchy but it used the same
name in the schemata file that Ben highlighted could be problematic.

I believe there is agreement that the resource should include the resource scope in its name. This
is currently done for L2 and L3, and planned to be done for MB_NODE (the "MB" resource does not have
"L3" in its name, this cannot be changed now, but "MB" resource is implicitly "L3" scope). The "NODE"
in MB_NODE is thus memory bandwidth allocation at NUMA node scope - "NODE" in MB_NODE is the *resource*
scope.

Considering this I do not think that GLBE should use MB_NODE as you also state above. Since it does
not allocate memory bandwidth *resource* at node scope. 

We also discussed before (https://lore.kernel.org/lkml/c78169bc-e2d6-4583-96ec-09fa6dd6653a@intel.com/ )
of having the control's scope part of the control's name.

We could have a rule of thumb to include both resource scope and control scope in the
control name but only one instance is displayed if they are the same.

For example, what do you think of something like below as an alternate proposal of what I mentioned in 
https://lore.kernel.org/lkml/f5b6cec4-03d8-4a11-884d-d4579dab6b22@intel.com/:

GLBE (booted with NPS < 4):
info/
└── MB/                /* memory bandwidth allocation implicitly/legacy at L3 scope, think of this as "MB_L3" */
    └── schemata/
        ├── MB/          /* control scope = L3 */
        └── MB_L3NODE/   /* control scope = node ("L3" resource scope + "node" control scope) */

GLBE (booted with NPS = 4):
info/
└── MB/                /* memory bandwidth allocation implicitly/legacy at L3 scope, think of this as "MB_L3" */
    └── schemata/
        ├── MB/          /* control scope = L3 */
        └── MB_L3SYSTEM/ /* control scope = system ("L3" resource scope + "system" control scope*/

MPAM MSC:
info/
├── MB/                /* memory bandwidth allocation at L3 scope */
│   └── schemata/
│       └── MB/        /* control scope = L3 */
└── MB_NODE/           /* memory bandwidth allocation at node scope */
    └── schemata/
        └── MB_NODE/   /* control scope = node */

The GLBE controls make it clear that the *same* resource is allocated using two
different controls that have different scope. This creates an implicit dependency
between the two that is not quite captured but having two controls for the same
resource would already give user space some insight that their control values
need to be considered with care.

I'd appreciate your, Ben's, and anybody else's thoughts on this.

>> Should resctrl prepare for some future where an AMD system may support memory bandwidth
>> allocation at the NUMA node while also supporting GLBE (memory bandwidth allocation at L3)?
> 
> No, I don't believe that's the case. At least, I haven't come across any information suggesting that.
Apologies, this was an unreasonable question from my side. We cannot predict the future.
We try to prepare for it with what we know today but new features always seem to come with unique
capabilities.

Reinette