* [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept
@ 2026-08-04 5:26 Reinette Chatre
2026-08-05 16:59 ` Ben Horgan
0 siblings, 1 reply; 11+ messages in thread
From: Reinette Chatre @ 2026-08-04 5:26 UTC (permalink / raw)
To: Tony Luck, Ben Horgan, James Morse, Dave Martin, Babu Moger,
Drew Fustini, Fenghua Yu, Chen Yu
Cc: Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman,
x86@kernel.org, linux-kernel@vger.kernel.org, reinette.chatre
Hi Everybody,
I understand that folks are eager for this work to settle so that features that
depend on it can make progress.
I do have a request if you have a feature that depends on this PoC:
Please do not fork this PoC and keep changes to the PoC code in your fork
to meet the requirements of the feature you are working on. This is resulting
in duplicate (even triplicate) changes to PoC floating around. Instead,
please collaborate on this PoC by responding to the patches that need changes/fixes
to support your feature. If the patches do not exist yet, please participate in
the discussions that plan these changes so the work adding the discussed features
is not duplicated. If you already have the code, please feel free to send as fixups.
Doing so will help this PoC to settle faster and help meet the requirements of the
feature(s) a couple of folks are working on.
RFC v1: https://lore.kernel.org/lkml/aab804b9-e8b5-40ad-a85b-af7033391243@intel.com/
git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1
RFC v2 of this PoC can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v2
The list of changes is long. For an overview of the new interfaces, please see the new
documentation patch:
https://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git/patch/?id=2e2d0140e857eae2ba3e71e8d9c595897649599a
Remaining Opens:
---------------
- MPAM only compile tested.
- Only "SAMPLE" code provided for x86 to demonstrate emulated controls.
- Architecture should take care to restore control value of legacy control when
switching mode from "native" to "legacy". This may be complicated by a
legacy control backed by multiple native controls and thus best handled by
architecture. This adds a complication that resctrl cannot promise that
switching between control modes will not be destructive since a reverse mapping
may not exist.
- When switching control mode to native the top level control info files
stop returning information. This could be considered "work as intended"?
- Add relationships between controls. For example, cannot set "MIN" control
value to be larger than "MAX. Unclear how to do this in resctrl fs when
considering that the relationship may span between enabled and disabled
controls. This is potentially something that needs to be managed by architecture
and thus potentially without a consistent failure to user space.
Changes since RFC v1:
--------------------
There are many changes since RFC v1 since I aimed to address all feedback
(but please see the deferred/dropped list below). If I missed anything it
was not intentional. Please feel free to point it out to me.
User visible changes:
--------------------
- Expose bitmap control properties to user space.
-- Only expose two properties:
- resctrl_ctrl_bitmap::min_cbm_bits exposed via new file "min_bits".
This drops the familiar "cbm" term to not make the control specific
to capacity.
- resctrl_ctrl_bitmap::cbm_len exposed via "max" file
-- While "shareable_bits" is technically a bitmap control property this does
seem an opportunity to deprecate it since the IO alloc support proved
that it does not accurately represent shared allocations, "bit_usage"
is the best for that.
- Add support for control flags with "linear" as first flag of the "scalar"
control and "sparse" the first flag of the "bitmap" control.
Include flags in output of the "type" file.
Suggested by Chenyu: https://lore.kernel.org/lkml/90ae82da-02b2-4058-8c1e-f16df3226d8a@intel.com/
- Add a "default" field to a control using the "reset_val" name suggested
by Drew and return that instead of always using "max" for the scalar control.
The "MIN" control can return its minimum value as default.
Suggested by Ben in https://lore.kernel.org/lkml/29c95b69-e1a4-46b1-ab8b-45c09308b924@arm.com/
Use case from Drew: https://lore.kernel.org/lkml/aiCBratZchVFVhws@gen8/
Drew suggested name as "reset_val" https://lore.kernel.org/lkml/aiOrznnZjTGywMna@thelio>
- Change "resource_schemata" to "schemata".
Fenghua: https://lore.kernel.org/lkml/d0f84cef-58f5-4b84-8171-a0dfe8fa6829@nvidia.com/
- Fix the control alignment. (Chenyu and Tony)
- Drop the "per control" scope. Move the "scope" back to being per-resource and
do not expose a per control "scope" file. This helps to support new resources
like node scoped MBA that cannot emulate the legacy MB control. Also
supports emulated controls by ensuring that all controls associated with
a resource have the same scope.
- Support emulating legacy control.
- Support multiple backing controls for a legacy control.
- Expose new per-resource "control_mode" file that user space can use to
switch between legacy and native controls. Only the enabled controls
are shown to user in schemata file.
- The new "native" control mode is only supported for the MBA resource.
The solution is generic but there are implications for, for example, the
resctrl features that rely on the legacy cache control (cache pseudo-locking
and IO alloc). Defer support for other resource since there are no immediate
plans to have, for example, the legacy cache control backed by something else.
- Architecture differences imply a lot of architecture flexibility in
support for emulating controls.
Architecture can expect resctrl fs to only stage control values in
controls that are enabled. Within architecture there could be rules
that determine what to program based on the control's emulated controls
or it could dynamically program update callbacks when the user changes
the control mode. RFC includes sample code identified by "SAMPLE" for the
latter that demonstrates one way to solve the x86 variant where the hardware
self supports both legacy and finer grained controls.
- The top level legacy control property files stop returning data when
the mode switches to "native".
- Draft of documentation that describes multiple controls and emulated
controls.
- Fix Intel MBA tolerance to be 10 to match the "round-up" implementation.
Changes not user visible:
-------------------------
- Rebase on tip x86/cache with following applied on top:
"x86,fs/resctrl: Improve resctrl quality and consistency"
https://lore.kernel.org/lkml/cover.1782857711.git.reinette.chatre@intel.com/
"x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch"
https://lore.kernel.org/lkml/20260709093111.367851-1-ben.horgan@arm.com/
- Fix initialization of controls list in MPAM. (Ben)
- Dan Carpenter reported an issue with missing unlocks in
rdt_bit_usage_show() that was resolved as part of rebase on top of the
new info_kn_lock()/info_kn_unlock() helpers.
- Drop "mpam,x86,fs/resctrl: Make memory bandwidth delay a resource
property" so that memory bandwidth delay linear flag can be a control flag
instead.
- Separate adding the control to struct msr_param from the change that
makes hardware update functions part of control.
("x86/resctrl: Make control update properties part of the control") to
create new patch that is moved earlier in series to support bandwidth
linear flag property being part of control.
New patch: "x86/resctrl: Include control with information controlling hardware change"
- Rename the controls to match their type to reduce confusion.
Suggested by Chenyu: https://lore.kernel.org/lkml/773353e8-b2a7-4c20-b6fa-195b0b301107@intel.com/
New names:
struct resctrl_membw -> struct resctrl_ctrl_scalar
struct resctrl_cache -> struct resctrl_ctrl_bitmap
resctrl_ctrl::membw -> resctrl_ctrl::scalar
resctrl_ctrl::cache -> resctrl_ctrl::bitmap
- Switch subject prefix to be "arm,x86,fs/resctrl: ..." when changing both
architectures and resctrl fs code.
- Switch to MPAM subject prefix of "arm_mpam: resctrl:" when only changing
MPAM code.
Changes deferred/dropped:
------------------------
- Let "scope" form part of the control name.
Drop this since the current plan is to have the scope be part of the resource
name for new resources.
- Introduce new per-control "status" that can be "enabled" or "disabled".
Drop this. While controls now have an "enabled" or "disabled" state internally,
this is not exposed to user space via a file. Instead, the "enabled" or
"disabled" state is exposed to user space via the entries appearing in schemata
file. This simplifies the subtleties when the hardware self supports both
legacy and finer grained interfaces (aka RDT's region-aware).
- Defer emulated control support for cache controls.
- Supporting different monitoring scope is not part of this PoC.
Any feedback is appreciated.
Reinette
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-04 5:26 [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept Reinette Chatre @ 2026-08-05 16:59 ` Ben Horgan 2026-08-05 23:57 ` Reinette Chatre 0 siblings, 1 reply; 11+ messages in thread From: Ben Horgan @ 2026-08-05 16:59 UTC (permalink / raw) To: Reinette Chatre, Tony Luck, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu Cc: Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Reinette, On 8/4/26 06:26, Reinette Chatre wrote: > Hi Everybody, > > I understand that folks are eager for this work to settle so that features that > depend on it can make progress. > > I do have a request if you have a feature that depends on this PoC: > > Please do not fork this PoC and keep changes to the PoC code in your fork > to meet the requirements of the feature you are working on. This is resulting > in duplicate (even triplicate) changes to PoC floating around. Instead, > please collaborate on this PoC by responding to the patches that need changes/fixes > to support your feature. If the patches do not exist yet, please participate in > the discussions that plan these changes so the work adding the discussed features > is not duplicated. If you already have the code, please feel free to send as fixups. > Doing so will help this PoC to settle faster and help meet the requirements of the > feature(s) a couple of folks are working on. > > RFC v1: https://lore.kernel.org/lkml/aab804b9-e8b5-40ad-a85b-af7033391243@intel.com/ > git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v1 > > RFC v2 of this PoC can be found at: > git://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git branch resctrl/controls_rfc_v2 > > The list of changes is long. For an overview of the new interfaces, please see the new > documentation patch: > https://git.kernel.org/pub/scm/linux/kernel/git/reinette/linux.git/patch/?id=2e2d0140e857eae2ba3e71e8d9c595897649599a > > Remaining Opens: > --------------- > - MPAM only compile tested. > - Only "SAMPLE" code provided for x86 to demonstrate emulated controls. > - Architecture should take care to restore control value of legacy control when > switching mode from "native" to "legacy". This may be complicated by a > legacy control backed by multiple native controls and thus best handled by > architecture. This adds a complication that resctrl cannot promise that > switching between control modes will not be destructive since a reverse mapping > may not exist. > - When switching control mode to native the top level control info files > stop returning information. This could be considered "work as intended"? > - Add relationships between controls. For example, cannot set "MIN" control > value to be larger than "MAX. Unclear how to do this in resctrl fs when > considering that the relationship may span between enabled and disabled > controls. This is potentially something that needs to be managed by architecture > and thus potentially without a consistent failure to user space. > > Changes since RFC v1: > -------------------- > There are many changes since RFC v1 since I aimed to address all feedback > (but please see the deferred/dropped list below). If I missed anything it > was not intentional. Please feel free to point it out to me. > > User visible changes: > -------------------- > - Expose bitmap control properties to user space. > -- Only expose two properties: > - resctrl_ctrl_bitmap::min_cbm_bits exposed via new file "min_bits". > This drops the familiar "cbm" term to not make the control specific > to capacity. > - resctrl_ctrl_bitmap::cbm_len exposed via "max" file > -- While "shareable_bits" is technically a bitmap control property this does > seem an opportunity to deprecate it since the IO alloc support proved > that it does not accurately represent shared allocations, "bit_usage" > is the best for that. > - Add support for control flags with "linear" as first flag of the "scalar" > control and "sparse" the first flag of the "bitmap" control. > Include flags in output of the "type" file. > Suggested by Chenyu: https://lore.kernel.org/lkml/90ae82da-02b2-4058-8c1e-f16df3226d8a@intel.com/ > - Add a "default" field to a control using the "reset_val" name suggested > by Drew and return that instead of always using "max" for the scalar control. > The "MIN" control can return its minimum value as default. > Suggested by Ben in https://lore.kernel.org/lkml/29c95b69-e1a4-46b1-ab8b-45c09308b924@arm.com/ > Use case from Drew: https://lore.kernel.org/lkml/aiCBratZchVFVhws@gen8/ > Drew suggested name as "reset_val" https://lore.kernel.org/lkml/aiOrznnZjTGywMna@thelio> > - Change "resource_schemata" to "schemata". > Fenghua: https://lore.kernel.org/lkml/d0f84cef-58f5-4b84-8171-a0dfe8fa6829@nvidia.com/ > - Fix the control alignment. (Chenyu and Tony) > - Drop the "per control" scope. Move the "scope" back to being per-resource and > do not expose a per control "scope" file. This helps to support new resources > like node scoped MBA that cannot emulate the legacy MB control. Also > supports emulated controls by ensuring that all controls associated with > a resource have the same scope. > - Support emulating legacy control. > - Support multiple backing controls for a legacy control. > - Expose new per-resource "control_mode" file that user space can use to > switch between legacy and native controls. Only the enabled controls > are shown to user in schemata file. > - The new "native" control mode is only supported for the MBA resource. > The solution is generic but there are implications for, for example, the > resctrl features that rely on the legacy cache control (cache pseudo-locking > and IO alloc). Defer support for other resource since there are no immediate > plans to have, for example, the legacy cache control backed by something else. > - Architecture differences imply a lot of architecture flexibility in > support for emulating controls. > Architecture can expect resctrl fs to only stage control values in > controls that are enabled. Within architecture there could be rules > that determine what to program based on the control's emulated controls > or it could dynamically program update callbacks when the user changes > the control mode. RFC includes sample code identified by "SAMPLE" for the > latter that demonstrates one way to solve the x86 variant where the hardware > self supports both legacy and finer grained controls. > - The top level legacy control property files stop returning data when > the mode switches to "native". > - Draft of documentation that describes multiple controls and emulated > controls. > - Fix Intel MBA tolerance to be 10 to match the "round-up" implementation. > > Changes not user visible: > ------------------------- > - Rebase on tip x86/cache with following applied on top: > "x86,fs/resctrl: Improve resctrl quality and consistency" > https://lore.kernel.org/lkml/cover.1782857711.git.reinette.chatre@intel.com/ > "x86,fs/resctrl,arm_mpam: Factor MBA parse-time conversion to be per-arch" > https://lore.kernel.org/lkml/20260709093111.367851-1-ben.horgan@arm.com/ > - Fix initialization of controls list in MPAM. (Ben) > - Dan Carpenter reported an issue with missing unlocks in > rdt_bit_usage_show() that was resolved as part of rebase on top of the > new info_kn_lock()/info_kn_unlock() helpers. > - Drop "mpam,x86,fs/resctrl: Make memory bandwidth delay a resource > property" so that memory bandwidth delay linear flag can be a control flag > instead. > - Separate adding the control to struct msr_param from the change that > makes hardware update functions part of control. > ("x86/resctrl: Make control update properties part of the control") to > create new patch that is moved earlier in series to support bandwidth > linear flag property being part of control. > New patch: "x86/resctrl: Include control with information controlling hardware change" > - Rename the controls to match their type to reduce confusion. > Suggested by Chenyu: https://lore.kernel.org/lkml/773353e8-b2a7-4c20-b6fa-195b0b301107@intel.com/ > New names: > struct resctrl_membw -> struct resctrl_ctrl_scalar > struct resctrl_cache -> struct resctrl_ctrl_bitmap > resctrl_ctrl::membw -> resctrl_ctrl::scalar > resctrl_ctrl::cache -> resctrl_ctrl::bitmap > - Switch subject prefix to be "arm,x86,fs/resctrl: ..." when changing both > architectures and resctrl fs code. > - Switch to MPAM subject prefix of "arm_mpam: resctrl:" when only changing > MPAM code. > > Changes deferred/dropped: > ------------------------ > - Let "scope" form part of the control name. > Drop this since the current plan is to have the scope be part of the resource > name for new resources. > - Introduce new per-control "status" that can be "enabled" or "disabled". > Drop this. While controls now have an "enabled" or "disabled" state internally, > this is not exposed to user space via a file. Instead, the "enabled" or > "disabled" state is exposed to user space via the entries appearing in schemata > file. This simplifies the subtleties when the hardware self supports both > legacy and finer grained interfaces (aka RDT's region-aware). > - Defer emulated control support for cache controls. > - Supporting different monitoring scope is not part of this PoC. > > Any feedback is appreciated. Just given a go at running this on a model with MPAM and I can mount resctrl with this small patch to initialise the emulated_by lists so that list_empty() behaves. diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c index 21dcbf5764cc..638095649151 100644 --- a/drivers/resctrl/mpam_resctrl.c +++ b/drivers/resctrl/mpam_resctrl.c @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) case RDT_RESOURCE_L3: mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) r->ctrl_scope = RESCTRL_L3_CACHE; mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; I haven't a good look at the code yet but from the list of changes you have above it seems to be heading in the right direction. Thank you for all the work on this and for always considering MPAM in the design. I'm out of office for a couple of weeks but I'll give this a better look when I'm back. One little thing that seems odd is that control_mode shows 'native' as an option even if there are no emulated controls. $ cat info/L3/control_mode [legacy] native Thanks, Ben > > Reinette ^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-05 16:59 ` Ben Horgan @ 2026-08-05 23:57 ` Reinette Chatre 2026-08-06 17:04 ` Luck, Tony 0 siblings, 1 reply; 11+ messages in thread From: Reinette Chatre @ 2026-08-05 23:57 UTC (permalink / raw) To: Ben Horgan, Tony Luck, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu Cc: Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Ben, On 8/5/26 9:59 AM, Ben Horgan wrote: > Just given a go at running this on a model with MPAM and I can mount resctrl with this > small patch to initialise the emulated_by lists so that list_empty() behaves. > > diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c > index 21dcbf5764cc..638095649151 100644 > --- a/drivers/resctrl/mpam_resctrl.c > +++ b/drivers/resctrl/mpam_resctrl.c > @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) > case RDT_RESOURCE_L3: > mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; > mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; > + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); > INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); > __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); > mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; > @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) > r->ctrl_scope = RESCTRL_L3_CACHE; > mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; > mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; > + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); > INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); > > r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 > I haven't a good look at the code yet but from the list of changes you have above it seems to be > heading in the right direction. Thank you for all the work on this and for always considering MPAM > in the design. I'm out of office for a couple of weeks but I'll give this a better look when I'm back. > > One little thing that seems odd is that control_mode shows 'native' as an option even if there are > no emulated controls. > $ cat info/L3/control_mode > [legacy] native > Yes, that is odd. Especially for the L3 resource that, on top of not having no emulated controls, does not even support switching of the control mode. I'll fix this. Thank you very much. Reinette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-05 23:57 ` Reinette Chatre @ 2026-08-06 17:04 ` Luck, Tony 2026-08-06 17:11 ` Reinette Chatre 0 siblings, 1 reply; 11+ messages in thread From: Luck, Tony @ 2026-08-06 17:04 UTC (permalink / raw) To: Reinette Chatre Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote: > Hi Ben, > > On 8/5/26 9:59 AM, Ben Horgan wrote: > > Just given a go at running this on a model with MPAM and I can mount resctrl with this > > small patch to initialise the emulated_by lists so that list_empty() behaves. > > > > diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c > > index 21dcbf5764cc..638095649151 100644 > > --- a/drivers/resctrl/mpam_resctrl.c > > +++ b/drivers/resctrl/mpam_resctrl.c > > @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) > > case RDT_RESOURCE_L3: > > mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; > > mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; > > + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); > > INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); > > __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); > > mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; > > @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) > > r->ctrl_scope = RESCTRL_L3_CACHE; > > mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; > > mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; > > + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); > > INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); > > > > r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; > > Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 Is same needed for x86? I don't see any initialization of the "r_ctrl.emulated_by" lists in similar initialization functions. -Tony ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-06 17:04 ` Luck, Tony @ 2026-08-06 17:11 ` Reinette Chatre 2026-08-07 14:14 ` Moger, Babu 0 siblings, 1 reply; 11+ messages in thread From: Reinette Chatre @ 2026-08-06 17:11 UTC (permalink / raw) To: Luck, Tony Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Tony, On 8/6/26 10:04 AM, Luck, Tony wrote: > On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote: >> Hi Ben, >> >> On 8/5/26 9:59 AM, Ben Horgan wrote: >>> Just given a go at running this on a model with MPAM and I can mount resctrl with this >>> small patch to initialise the emulated_by lists so that list_empty() behaves. >>> >>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c >>> index 21dcbf5764cc..638095649151 100644 >>> --- a/drivers/resctrl/mpam_resctrl.c >>> +++ b/drivers/resctrl/mpam_resctrl.c >>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>> case RDT_RESOURCE_L3: >>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; >>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>> __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); >>> mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; >>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>> r->ctrl_scope = RESCTRL_L3_CACHE; >>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; >>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>> >>> r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; >> >> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 > > Is same needed for x86? I don't see any initialization of the > "r_ctrl.emulated_by" lists in similar initialization functions. x86 "emulated_by" list initialization should be in both branches. Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization (for x86 and MPAM) into commit: 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls") Reinette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-06 17:11 ` Reinette Chatre @ 2026-08-07 14:14 ` Moger, Babu 2026-08-07 15:35 ` Reinette Chatre 2026-08-07 15:35 ` Chen, Yu C 0 siblings, 2 replies; 11+ messages in thread From: Moger, Babu @ 2026-08-07 14:14 UTC (permalink / raw) To: Reinette Chatre, Luck, Tony Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Reinette, Thanks for the patches. I am yet to look at all the patches. On 8/6/2026 12:11 PM, Reinette Chatre wrote: > Hi Tony, > > On 8/6/26 10:04 AM, Luck, Tony wrote: >> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote: >>> Hi Ben, >>> >>> On 8/5/26 9:59 AM, Ben Horgan wrote: >>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this >>>> small patch to initialise the emulated_by lists so that list_empty() behaves. >>>> >>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c >>>> index 21dcbf5764cc..638095649151 100644 >>>> --- a/drivers/resctrl/mpam_resctrl.c >>>> +++ b/drivers/resctrl/mpam_resctrl.c >>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>> case RDT_RESOURCE_L3: >>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; >>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>> __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); >>>> mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; >>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>> r->ctrl_scope = RESCTRL_L3_CACHE; >>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; >>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>> >>>> r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; >>> >>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 >> >> Is same needed for x86? I don't see any initialization of the >> "r_ctrl.emulated_by" lists in similar initialization functions. > x86 "emulated_by" list initialization should be in both branches. > > Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization > (for x86 and MPAM) into commit: > 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") > > In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can > be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls") > Looking at the commit: commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") Based on the patch description, emulated controls are intended to be used only when there is a difference between the native and legacy controls. If no such difference exists, both modes should operate identically. For MBA(AMD) and GMBA, there does not appear to be any difference between the two modes. Is that understanding correct? Thanks Babu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-07 14:14 ` Moger, Babu @ 2026-08-07 15:35 ` Reinette Chatre 2026-08-07 18:33 ` Moger, Babu 2026-08-07 15:35 ` Chen, Yu C 1 sibling, 1 reply; 11+ messages in thread From: Reinette Chatre @ 2026-08-07 15:35 UTC (permalink / raw) To: Moger, Babu, Luck, Tony Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Babu, On 8/7/26 7:14 AM, Moger, Babu wrote: > On 8/6/2026 12:11 PM, Reinette Chatre wrote: >> On 8/6/26 10:04 AM, Luck, Tony wrote: >>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote: >>>> On 8/5/26 9:59 AM, Ben Horgan wrote: >>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this >>>>> small patch to initialise the emulated_by lists so that list_empty() behaves. >>>>> >>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c >>>>> index 21dcbf5764cc..638095649151 100644 >>>>> --- a/drivers/resctrl/mpam_resctrl.c >>>>> +++ b/drivers/resctrl/mpam_resctrl.c >>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>>> case RDT_RESOURCE_L3: >>>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; >>>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>>> __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); >>>>> mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; >>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>>> r->ctrl_scope = RESCTRL_L3_CACHE; >>>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; >>>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>>> >>>>> r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; >>>> >>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 >>> >>> Is same needed for x86? I don't see any initialization of the >>> "r_ctrl.emulated_by" lists in similar initialization functions. >> x86 "emulated_by" list initialization should be in both branches. >> >> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization >> (for x86 and MPAM) into commit: >> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") >> >> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can >> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls") >> > > Looking at the commit: > > commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") > > Based on the patch description, emulated controls are intended to be > used only when there is a difference between the native and legacy > controls. If no such difference exists, both modes should operate > identically. The original AMD MBA enabling did not follow the original percentage based MBA control so resctrl essentially has *two* "legacy" MB controls today: one for AMD and one for Intel, MPAM, and RISC-V (planned afaik). That cannot be changed now. AMD would continue to expose the MB control that is not the percentage based "legacy" control but actually AMD's native control. In short, yes, on AMD's "MB" control the "legacy" and "native" control modes should operate identically. At least now users could use the files in info/MB/schemata/MB/* to learn the properties of the control. > For MBA(AMD) and GMBA, there does not appear to be any difference > between the two modes. Is that understanding correct? It is not clear to me how resctrl should support AMD's GLBE. Note I am intentionally not using GMBA since that already makes an assumption on how resctrl will support this. Do you perhaps have an answer for Ben's question in https://lore.kernel.org/lkml/1444b515-752e-4167-87f5-30ace189e05c@arm.com/ ? GLBE claims to enable users to allocate memory bandwidth at node scope but the memory is managed at L3. This results in scenarios where, for example, a NUMA node can be online and used but resctrl cannot expose it for bandwidth allocation when all the CPUs at that NUMA node scope are offline. GLBE is thus not actually allocating memory bandwidth at the NUMA node. Should resctrl prepare for some future where an AMD system may support memory bandwidth allocation at the NUMA node while also supporting GLBE (memory bandwidth allocation at L3)? Reinette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-07 15:35 ` Reinette Chatre @ 2026-08-07 18:33 ` Moger, Babu 2026-08-07 22:53 ` Reinette Chatre 0 siblings, 1 reply; 11+ messages in thread From: Moger, Babu @ 2026-08-07 18:33 UTC (permalink / raw) To: Reinette Chatre, Luck, Tony Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Reinette/Ben, On 8/7/2026 10:35 AM, Reinette Chatre wrote: > Hi Babu, > > On 8/7/26 7:14 AM, Moger, Babu wrote: >> On 8/6/2026 12:11 PM, Reinette Chatre wrote: >>> On 8/6/26 10:04 AM, Luck, Tony wrote: >>>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote: >>>>> On 8/5/26 9:59 AM, Ben Horgan wrote: >>>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this >>>>>> small patch to initialise the emulated_by lists so that list_empty() behaves. >>>>>> >>>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c >>>>>> index 21dcbf5764cc..638095649151 100644 >>>>>> --- a/drivers/resctrl/mpam_resctrl.c >>>>>> +++ b/drivers/resctrl/mpam_resctrl.c >>>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>>>> case RDT_RESOURCE_L3: >>>>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; >>>>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>>>> __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); >>>>>> mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; >>>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>>>> r->ctrl_scope = RESCTRL_L3_CACHE; >>>>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; >>>>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>>>> >>>>>> r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; >>>>> >>>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 >>>> >>>> Is same needed for x86? I don't see any initialization of the >>>> "r_ctrl.emulated_by" lists in similar initialization functions. >>> x86 "emulated_by" list initialization should be in both branches. >>> >>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization >>> (for x86 and MPAM) into commit: >>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") >>> >>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can >>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls") >>> >> >> Looking at the commit: >> >> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") >> >> Based on the patch description, emulated controls are intended to be >> used only when there is a difference between the native and legacy >> controls. If no such difference exists, both modes should operate >> identically. > > The original AMD MBA enabling did not follow the original percentage based MBA > control so resctrl essentially has *two* "legacy" MB controls today: one for AMD > and one for Intel, MPAM, and RISC-V (planned afaik). That cannot be changed now. > AMD would continue to expose the MB control that is not the percentage based "legacy" > control but actually AMD's native control. In short, yes, on AMD's "MB" control > the "legacy" and "native" control modes should operate identically. > At least now users could use the files in info/MB/schemata/MB/* to learn the > properties of the control. >> For MBA(AMD) and GMBA, there does not appear to be any difference >> between the two modes. Is that understanding correct? > It is not clear to me how resctrl should support AMD's GLBE. Note I am intentionally > not using GMBA since that already makes an assumption on how resctrl will support this. > Do you perhaps have an answer for Ben's question in > https://lore.kernel.org/lkml/1444b515-752e-4167-87f5-30ace189e05c@arm.com/ ? Yes. We are already at RFC v2. Let me respond here. Please see my response below. > > GLBE claims to enable users to allocate memory bandwidth at node scope but the memory > is managed at L3. This results in scenarios where, for example, a NUMA node can be > online and used but resctrl cannot expose it for bandwidth allocation when all the CPUs > at that NUMA node scope are offline. GLBE is thus not actually allocating memory bandwidth > at the NUMA node. Yea. That is correct. There are a couple of key differences when compared to the pure NUMA scope. 1. In some cases, a NUMA node is treated as the entire system. https://lore.kernel.org/lkml/8f77f498b1c77fa8fd8f5d5687f03ae598068544.1776980182.git.babu.moger@amd.com/ 2. When a user updates the settings on a GLBE for a specific node, we need to update the MSRs within that node at the L3 scope: https://lore.kernel.org/lkml/a2a06bd290e68f902be9e7cc3ad35f0a2211b950.1776980182.git.babu.moger@amd.com/ Considering these differences, I think we should probably treat GLBE separate from MB_NODE scope. What do you think? > > Should resctrl prepare for some future where an AMD system may support memory bandwidth > allocation at the NUMA node while also supporting GLBE (memory bandwidth allocation at L3)? No, I don't believe that's the case. At least, I haven't come across any information suggesting that. Thanks, Babu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-07 18:33 ` Moger, Babu @ 2026-08-07 22:53 ` Reinette Chatre 0 siblings, 0 replies; 11+ messages in thread From: Reinette Chatre @ 2026-08-07 22:53 UTC (permalink / raw) To: Moger, Babu, Luck, Tony Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Chen Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org Hi Babu and Ben, On 8/7/26 11:33 AM, Moger, Babu wrote: > Hi Reinette/Ben, > > On 8/7/2026 10:35 AM, Reinette Chatre wrote: >> Hi Babu, >> >> On 8/7/26 7:14 AM, Moger, Babu wrote: >>> On 8/6/2026 12:11 PM, Reinette Chatre wrote: >>>> On 8/6/26 10:04 AM, Luck, Tony wrote: >>>>> On Wed, Aug 05, 2026 at 04:57:04PM -0700, Reinette Chatre wrote: >>>>>> On 8/5/26 9:59 AM, Ben Horgan wrote: >>>>>>> Just given a go at running this on a model with MPAM and I can mount resctrl with this >>>>>>> small patch to initialise the emulated_by lists so that list_empty() behaves. >>>>>>> >>>>>>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c >>>>>>> index 21dcbf5764cc..638095649151 100644 >>>>>>> --- a/drivers/resctrl/mpam_resctrl.c >>>>>>> +++ b/drivers/resctrl/mpam_resctrl.c >>>>>>> @@ -1018,6 +1018,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>>>>> case RDT_RESOURCE_L3: >>>>>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_BITMAP; >>>>>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>>>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>>>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>>>>> __set_bit(RESCTRL_BITMAP_FLAG_SPARSE, mpam_ctrl->r_ctrl.bitmap.flags); >>>>>>> mpam_ctrl->r_ctrl.bitmap.cbm_len = class->props.cpbm_wd; >>>>>>> @@ -1048,6 +1049,7 @@ static int mpam_resctrl_control_init(struct mpam_resctrl_res *res) >>>>>>> r->ctrl_scope = RESCTRL_L3_CACHE; >>>>>>> mpam_ctrl->r_ctrl.type = RESCTRL_CTRL_SCALAR; >>>>>>> mpam_ctrl->r_ctrl.name = RESCTRL_CTRL_NAME_DEF; >>>>>>> + INIT_LIST_HEAD(&mpam_ctrl->r_ctrl.emulated_by); >>>>>>> INIT_LIST_HEAD_RCU(&mpam_ctrl->r_ctrl.domains); >>>>>>> >>>>>>> r->bw_throttle_mode = THREAD_THROTTLE_UNDEFINED; >>>>>> >>>>>> Thank you for this. Added this and it is now available in branch resctrl/controls_rfc_v2.1 >>>>> >>>>> Is same needed for x86? I don't see any initialization of the >>>>> "r_ctrl.emulated_by" lists in similar initialization functions. >>>> x86 "emulated_by" list initialization should be in both branches. >>>> >>>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list initialization >>>> (for x86 and MPAM) into commit: >>>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") >>>> >>>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list initialization can >>>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated controls") >>>> >>> >>> Looking at the commit: >>> >>> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") >>> >>> Based on the patch description, emulated controls are intended to be >>> used only when there is a difference between the native and legacy >>> controls. If no such difference exists, both modes should operate >>> identically. >> >> The original AMD MBA enabling did not follow the original percentage based MBA >> control so resctrl essentially has *two* "legacy" MB controls today: one for AMD >> and one for Intel, MPAM, and RISC-V (planned afaik). That cannot be changed now. >> AMD would continue to expose the MB control that is not the percentage based "legacy" >> control but actually AMD's native control. In short, yes, on AMD's "MB" control >> the "legacy" and "native" control modes should operate identically. >> At least now users could use the files in info/MB/schemata/MB/* to learn the >> properties of the control. >>> For MBA(AMD) and GMBA, there does not appear to be any difference >>> between the two modes. Is that understanding correct? >> It is not clear to me how resctrl should support AMD's GLBE. Note I am intentionally >> not using GMBA since that already makes an assumption on how resctrl will support this. >> Do you perhaps have an answer for Ben's question in >> https://lore.kernel.org/lkml/1444b515-752e-4167-87f5-30ace189e05c@arm.com/ ? > > Yes. We are already at RFC v2. Let me respond here. Please see my response below. > >> >> GLBE claims to enable users to allocate memory bandwidth at node scope but the memory >> is managed at L3. This results in scenarios where, for example, a NUMA node can be >> online and used but resctrl cannot expose it for bandwidth allocation when all the CPUs >> at that NUMA node scope are offline. GLBE is thus not actually allocating memory bandwidth >> at the NUMA node. > > Yea. That is correct. > > There are a couple of key differences when compared to the pure NUMA scope. > > 1. In some cases, a NUMA node is treated as the entire system. > > > https://lore.kernel.org/lkml/8f77f498b1c77fa8fd8f5d5687f03ae598068544.1776980182.git.babu.moger@amd.com/ > > 2. When a user updates the settings on a GLBE for a specific node, we need to update the MSRs within that node at the L3 scope: > > https://lore.kernel.org/lkml/a2a06bd290e68f902be9e7cc3ad35f0a2211b950.1776980182.git.babu.moger@amd.com/ > > > Considering these differences, I think we should probably treat GLBE > separate from MB_NODE scope. What do you think? I think so too, but to Ben's point we need to be clear on terminology here. There are two usages of "scope" to consider: 1) The scope of the *resource* being allocated. Here "scope" applies to the resource. For cache there is L2 and L3 that indicates the scope of the cache resource. There is also now two different scope to consider for memory bandwidth allocation, "L3 scope" for memory bandwidth at egress from L3 and "NUMA/node" scope for memory bandwidth at ingress to NUMA node. 2) The scope of the *control* used to allocate the resource. So far the scope of the resource has been assumed that to be the same as the scope of the control. That is, a resource at particular scope is allocated at that same scope. GLBE taught us this is not always the case - memory bandwidth at L3 scope can be allocated at NUMA(with some caveat as you point out) scope. We have gone back and forth on this. Currently upstream resctrl has control scope as a property of the resource with expectation that the resource and its controls have the same scope. As a change to this RFC v1 of this PoC had "scope" as a property of the control to support a control to have different scope as resource. After deciding to treat MB_NODE as a new resource to support the CPU-less nodes, RFC v2 changed this back to have scope a property of the resource. First, it looks to me as though the RFC v1 separating control scope from resource scope needs to return. Second, how to name these resources/controls needs to be decided. This is what prompted my question about future considerations to you because my original proposal in https://lore.kernel.org/lkml/f5b6cec4-03d8-4a11-884d-d4579dab6b22@intel.com/ was able to convey the resource and allocation scope via info hierarchy but it used the same name in the schemata file that Ben highlighted could be problematic. I believe there is agreement that the resource should include the resource scope in its name. This is currently done for L2 and L3, and planned to be done for MB_NODE (the "MB" resource does not have "L3" in its name, this cannot be changed now, but "MB" resource is implicitly "L3" scope). The "NODE" in MB_NODE is thus memory bandwidth allocation at NUMA node scope - "NODE" in MB_NODE is the *resource* scope. Considering this I do not think that GLBE should use MB_NODE as you also state above. Since it does not allocate memory bandwidth *resource* at node scope. We also discussed before (https://lore.kernel.org/lkml/c78169bc-e2d6-4583-96ec-09fa6dd6653a@intel.com/ ) of having the control's scope part of the control's name. We could have a rule of thumb to include both resource scope and control scope in the control name but only one instance is displayed if they are the same. For example, what do you think of something like below as an alternate proposal of what I mentioned in https://lore.kernel.org/lkml/f5b6cec4-03d8-4a11-884d-d4579dab6b22@intel.com/: GLBE (booted with NPS < 4): info/ └── MB/ /* memory bandwidth allocation implicitly/legacy at L3 scope, think of this as "MB_L3" */ └── schemata/ ├── MB/ /* control scope = L3 */ └── MB_L3NODE/ /* control scope = node ("L3" resource scope + "node" control scope) */ GLBE (booted with NPS = 4): info/ └── MB/ /* memory bandwidth allocation implicitly/legacy at L3 scope, think of this as "MB_L3" */ └── schemata/ ├── MB/ /* control scope = L3 */ └── MB_L3SYSTEM/ /* control scope = system ("L3" resource scope + "system" control scope*/ MPAM MSC: info/ ├── MB/ /* memory bandwidth allocation at L3 scope */ │ └── schemata/ │ └── MB/ /* control scope = L3 */ └── MB_NODE/ /* memory bandwidth allocation at node scope */ └── schemata/ └── MB_NODE/ /* control scope = node */ The GLBE controls make it clear that the *same* resource is allocated using two different controls that have different scope. This creates an implicit dependency between the two that is not quite captured but having two controls for the same resource would already give user space some insight that their control values need to be considered with care. I'd appreciate your, Ben's, and anybody else's thoughts on this. >> Should resctrl prepare for some future where an AMD system may support memory bandwidth >> allocation at the NUMA node while also supporting GLBE (memory bandwidth allocation at L3)? > > No, I don't believe that's the case. At least, I haven't come across any information suggesting that. Apologies, this was an unreasonable question from my side. We cannot predict the future. We try to prepare for it with what we know today but new features always seem to come with unique capabilities. Reinette ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-07 14:14 ` Moger, Babu 2026-08-07 15:35 ` Reinette Chatre @ 2026-08-07 15:35 ` Chen, Yu C 2026-08-07 17:57 ` Moger, Babu 1 sibling, 1 reply; 11+ messages in thread From: Chen, Yu C @ 2026-08-07 15:35 UTC (permalink / raw) To: Moger, Babu Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org, Reinette Chatre, Luck, Tony, chen.yu@linux.dev Hi Babu, On 8/7/2026 10:14 PM, Moger, Babu wrote: [ ... ] >>> Is same needed for x86? I don't see any initialization of the >>> "r_ctrl.emulated_by" lists in similar initialization functions. >> x86 "emulated_by" list initialization should be in both branches. >> >> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list >> initialization >> (for x86 and MPAM) into commit: >> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control mode") >> >> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list >> initialization can >> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated >> controls") >> > > Looking at the commit: > > commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and > control mode") > > Based on the patch description, emulated controls are intended to be > used only when there is a difference between the native and legacy > controls. If no such difference exists, both modes should operate > identically. > > For MBA(AMD) and GMBA, there does not appear to be any difference > between the two modes. Is that understanding correct? > Yes I think so. __rdt_get_mem_config_amd() does INIT_LIST_HEAD(&hw_ctrl->r_ctrl.emulated_by), so the list is a valid empty one and resctrl_enabled_ctrl_iter_next() short-circuits on list_empty() regardless of ctrl_mode, meaning MBA expose the same single control in either mode. thanks, Chenyu ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept 2026-08-07 15:35 ` Chen, Yu C @ 2026-08-07 17:57 ` Moger, Babu 0 siblings, 0 replies; 11+ messages in thread From: Moger, Babu @ 2026-08-07 17:57 UTC (permalink / raw) To: Chen, Yu C Cc: Ben Horgan, James Morse, Dave Martin, Babu Moger, Drew Fustini, Fenghua Yu, Borislav Petkov, Thomas Gleixner, Dave Hansen, Peter Newman, x86@kernel.org, linux-kernel@vger.kernel.org, Reinette Chatre, Luck, Tony, chen.yu@linux.dev Hi Chenyu, On 8/7/2026 10:35 AM, Chen, Yu C wrote: > Hi Babu, > > On 8/7/2026 10:14 PM, Moger, Babu wrote: > > [ ... ] > >>>> Is same needed for x86? I don't see any initialization of the >>>> "r_ctrl.emulated_by" lists in similar initialization functions. >>> x86 "emulated_by" list initialization should be in both branches. >>> >>> Branch resctrl/controls_rfc_v2.1 combined all "emulated_by" list >>> initialization >>> (for x86 and MPAM) into commit: >>> 3407e523c988 ("fs/resctrl: Introduce emulated controls and control >>> mode") >>> >>> In the original resctrl/controls_rfc_v2 the x86 "emulated_by" list >>> initialization can >>> be found in commit fbed64f80515 ("x86/resctrl: SAMPLE: Emulated >>> controls") >>> >> >> Looking at the commit: >> >> commit 3407e523c988 ("fs/resctrl: Introduce emulated controls and >> control mode") >> >> Based on the patch description, emulated controls are intended to be >> used only when there is a difference between the native and legacy >> controls. If no such difference exists, both modes should operate >> identically. >> >> For MBA(AMD) and GMBA, there does not appear to be any difference >> between the two modes. Is that understanding correct? >> > > Yes I think so. __rdt_get_mem_config_amd() does > INIT_LIST_HEAD(&hw_ctrl->r_ctrl.emulated_by), so the list is > a valid empty one and resctrl_enabled_ctrl_iter_next() short-circuits > on list_empty() regardless of ctrl_mode, meaning MBA > expose the same single control in either mode. > Got it, Thank you Babu ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-08-07 22:53 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-08-04 5:26 [RFC v2] arm,x86,fs/resctrl: Generic schema description Proof of Concept Reinette Chatre 2026-08-05 16:59 ` Ben Horgan 2026-08-05 23:57 ` Reinette Chatre 2026-08-06 17:04 ` Luck, Tony 2026-08-06 17:11 ` Reinette Chatre 2026-08-07 14:14 ` Moger, Babu 2026-08-07 15:35 ` Reinette Chatre 2026-08-07 18:33 ` Moger, Babu 2026-08-07 22:53 ` Reinette Chatre 2026-08-07 15:35 ` Chen, Yu C 2026-08-07 17:57 ` Moger, Babu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox