Re: [PATCH v2 0/8] x86/resctrl: Support for AMD Global (Slow) Memory Bandwidth Allocation

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 4/30/26 4:04 PM, Moger, Babu wrote:
> Hi Reinette,
> 
> On 4/29/2026 5:34 PM, Reinette Chatre wrote:
>> Hi Babu,
>>
>> On 4/23/26 6:41 PM, Babu Moger wrote:
>>>
>>> This series adds resctrl support for two new AMD memory-bandwidth
>>> allocation features:
>>>
>>>    - GMBA  - Global Memory Bandwidth Allocation (hardware name: GLBE).
>>>              Bounds DRAM bandwidth for groups of threads that span
>>>              multiple L3 QoS domains, rather than being per-L3 like MBA.
>>>
>>>    - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name:
>>>              GLSBE). The CXL.memory / slow-memory counterpart of GMBA,
>>>              analogous to how SMBA relates to MBA.
>>>
>>> Both features share a new "NPS-node" control domain: a set of QoS (L3)
>>> domains grouped together and aligned to the system's NPS (Nodes Per
>>> Socket) BIOS configuration. Although the control domain is NPS-scoped,
>>> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600,
>>> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming
>>> a single control domain therefore requires writing the MSR on one CPU
>>> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8
>>> and 3/8 introduce that infrastructure so the new resources can reuse
>>> it.
>>>
>>> The features are documented in:
>>>
>>>    AMD64 Zen6 Platform Quality of Service (PQOS) Extensions,
>>>    Publication # 69193 Revision 1.00, Issue Date March 2026
>>>
>>> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537
>>>
>>> Series overview
>>> ---------------
>>>
>>> Patches 1-5 to enable GMBA:
>>>
>>>    1/8  x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE)
>>>
>>>    2/8  x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains
>>>         Add a new ctrl_scope value for resctrl resources whose control
>>>         domain spans multiple L3s within an NPS node.
>>>
>>>    3/8  x86/resctrl: Update control MSRs per L3 for NPS-scoped resources
>>>         Add resctrl_arch_update_nps(): builds a cpumask with one CPU per
>>>         distinct L3 in the domain, then issues rdt_ctrl_update() via
>>>         smp_call_function_many() on that mask. Falls back to the full
>>>         domain mask if the scratch masks cannot be built. Route
>>>         resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls()
>>>         through this helper when ctrl_scope == RESCTRL_NPS_NODE.
>>>
>>>    4/8  x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation
>>>         Register RDT_RESOURCE_GMBA in rdt_resources_all[] with
>>>         ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to
>>>         discover feature details.
>>>
>>>    5/8  fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation
>>>         Add examples in Documentation/filesystems/resctrl.rst.
>>>
>>> Patches 6-8 to enable GSMBA in the same shape:
>>>
>>>    6/8  x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation
>>>
>>>    7/8  x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation
>>>         Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE.
>>>
>>>    8/8  fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation
>>>         Add examples in Documentation/filesystems/resctrl.rst.
>>>
>>> Changes since v1
>>> ----------------
>>>    - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and
>>>      Privilege Level Zero Association (PLZA). This series only handles
>>>      Global Memory Bandwidth Allocation. Both the features are sent separately.
>>>
>>>    - Documentation
>>>        * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst.
>>>        * Added examples to update GMBA and GSMBA in resctrl.rst documentation.
>>>
>>>    - Major changes are releated to RESCTRL_NPS_NODE scope handling.
>>>
>>>    - Commit messages
>>>        * Reworked the changelogs in all the patches.
>>>
>>> Previous Revisions:
>>> v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/
>>
>> What are your expectations from this submission? From what I can tell this ignores
>> v1 feedback in several ways:
>> - It introduces two new resources, GMBA and GSMBA, when the previous discussion agreed that
>>    these are not actually new resources but instead new controls for the existing MBA/SMBA resources.
>> - It does not mention or attempt to address dependency on new resource schema descriptions [1]
>>    to support user space in understanding how to interact with the new GMBA/GSMBA controls but
>>    instead defers that to a snippet in the documentation that user space needs to
>>    parse to know this control operates at multiples of 1GB/s.
>>
>> Apart from ignoring v1 feedback this new version appears to complicate user interface even more
>> since now it is possible for there to be a single control that may operate at different scopes but from
>> what I can tell there is nothing that helps user understand whether, for example, domain "0" means
>> the whole system or a NUMA node?
>>
>> We have discussed several times now how resctrl interface needs to be enhanced to support
>> this and other upcoming features from Intel, RISC-V, Arm MPAM, and NVidia. It is thus
>> unexpected that this submission ignores all the previous discussions.
> 
> I think there may be some misunderstanding on this topic.
> 
> Yes, we discussed it earlier. It depends on other requirements (region-aware aspects), so I assumed it would be handled by someone with full context and addressed as a separate feature. I didn’t have complete visibility into all the requirements.

Please read https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/ again.

You should have complete visibility into the foundation of this work since one of the
primary goals is to address the resctrl interface breakage that came with the initial AMD
support for MBA that resctrl has been living with until now. 

With this series you completely disregard attempts to support users in understanding
how to interact with the schemata file and instead introduce *another* obfuscated control. I
will not support this.

Also, no, this does not depend on region-aware work. Needing to support multiple controls for
a single resource is independent from region-aware.

>> Since there are so many dependencies on the new schema format support I am prioritizing this
>> and created a PoC that I am currently refining and hope to share soon. We can collaborate on this
>> to ensure that it provides a good foundation for the GMBA and GSMBA support.
> 
> That is good to know. Let me know when you are ready.
> 
> Could you please share which parts of the feature (e.g., Part 1, Part 2, etc.) you are planning to cover in your PoC?

All three parts mentioned in https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/

This does not address all the features discussed, for example it does not support emulated controls,
but I hope it is enough of a foundation to build on.

>> Consider what I describe in [2] - even in that response I speculate that a "scope" may be needed and
>> this seems to be case. I believe would help this "NPS = 4" scenario. Adding "scope" to what I shared in
>> [2] may look like:
>>
>> info/
>> └── MB/
>>      └── resource_schemata/
>>          ├── GMB/
>>          │   ├── max:4096
>>          │   ├── min:1
>>          │   ├── resolution:1
>>          │   ├── scale:1
>>          │   ├── tolerance:0
>>          │   ├── type:scalar linear
>>          │   ├── scope:NODE
>>          │   └── unit:GBps
>>          └── MB/
>>              ├── max:8192
>>              ├── min:1
>>              ├── resolution:8
>>              ├── scale:1
>>              ├── tolerance:0
>>              ├── type:scalar linear
>>              ├── scope:L3
>>              └── unit:GBps
>>
>> With a "scope" property of the control user space can know what the domain ID in the
>> schemata file refers to. In above example the "GMB" control has "NODE" scope so user space knows
>> that a domain ID refers to NUMA node. If the system is "NPS = 4" then the scope could be,
>> for example, "SYSTEM" (for the lack of a better term) so that user space knows that "0" means
>> entire system. What do you think?
> 
> Yes. Sound good to me.
> 
>>
>> Also note how the other control properties helps user understand what the schemata file control
>> values mean. This is what I expected the GMBA/GSMBA enabling to look like ... and you seemed to
>> agree [3] in v1 discussion. What changed?
> 
> Again, I was under the impression that this would be handled as a separate feature and patch series.

Right. And since GMBA/GSMBA depends on it this submission without resolving this this dependency first
is unexpected.

Reinette