Re: [PATCH v2 0/8] x86/resctrl: Support for AMD Global (Slow) Memory Bandwidth Allocation

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 4/23/26 6:41 PM, Babu Moger wrote:
> 
> This series adds resctrl support for two new AMD memory-bandwidth
> allocation features:
> 
>   - GMBA  - Global Memory Bandwidth Allocation (hardware name: GLBE).
>             Bounds DRAM bandwidth for groups of threads that span
>             multiple L3 QoS domains, rather than being per-L3 like MBA.
> 
>   - GSMBA - Global Slow Memory Bandwidth Allocation (hardware name:
>             GLSBE). The CXL.memory / slow-memory counterpart of GMBA,
>             analogous to how SMBA relates to MBA.
> 
> Both features share a new "NPS-node" control domain: a set of QoS (L3)
> domains grouped together and aligned to the system's NPS (Nodes Per
> Socket) BIOS configuration. Although the control domain is NPS-scoped,
> the underlying bandwidth-limit MSRs (MSR_IA32_GMBA_BW_BASE 0xc0000600,
> MSR_IA32_GSMBA_BW_BASE 0xc0000680) are instantiated per L3. Programming
> a single control domain therefore requires writing the MSR on one CPU
> per L3 that the domain spans - a new pattern for resctrl. Patches 2/8
> and 3/8 introduce that infrastructure so the new resources can reuse
> it.
> 
> The features are documented in:
> 
>   AMD64 Zen6 Platform Quality of Service (PQOS) Extensions,
>   Publication # 69193 Revision 1.00, Issue Date March 2026
> 
> available at https://bugzilla.kernel.org/show_bug.cgi?id=206537
> 
> Series overview
> ---------------
> 
> Patches 1-5 to enable GMBA:
> 
>   1/8  x86,fs/resctrl: Add support for Global Bandwidth Enforcement (GLBE)
> 
>   2/8  x86/resctrl: Add RESCTRL_NPS_NODE scope for AMD NPS-aligned domains
>        Add a new ctrl_scope value for resctrl resources whose control
>        domain spans multiple L3s within an NPS node.
> 
>   3/8  x86/resctrl: Update control MSRs per L3 for NPS-scoped resources
>        Add resctrl_arch_update_nps(): builds a cpumask with one CPU per
>        distinct L3 in the domain, then issues rdt_ctrl_update() via
>        smp_call_function_many() on that mask. Falls back to the full
>        domain mask if the scratch masks cannot be built. Route
>        resctrl_arch_update_domains() and resctrl_arch_reset_all_ctrls()
>        through this helper when ctrl_scope == RESCTRL_NPS_NODE.
> 
>   4/8  x86,fs/resctrl: Add the resource for Global Memory Bandwidth Allocation
>        Register RDT_RESOURCE_GMBA in rdt_resources_all[] with
>        ctrl_scope=RESCTRL_NPS_NODE and schema_fmt=RANGE, add commands to
>        discover feature details.
> 
>   5/8  fs/resctrl: Add the documentation for Global Memory Bandwidth Allocation
>        Add examples in Documentation/filesystems/resctrl.rst.
> 
> Patches 6-8 to enable GSMBA in the same shape:
> 
>   6/8  x86,fs/resctrl: Add support for Global Slow Memory Bandwidth Allocation
> 
>   7/8  x86,fs/resctrl: Add the resource for Global Slow Memory Bandwidth Allocation
>        Register RDT_RESOURCE_GSMBA with ctrl_scope=RESCTRL_NPS_NODE.
> 
>   8/8  fs/resctrl: Add the documentation for Global Slow Memory Bandwidth Allocation
>        Add examples in Documentation/filesystems/resctrl.rst.
> 
> Changes since v1
> ----------------
>   - Earlier sent RFC(v1) with Global Bandwidth Enforcement (GLBE) and
>     Privilege Level Zero Association (PLZA). This series only handles
>     Global Memory Bandwidth Allocation. Both the features are sent separately.
> 
>   - Documentation
>       * Fixed grammar in the GMBA / GSMBA sections of resctrl.rst.
>       * Added examples to update GMBA and GSMBA in resctrl.rst documentation.
> 
>   - Major changes are releated to RESCTRL_NPS_NODE scope handling.
> 
>   - Commit messages
>       * Reworked the changelogs in all the patches.
> 
> Previous Revisions:
> v1 : https://lore.kernel.org/lkml/cover.1769029977.git.babu.moger@amd.com/

What are your expectations from this submission? From what I can tell this ignores
v1 feedback in several ways:
- It introduces two new resources, GMBA and GSMBA, when the previous discussion agreed that
  these are not actually new resources but instead new controls for the existing MBA/SMBA resources.
- It does not mention or attempt to address dependency on new resource schema descriptions [1]
  to support user space in understanding how to interact with the new GMBA/GSMBA controls but
  instead defers that to a snippet in the documentation that user space needs to
  parse to know this control operates at multiples of 1GB/s. 

Apart from ignoring v1 feedback this new version appears to complicate user interface even more
since now it is possible for there to be a single control that may operate at different scopes but from
what I can tell there is nothing that helps user understand whether, for example, domain "0" means
the whole system or a NUMA node? 

We have discussed several times now how resctrl interface needs to be enhanced to support
this and other upcoming features from Intel, RISC-V, Arm MPAM, and NVidia. It is thus
unexpected that this submission ignores all the previous discussions. 

Since there are so many dependencies on the new schema format support I am prioritizing this
and created a PoC that I am currently refining and hope to share soon. We can collaborate on this
to ensure that it provides a good foundation for the GMBA and GSMBA support. 

Consider what I describe in [2] - even in that response I speculate that a "scope" may be needed and
this seems to be case. I believe would help this "NPS = 4" scenario. Adding "scope" to what I shared in
[2] may look like:

info/
└── MB/
    └── resource_schemata/
        ├── GMB/
        │   ├── max:4096
        │   ├── min:1
        │   ├── resolution:1
        │   ├── scale:1
        │   ├── tolerance:0
        │   ├── type:scalar linear
        │   ├── scope:NODE
        │   └── unit:GBps
        └── MB/
            ├── max:8192
            ├── min:1
            ├── resolution:8
            ├── scale:1
            ├── tolerance:0
            ├── type:scalar linear
            ├── scope:L3
            └── unit:GBps

With a "scope" property of the control user space can know what the domain ID in the
schemata file refers to. In above example the "GMB" control has "NODE" scope so user space knows
that a domain ID refers to NUMA node. If the system is "NPS = 4" then the scope could be,
for example, "SYSTEM" (for the lack of a better term) so that user space knows that "0" means
entire system. What do you think?

Also note how the other control properties helps user understand what the schemata file control
values mean. This is what I expected the GMBA/GSMBA enabling to look like ... and you seemed to
agree [3] in v1 discussion. What changed?

Reinette

[1] https://lore.kernel.org/lkml/aPtfMFfLV1l%2FRB0L@e133380.arm.com/
[2] https://lore.kernel.org/lkml/06a237bd-c370-4d3f-99de-124e8c50e711@intel.com/  
[3] https://lore.kernel.org/lkml/91d50431-41f3-49d7-a9e6-a3bee2de5162@amd.com/