Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot

[Babu Moger <babu.moger@amd.com>]

On 7/17/26 16:13, Babu Moger wrote:
> The kernel currently enables the ABMC-based "mbm_event" mode by default on
> hardware that supports it. However, this can cause bandwidth monitoring
> failures with existing userspace tools such as pqos.
> 
> The pqos tool mounts the resctrl filesystem and creates 16 or more resctrl
> groups by default. On systems with 32 or fewer ABMC counters, this default
> configuration can consume all available counters, since each group requires
> one counter for local MBM and another for total MBM. If additional
> monitoring groups are created, counter resources are exhausted and pqos
> tool reports memory bandwidth counters as zero for those groups.
> 
> Avoid this compatibility issue by leaving mbm_assign_mode in the "default"
> mode during initialization. Users who want to use ABMC can continue to
> enable it explicitly:
> 
>   echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> 
> Update the resctrl documentation to reflect the new boot-time default and
> adjust the mbm_assign_mode examples accordingly.
> 
> Signed-off-by: Babu Moger <babu.moger@amd.com>
> ---
> There are plans to enable "mbm_event" by default once additional counters
> are available. For now, keep the default mode to maintain compatibility
> with existing tools.
> ---
>   Documentation/filesystems/resctrl.rst | 71 ++++++++++++++++-----------
>   arch/x86/kernel/cpu/resctrl/monitor.c |  1 -
>   2 files changed, 42 insertions(+), 30 deletions(-)
> 
> diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
> index e4b66af55ffb..e38bfd15fc3d 100644
> --- a/Documentation/filesystems/resctrl.rst
> +++ b/Documentation/filesystems/resctrl.rst
> @@ -355,8 +355,8 @@ with the following files:
>   	::
>   
>   	  # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> -	  [mbm_event]
> -	  default
> +	  [default]
> +	  mbm_event
>   
>   	"mbm_event":
>   
> @@ -375,19 +375,23 @@ with the following files:
>   	to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
>   
>   	The mode is beneficial for AMD platforms that support more CTRL_MON
> -	and MON groups than available hardware counters. By default, this
> -	feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
> -	Monitoring Counters) capability, ensuring counters remain assigned even
> -	when the corresponding RMID is not actively used by any processor.
> +	and MON groups than available hardware counters. On platforms with the
> +	ABMC (Assignable Bandwidth Monitoring Counters) capability, mbm_event
> +	mode ensures counters remain assigned even when the corresponding RMID
> +	is not actively used by any processor.
>   
>   	"default":
>   
>   	In default mode, resctrl assumes there is a hardware counter for each
> -	event within every CTRL_MON and MON group. On AMD platforms, it is
> -	recommended to use the mbm_event mode, if supported, to prevent reset of MBM
> -	events between reads resulting from hardware re-allocating counters. This can
> -	result in misleading values or display "Unavailable" if no counter is assigned
> -	to the event.
> +	event within every CTRL_MON and MON group. This mode is enabled by default.
> +
> +	On AMD platforms with more CTRL_MON and MON groups than the available
> +	hardware counters, hardware may re-allocate counters between reads
> +	while in default mode. This can result in misleading memory bandwidth values
> +	or display "Unavailable" if no counter is allocated to the event. In such
> +	cases, it is recommended to use the mbm_event mode, if supported, to prevent
> +	reset of MBM events between reads resulting from hardware re-allocating
> +	counters.
>   
>   	* To enable "mbm_event" counter assignment mode:
>   	  ::
> @@ -471,8 +475,8 @@ with the following files:
>   
>   	Determines if a counter will automatically be assigned to an RMID, MBM event
>   	pair when its associated monitor group is created via mkdir. Enabled by default
> -	on boot, also when switched from "default" mode to "mbm_event" counter assignment
> -	mode. Users can disable this capability by writing to the interface.
> +	when switched to "mbm_event" counter assignment mode. Users can disable this
> +	capability by writing to the interface.
>   
>   	"0":
>   		Auto assignment is disabled.
> @@ -1788,32 +1792,41 @@ a. Check if MBM counter assignment mode is supported.
>   
>     # mount -t resctrl resctrl /sys/fs/resctrl/
>   
> +  # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
> +  [default]
> +  mbm_event
> +
> +The "mbm_event" and "default" modes are supported. The "default" mode
> +is enabled by default.
> +
> +b. Enable "mbm_event" counter assignment mode.
> +::
> +
> +  # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
>     # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
>     [mbm_event]
>     default
>   
> -The "mbm_event" mode is detected and enabled.
> -
> -b. Check how many assignable counters are supported.
> +c. Check how many assignable counters are supported.
>   ::
>   
>     # cat /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs
>     0=32;1=32
>   
> -c. Check how many assignable counters are available for assignment in each domain.
> +d. Check how many assignable counters are available for assignment in each domain.
>   ::
>   
>     # cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
>     0=30;1=30
>   
> -d. To list the default group's assign states.
> +e. To list the default group's assign states.
>   ::
>   
>     # cat /sys/fs/resctrl/mbm_L3_assignments
>     mbm_total_bytes:0=e;1=e
>     mbm_local_bytes:0=e;1=e

Shasiko reported.

https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com


--- Patch [1]: [PATCH 1/2] x86/resctrl, Documentation: Keep 
mbm_assign_mode "default" on boot ---
commit 41dfae2edbe7538cef040aa9e0f1f0f5a5873564
Author: Babu Moger <babu.moger@amd.com>

x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot

This commit changes the default MBM assignment mode to "default" on boot to
maintain compatibility with existing userspace tools like pqos. Users 
who want
ABMC features must now explicitly enable the "mbm_event" mode.

 > diff --git a/Documentation/filesystems/resctrl.rst 
b/Documentation/filesystems/resctrl.rst
 > index e4b66af55ffba..e38bfd15fc3d0 100644
 > --- a/Documentation/filesystems/resctrl.rst
 > +++ b/Documentation/filesystems/resctrl.rst

[ ... ]

 > @@ -1788,32 +1792,41 @@ a. Check if MBM counter assignment mode is 
supported.
 >
 >    # mount -t resctrl resctrl /sys/fs/resctrl/
 >
 > +  # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
 > +  [default]
 > +  mbm_event
 > +
 > +The "mbm_event" and "default" modes are supported. The "default" mode
 > +is enabled by default.
 > +
 > +b. Enable "mbm_event" counter assignment mode.
 > +::
 > +
 > +  # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
 >    # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
 >    [mbm_event]
 >    default

[ ... ]

 > -d. To list the default group's assign states.
 > +e. To list the default group's assign states.
 >  ::
 >
 >    # cat /sys/fs/resctrl/mbm_L3_assignments
 >    mbm_total_bytes:0=e;1=e
 >    mbm_local_bytes:0=e;1=e

Does switching to "mbm_event" mode automatically assign counters to the
default group as shown here?

Because this patch changes the boot default to "default", users must now
transition to "mbm_event" dynamically. Looking at the transition path:

fs/resctrl/monitor.c:resctrl_mbm_assign_mode_write() {
     ...
     /* Enable auto assignment when switching to "mbm_event" mode */
     if (enable)
         r->mon.mbm_assign_on_mkdir = true;
     /*
      * Reset all the non-achitectural RMID state and assignable counters.
      */
     list_for_each_entry_rcu(d, &r->mon_domains, hdr.list, 
lockdep_is_cpus_held()) {
         mbm_cntr_free_all(r, d);
         resctrl_reset_rmid_all(r, d);
     }
     ...
}

This path clears all existing counters and sets mbm_assign_on_mkdir for
new groups, but does not appear to assign counters for the already-existing
default group.

Prior to this patch, booting directly into "mbm_event" mode handled the
default group assignment during rdt_get_tree().

Will the default group actually show _ (unassigned) states here instead
of the documented e (assigned) states, causing bandwidth monitoring to
return "Unassigned" until manually fixed?

===========================================================================

Yes. It does not automatically assign the counters to already created 
groups. User need to assign the counters manually to read the events.
The documentation needs to updated.

Thanks
Babu