Re: [PATCH 1/2] x86/resctrl, Documentation: Keep mbm_assign_mode "default" on boot

[Babu Moger <babu.moger@amd.com>]

Hi Reinette,

Thanks for quick response.

On 7/17/26 17:56, Reinette Chatre wrote:
> Hi Babu,
> 
> On 7/17/26 2:13 PM, Babu Moger wrote:
>> The kernel currently enables the ABMC-based "mbm_event" mode by default on
>> hardware that supports it. However, this can cause bandwidth monitoring
>> failures with existing userspace tools such as pqos.
>>
>> The pqos tool mounts the resctrl filesystem and creates 16 or more resctrl
>> groups by default. On systems with 32 or fewer ABMC counters, this default
>> configuration can consume all available counters, since each group requires
>> one counter for local MBM and another for total MBM. If additional
>> monitoring groups are created, counter resources are exhausted and pqos
>> tool reports memory bandwidth counters as zero for those groups.
> 
> It is not obvious to me that this is a problem. If I understand correctly
> there are two scenarios possible with this pqos behavior:
> 
> - ABMC is not in use ("mbm_assign_mode" is set to "default")
>    - pqos can create 16 or more monitor groups
>    - hardware still supports a limited number of counters with consequence that
>      underlying counters reset at any time as the different monitoring groups
>      need to be tracked.
>    - pqos can read monitoring data of all 16 monitor groups, sometimes reading the
>      events would return "Unavailable", sometimes reading the events return data.
>    - *None* of the monitoring numbers returned are guaranteed to be accurate.
> 
> - ABMC is in use ("mbm_assign_mode" is set to "mbm_event"):
>    - pqos can create 16 or more monitor groups
>    - only a subset of monitoring groups have counters assigned and these counters
>      are guaranteed to only track the monitor groups/events they are assigned to
>    - pqos can read monitoring data of all 16 monitor groups with two possibilities:
>      - monitor group/event has counter assigned: monitoring numbers are guaranteed to be accurate
>      - monitor group/event does not have counter assigned: monitoring numbers return 0
> 
> If my understanding is correct then the preference is to rather have wrong data than
> see 0? This does not sound right. What am I missing?


This hardware can monitor up to 64 RMIDs without any counter resets.

As you know, the pqos tool creates COS1 through COS15 regardless of the 
command-line options used, resulting in a total of 16 groups including 
the default group. With ABMC enabled, this consumes all available ABMC 
counters(32 counters, 2 counters for each group).

When pqos is invoked with the -m option, it creates additional 
monitoring groups. For example:

pqos -m all:0     -> creates 1 monitoring group
pqos -m all:0,1   -> creates 2 monitoring groups


Since all ABMC counters have already been allocated to the default set 
of groups, no counters remain for these additional monitoring groups. As 
a result, the monitoring commands report zero values, effectively making 
monitoring unusable.

In contrast, the default monitoring mode can still support up to 48 
additional monitoring groups (64 total RMIDs minus the 16 default groups 
created by pqos).

For this reason, I still believe keeping the default monitoring mode as 
the default is the better option.

Thanks
Babu

> 
> The changelog starts with "this can cause bandwidth monitoring failures with existing
> userspace tools such as pqos". How could returning accurate memory bandwidth data be
> considered a failure? How does this issue manifest itself?
> 
>>
>> Avoid this compatibility issue by leaving mbm_assign_mode in the "default"
>> mode during initialization. Users who want to use ABMC can continue to
>> enable it explicitly:
>>
>>   echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
>>
>> Update the resctrl documentation to reflect the new boot-time default and
>> adjust the mbm_assign_mode examples accordingly.
>>
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> ---
>> There are plans to enable "mbm_event" by default once additional counters
>> are available. For now, keep the default mode to maintain compatibility
>> with existing tools.
> 
> This is not ideal. resctrl should aim to provide a consistent user interface
> across kernel versions.
> 
> Reinette
> 
>