[PATCH v2 0/3] x86/resctrl: Keep default MBM mode at boot and fix ABMC

[PATCH v2 0/3] x86/resctrl: Keep default MBM mode at boot and fix ABMC

[Babu Moger]

Hi All,

This series restores default MBM assignment mode at boot, assigns counters
when switching to mbm_event, and fixes ABMC programming for more than 32
counters.

Commit 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
enabled ABMC at boot. That breaks userspace that assumes default mode,
notably pqos from intel-cmt-cat [1]. pqos creates 16 or more groups (two
counters each) and exhausts a 32-counter pool, so further groups report
zero bandwidth.

mbm_event remains available as an opt-in for snapshot/rotation when there
are more groups than counters. Default mode can monitor more groups on
current AMD platforms, but there is a known issue with default mode when
monitoring more groups. It is documented now.

Patch 1: encode the full ABMC counter range in L3_QOS_ABMC_CFG (cntr_id
was 5 bits), expand bw_src, and use u64 to avoid 32-bit truncation.
The APM [2] at [3] will be updated for the expanded fields.

Patch 2: when switching to mbm_event, assign counters to existing
groups, including the default group, matching mkdir auto-assignment.

Patch 3: leave mbm_assign_mode as default at boot.

Based on tip/master:

  461735aa6e8e Merge branch into tip/master: 'x86/bugs'

[1] https://github.com/intel/intel-cmt-cat/issues/311
[2] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
    Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
    Bandwidth Monitoring (ABMC)"
[3] https://bugzilla.kernel.org/show_bug.cgi?id=206537

v2:
- Added patch 2 to address Sashiko's comment regarding the documentation issue.
  In fact, it exposed a real issue. When switching to mbm_event mode, existing
  monitoring groups should be assigned counters whenever counters are available.
  This provides a smooth transition between modes and aligns the behavior with
  the existing auto-assignment mechanism.
  https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com

- Combine the two v1 patches and add patch 2 (assign existing groups
  on mbm_event switch; Sashiko review).
- Keep boot-default separate from the encoding/truncation fixes.
- Document the default-mode counter-pool limitation.

v1:
  https://lore.kernel.org/lkml/980f39d3a0e0d9f73925e362f835aeef070a1bc5.1784322818.git.babu.moger@amd.com/

Thanks
Babu Moger

Babu Moger (3):
  x86/resctrl: Fix ABMC counter programming for extended counter ranges
  fs/resctrl: Assign counters to existing groups when enabling mbm_event
  x86/resctrl: Keep mbm_assign_mode in default mode at boot

 Documentation/filesystems/resctrl.rst  | 86 ++++++++++++++++----------
 arch/x86/kernel/cpu/resctrl/internal.h | 18 +++---
 arch/x86/kernel/cpu/resctrl/monitor.c  |  1 -
 fs/resctrl/monitor.c                   | 30 ++++++++-
 4 files changed, 89 insertions(+), 46 deletions(-)

-- 
2.43.0

[PATCH v2 1/3] x86/resctrl: Fix ABMC counter programming for extended counter ranges

[Babu Moger]

Memory Bandwidth Monitoring (MBM) can report incorrect values when ABMC is
enabled on systems supporting more than 32 ABMC counters. As the number of
active monitoring groups increases beyond the range supported by the
existing counter ID encoding, programming an ABMC counter may inadvertently
affect a different counter, resulting in unexpected counter resets and
abnormally large MBM readings.

The issue originates from the ABMC counter programming interface in the
L3_QOS_ABMC_CFG MSR. The counter ID field is currently defined as 5 bits,
which limits the addressable counter range to 32 counters. On systems
implementing more than 32 ABMC counters, counter IDs above 31 cannot be
encoded correctly. Consequently, programming a counter ID beyond the
supported range may target an unintended counter and reset bandwidth
statistics associated with another monitoring group.

While updating this logic, it was also observed that the bw_src field,
which encodes the RMID, is currently at its 12-bit limit with support for
4096 RMIDs. This field also needs to be updated for future expansion.

Also found one more pre-existing issue. This union structure can truncate
data on 32-bit x86 systems when unsigned long is used.

Fix the issues with the following changes:

1. Update the cntr_id field handling to support the full hardware ABMC
counter range and ensure that counter programming does not interfere with
unrelated counters.

2. Expand the bw_src field to 15 bits.

3. Change "unsigned long" to u64 to fix truncation on 32-bit x86.

The AMD64 Architecture Programmer's Manual [1] available at [2] will be
updated accordingly in a future revision to document the expanded cntr_id
and bw_src field definitions.

[1] AMD64 Architecture Programmer's Manual Volume 2: System Programming,
    Publication #24593, Revision 3.41, Section 19.3.3.3 "Assignable
    Bandwidth Monitoring (ABMC)"

Fixes: 84ecefb76674 ("x86/resctrl: Add data structures and definitions for ABMC assignment")
Signed-off-by: Babu Moger <babu.moger@amd.com>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 # [2]
---
v2: Moved the link tag to the last.

v1: https://lore.kernel.org/lkml/980f39d3a0e0d9f73925e362f835aeef070a1bc5.1784322818.git.babu.moger@amd.com/
---
 arch/x86/kernel/cpu/resctrl/internal.h | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
index e3cfa0c10e92..b3d780eda8a7 100644
--- a/arch/x86/kernel/cpu/resctrl/internal.h
+++ b/arch/x86/kernel/cpu/resctrl/internal.h
@@ -192,7 +192,6 @@ union cpuid_0x10_x_edx {
  * @bw_type		: Event configuration that represents the memory
  *			  transactions being tracked by the @cntr_id.
  * @bw_src		: Bandwidth source (RMID or CLOSID).
- * @reserved1		: Reserved.
  * @is_clos		: @bw_src field is a CLOSID (not an RMID).
  * @cntr_id		: Counter identifier.
  * @reserved		: Reserved.
@@ -210,16 +209,15 @@ union cpuid_0x10_x_edx {
  */
 union l3_qos_abmc_cfg {
 	struct {
-		unsigned long bw_type  :32,
-			      bw_src   :12,
-			      reserved1: 3,
-			      is_clos  : 1,
-			      cntr_id  : 5,
-			      reserved : 9,
-			      cntr_en  : 1,
-			      cfg_en   : 1;
+		u64	bw_type  :32,
+			bw_src   :15,
+			is_clos  : 1,
+			cntr_id  :12,
+			reserved : 2,
+			cntr_en  : 1,
+			cfg_en   : 1;
 	} split;
-	unsigned long full;
+	u64	full;
 };
 
 void rdt_ctrl_update(void *arg);
-- 
2.43.0

[PATCH v2 2/3] fs/resctrl: Assign counters to existing groups when enabling mbm_event

[Babu Moger]

resctrl_mbm_assign_mode_write() frees all counters and sets
mbm_assign_on_mkdir for subsequent mkdir, but does not assign counters to
groups that already exist, including the default group created at mount.
Those events then read "Unassigned" until the user assigns counters by
hand.

Enable mbm_assign_on_mkdir and assign counters to existing CTRL_MON and MON
groups with resctrl_assign_cntrs_allrdtgrp() so the switch matches mkdir
auto-assignment. Groups left without a counter still read "Unassigned".

Update Documentation/filesystems/resctrl.rst to describe this.

Fixes: 8004ea01cf63 ("fs/resctrl: Introduce the interface to switch between monitor modes")
Signed-off-by: Babu Moger <babu.moger@amd.com>
---
v2: New patch.
    This patch addresses the Sashiko comment about documentation issue where
    counters are not assigned automatically when mode is switched to mbm_event.
    https://sashiko.dev/#/patchset/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger%40amd.com
    In fact, it exposed a real issue. When switching to mbm_event mode, existing
    monitoring groups should be assigned counters whenever counters are available.
    This provides a smooth transition between modes and aligns the behavior with
    the existing auto-assignment mechanism.
---
 Documentation/filesystems/resctrl.rst |  7 +++++--
 fs/resctrl/monitor.c                  | 30 ++++++++++++++++++++++++---
 2 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index e4b66af55ffb..79feeb1dc296 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -371,8 +371,11 @@ with the following files:
 	of counters available is described in the "num_mbm_cntrs" file. Changing the
 	mode may cause all counters on the resource to reset.
 
-	Moving to mbm_event counter assignment mode requires users to assign the counters
-	to the events. Otherwise, the MBM event counters will return 'Unassigned' when read.
+	Moving to mbm_event counter assignment mode enables "mbm_assign_on_mkdir" and
+	assigns counters to the events of all existing groups, including the default
+	group, for as long as counters remain available. Events left without a counter
+	will return 'Unassigned' when read until the user assigns one using
+	"mbm_L3_assignments".
 
 	The mode is beneficial for AMD platforms that support more CTRL_MON
 	and MON groups than available hardware counters. By default, this
diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c
index 73413cb128ea..61463741b91b 100644
--- a/fs/resctrl/monitor.c
+++ b/fs/resctrl/monitor.c
@@ -1326,6 +1326,27 @@ void rdtgroup_assign_cntrs(struct rdtgroup *rdtgrp)
 					   &mon_event_all[QOS_L3_MBM_LOCAL_EVENT_ID]);
 }
 
+/*
+ * resctrl_assign_cntrs_allrdtgrp() - Assign counters to the MBM events of every
+ *				      existing group. Called when "mbm_event" mode
+ *				      is enabled.
+ *
+ * Groups created while in "default" mode have no counter assigned, including the
+ * default group created when resctrl is mounted. Assign counters to them so that
+ * enabling the mode leaves the same assignments that mkdir would have made.
+ */
+static void resctrl_assign_cntrs_allrdtgrp(void)
+{
+	struct rdtgroup *prgrp, *crgrp;
+
+	list_for_each_entry(prgrp, &rdt_all_groups, rdtgroup_list) {
+		rdtgroup_assign_cntrs(prgrp);
+
+		list_for_each_entry(crgrp, &prgrp->mon.crdtgrp_list, mon.crdtgrp_list)
+			rdtgroup_assign_cntrs(crgrp);
+	}
+}
+
 /*
  * rdtgroup_free_unassign_cntr() - Unassign and reset the counter ID configuration
  * for the event pointed to by @mevt within the domain @d and resctrl group @rdtgrp.
@@ -1599,9 +1620,6 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
 									   (READS_TO_LOCAL_MEM |
 									    READS_TO_LOCAL_S_MEM |
 									    NON_TEMP_WRITE_TO_LOCAL_MEM);
-		/* Enable auto assignment when switching to "mbm_event" mode */
-		if (enable)
-			r->mon.mbm_assign_on_mkdir = true;
 		/*
 		 * Reset all the non-achitectural RMID state and assignable counters.
 		 */
@@ -1609,6 +1627,12 @@ ssize_t resctrl_mbm_assign_mode_write(struct kernfs_open_file *of, char *buf,
 			mbm_cntr_free_all(r, d);
 			resctrl_reset_rmid_all(r, d);
 		}
+
+		if (enable) {
+			/* Enable auto assignment when switching to "mbm_event" mode */
+			r->mon.mbm_assign_on_mkdir = true;
+			resctrl_assign_cntrs_allrdtgrp();
+		}
 	}
 
 out_unlock:
-- 
2.43.0

[PATCH v2 3/3] x86/resctrl: Keep mbm_assign_mode in default mode at boot

[Babu Moger]

ABMC ("mbm_event" mode) allows explicit assignment of hardware MBM counters
to RMID/event pairs. It is intended for deployments that need to manage
counter assignment on platforms where the number of monitoring groups
exceeds the available hardware counters. Because hardware MBM counters are
scarce, mbm_event mode is best suited for snapshot-and-rotation workflows
that monitor a subset of groups at a time.

Commit 0f1576e43adc ("x86/resctrl: Configure mbm_event mode if supported")
enabled ABMC automatically during initialization. This causes problems on
systems with limited MBM counters and breaks existing userspace that
assumes the historical default mode, including the pqos tool from
intel-cmt-cat [1].

For example, pqos mounts resctrl and creates 16 or more monitoring groups,
using two counters per group (mbm_local_bytes and mbm_total_bytes). On
platforms that provide 32 MBM counters per domain, this consumes the entire
counter pool. Additional groups cannot be assigned counters and pqos
reports zero bandwidth for them.

Leave mbm_assign_mode in "default" mode during initialization. Default mode
can support more monitoring groups (up to 64) than mbm_event mode, which is
typically limited to 16 groups because of hardware counter availability.
Common deployments with a modest number of monitoring groups continue to
receive accurate bandwidth measurements.

Users that require ABMC functionality can enable it explicitly:

  echo mbm_event > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode

Note that default mode has a long-standing limitation when the number of
monitoring groups exceeds the available counter pool. After hardware
counter reallocation, reads may return "Unavailable" or misleading values.
Users that need stable measurements across a large number of monitoring
groups should use mbm_event mode and rotate assignments as needed.

Update Documentation/filesystems/resctrl.rst to reflect the default boot
behavior, document the limitations of default mode, and adjust
mbm_assign_mode examples accordingly.

Signed-off-by: Babu Moger <babu.moger@amd.com>
Link: https://github.com/intel/intel-cmt-cat/issues/311 # [1]
---
v2:
  Added documentation describing the known issue with the default mode.
  Will add cc to stable once we have all the things in order.
  Let me know if I missed anything.

v1:
  https://lore.kernel.org/lkml/8cb66e18e32e4087a9712c1e68ee6da614efe244.1784322818.git.babu.moger@amd.com/
---
 Documentation/filesystems/resctrl.rst | 79 +++++++++++++++++----------
 arch/x86/kernel/cpu/resctrl/monitor.c |  1 -
 2 files changed, 49 insertions(+), 31 deletions(-)

diff --git a/Documentation/filesystems/resctrl.rst b/Documentation/filesystems/resctrl.rst
index 79feeb1dc296..a43ed3c89a93 100644
--- a/Documentation/filesystems/resctrl.rst
+++ b/Documentation/filesystems/resctrl.rst
@@ -355,8 +355,8 @@ with the following files:
 	::
 
 	  # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
-	  [mbm_event]
-	  default
+	  [default]
+	  mbm_event
 
 	"mbm_event":
 
@@ -377,20 +377,30 @@ with the following files:
 	will return 'Unassigned' when read until the user assigns one using
 	"mbm_L3_assignments".
 
-	The mode is beneficial for AMD platforms that support more CTRL_MON
-	and MON groups than available hardware counters. By default, this
-	feature is enabled on AMD platforms with the ABMC (Assignable Bandwidth
-	Monitoring Counters) capability, ensuring counters remain assigned even
-	when the corresponding RMID is not actively used by any processor.
+	The mode is beneficial for AMD platforms that support more CTRL_MON and MON
+	groups than available hardware counters. The mbm_event mode ensures counters
+	remain assigned even when the corresponding RMID is not actively monitored.
 
 	"default":
 
 	In default mode, resctrl assumes there is a hardware counter for each
-	event within every CTRL_MON and MON group. On AMD platforms, it is
-	recommended to use the mbm_event mode, if supported, to prevent reset of MBM
-	events between reads resulting from hardware re-allocating counters. This can
-	result in misleading values or display "Unavailable" if no counter is assigned
-	to the event.
+	event within every CTRL_MON and MON group. This mode is enabled by default.
+
+	Default mode has a long-standing limitation on AMD platforms that support
+	more CTRL_MON and MON groups than hardware counters. Hardware dynamically
+	shares a smaller pool of counters among RMIDs. The size of that pool is
+	not enumerated to software (unlike "num_mbm_cntrs" in mbm_event mode), and
+	"num_rmids" may be much larger. On current AMD platforms this pool can
+	provide more counters than mbm_event mode (for example 64, versus 32 ABMC
+	counters), so more groups can be monitored accurately than with mbm_event.
+	Typical usage with fewer groups keeps a counter attached and readings
+	remain accurate. Creating more groups than that pool (for example 64 or
+	more) can cause hardware to re-allocate counters
+	between reads. Bandwidth values may then be misleading, or reads may return
+	"Unavailable" if no counter is allocated to the event. There is no
+	user-visible indication when this begins. Users who need stable readings
+	for many groups should switch to mbm_event mode, if supported, and assign
+	counters to the groups of interest (rotating assignments as needed).
 
 	* To enable "mbm_event" counter assignment mode:
 	  ::
@@ -474,8 +484,8 @@ with the following files:
 
 	Determines if a counter will automatically be assigned to an RMID, MBM event
 	pair when its associated monitor group is created via mkdir. Enabled by default
-	on boot, also when switched from "default" mode to "mbm_event" counter assignment
-	mode. Users can disable this capability by writing to the interface.
+	when switched to "mbm_event" counter assignment mode. Users can disable this
+	capability by writing to the interface.
 
 	"0":
 		Auto assignment is disabled.
@@ -1791,32 +1801,41 @@ a. Check if MBM counter assignment mode is supported.
 
   # mount -t resctrl resctrl /sys/fs/resctrl/
 
+  # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
+  [default]
+  mbm_event
+
+The "mbm_event" and "default" modes are supported. The "default" mode
+is enabled by default.
+
+b. Enable "mbm_event" counter assignment mode.
+::
+
+  # echo "mbm_event" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
   # cat /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
   [mbm_event]
   default
 
-The "mbm_event" mode is detected and enabled.
-
-b. Check how many assignable counters are supported.
+c. Check how many assignable counters are supported.
 ::
 
   # cat /sys/fs/resctrl/info/L3_MON/num_mbm_cntrs
   0=32;1=32
 
-c. Check how many assignable counters are available for assignment in each domain.
+d. Check how many assignable counters are available for assignment in each domain.
 ::
 
   # cat /sys/fs/resctrl/info/L3_MON/available_mbm_cntrs
   0=30;1=30
 
-d. To list the default group's assign states.
+e. To list the default group's assign states.
 ::
 
   # cat /sys/fs/resctrl/mbm_L3_assignments
   mbm_total_bytes:0=e;1=e
   mbm_local_bytes:0=e;1=e
 
-e.  To unassign the counter associated with the mbm_total_bytes event on domain 0.
+f.  To unassign the counter associated with the mbm_total_bytes event on domain 0.
 ::
 
   # echo "mbm_total_bytes:0=_" > /sys/fs/resctrl/mbm_L3_assignments
@@ -1824,7 +1843,7 @@ e.  To unassign the counter associated with the mbm_total_bytes event on domain
   mbm_total_bytes:0=_;1=e
   mbm_local_bytes:0=e;1=e
 
-f. To unassign the counter associated with the mbm_total_bytes event on all domains.
+g. To unassign the counter associated with the mbm_total_bytes event on all domains.
 ::
 
   # echo "mbm_total_bytes:*=_" > /sys/fs/resctrl/mbm_L3_assignments
@@ -1832,7 +1851,7 @@ f. To unassign the counter associated with the mbm_total_bytes event on all doma
   mbm_total_bytes:0=_;1=_
   mbm_local_bytes:0=e;1=e
 
-g. To assign a counter associated with the mbm_total_bytes event on all domains in
+h. To assign a counter associated with the mbm_total_bytes event on all domains in
 exclusive mode.
 ::
 
@@ -1841,7 +1860,7 @@ exclusive mode.
   mbm_total_bytes:0=e;1=e
   mbm_local_bytes:0=e;1=e
 
-h. Read the events mbm_total_bytes and mbm_local_bytes of the default group. There is
+i. Read the events mbm_total_bytes and mbm_local_bytes of the default group. There is
 no change in reading the events with the assignment.
 ::
 
@@ -1854,7 +1873,7 @@ no change in reading the events with the assignment.
   # cat /sys/fs/resctrl/mon_data/mon_L3_01/mbm_local_bytes
   121212144
 
-i. Check the event configurations.
+j. Check the event configurations.
 ::
 
   # cat /sys/fs/resctrl/info/L3_MON/event_configs/mbm_total_bytes/event_filter
@@ -1864,7 +1883,7 @@ i. Check the event configurations.
   # cat /sys/fs/resctrl/info/L3_MON/event_configs/mbm_local_bytes/event_filter
   local_reads,local_non_temporal_writes,local_reads_slow_memory
 
-j. Change the event configuration for mbm_local_bytes.
+k. Change the event configuration for mbm_local_bytes.
 ::
 
   # echo "local_reads, local_non_temporal_writes, local_reads_slow_memory, remote_reads" >
@@ -1873,7 +1892,7 @@ j. Change the event configuration for mbm_local_bytes.
   # cat /sys/fs/resctrl/info/L3_MON/event_configs/mbm_local_bytes/event_filter
   local_reads,local_non_temporal_writes,local_reads_slow_memory,remote_reads
 
-k. Now read the local events again. The first read may come back with "Unavailable"
+l. Now read the local events again. The first read may come back with "Unavailable"
 status. The subsequent read of mbm_local_bytes will display the current value.
 ::
 
@@ -1886,9 +1905,9 @@ status. The subsequent read of mbm_local_bytes will display the current value.
   # cat /sys/fs/resctrl/mon_data/mon_L3_01/mbm_local_bytes
   1566565
 
-l. Users have the option to go back to 'default' mbm_assign_mode if required. This can be
-done using the following command. Note that switching the mbm_assign_mode may reset all
-the MBM counters (and thus all MBM events) of all the resctrl groups.
+m. Users have the option to switch back to 'default' mbm_assign_mode if required. This
+can be done using the following command. Note that switching the mbm_assign_mode may
+reset all the MBM counters (and thus all MBM events) of all the resctrl groups.
 ::
 
   # echo "default" > /sys/fs/resctrl/info/L3_MON/mbm_assign_mode
@@ -1896,7 +1915,7 @@ the MBM counters (and thus all MBM events) of all the resctrl groups.
   mbm_event
   [default]
 
-m. Unmount the resctrl filesystem.
+n. Unmount the resctrl filesystem.
 ::
 
   # umount /sys/fs/resctrl/
diff --git a/arch/x86/kernel/cpu/resctrl/monitor.c b/arch/x86/kernel/cpu/resctrl/monitor.c
index 3838e0a13d36..8a0d6086518b 100644
--- a/arch/x86/kernel/cpu/resctrl/monitor.c
+++ b/arch/x86/kernel/cpu/resctrl/monitor.c
@@ -471,7 +471,6 @@ int __init rdt_get_l3_mon_config(struct rdt_resource *r)
 		r->mon.mbm_cntr_configurable = true;
 		cpuid_count(0x80000020, 5, &eax, &ebx, &ecx, &edx);
 		r->mon.num_mbm_cntrs = (ebx & GENMASK(15, 0)) + 1;
-		hw_res->mbm_cntr_assign_enabled = true;
 	}
 
 	r->mon_capable = true;
-- 
2.43.0