Re: [PATCH v11 00/23] x86/resctrl : Support AMD Assignable Bandwidth Monitoring Counters (ABMC)

[Dave Martin <Dave.Martin@arm.com>]

On Mon, Feb 17, 2025 at 10:45:29AM -0600, Moger, Babu wrote:
> Hi All,
> 
> On 2/17/25 04:26, Peter Newman wrote:
> > Hi Reinette,
> > 
> > On Fri, Feb 14, 2025 at 8:18 PM Reinette Chatre
> > <reinette.chatre@intel.com> wrote:
> >>
> >> Hi Babu,
> >>
> >> On 2/14/25 10:31 AM, Moger, Babu wrote:
> >>> On 2/14/2025 12:26 AM, Reinette Chatre wrote:
> >>>> On 2/13/25 9:37 AM, Dave Martin wrote:
> >>>>> On Wed, Feb 12, 2025 at 03:33:31PM -0800, Reinette Chatre wrote:
> >>>>>> On 2/12/25 9:46 AM, Dave Martin wrote:
> >>>>>>> On Wed, Jan 22, 2025 at 02:20:08PM -0600, Babu Moger wrote:
> >>
> >> (quoting relevant parts with goal to focus discussion on new possible syntax)
> >>
> >>>>>> I see the support for MPAM events distinct from the support of assignable counters.
> >>>>>> Once the MPAM events are sorted, I think that they can be assigned with existing interface.
> >>>>>> Please help me understand if you see it differently.
> >>>>>>
> >>>>>> Doing so would need to come up with alphabetical letters for these events,
> >>>>>> which seems to be needed for your proposal also? If we use possible flags of:
> >>>>>>
> >>>>>> mbm_local_read_bytes a
> >>>>>> mbm_local_write_bytes b
> >>>>>>
> >>>>>> Then mbm_assign_control can be used as:
> >>>>>> # echo '//0=ab;1=b' >/sys/fs/resctrl/info/L3_MON/mbm_assign_control
> >>>>>> # cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_local_read_bytes
> >>>>>> <value>
> >>>>>> # cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_local_bytes
> >>>>>> <sum of mbm_local_read_bytes and mbm_local_write_bytes>
> >>>>>>
> >>>>>> One issue would be when resctrl needs to support more than 26 events (no more flags available),
> >>>>>> assuming that upper case would be used for "shared" counters (unless this interface is defined
> >>>>>> differently and only few uppercase letters used for it). Would this be too low of a limit?
> >>
> >> As mentioned above, one possible issue with existing interface is that
> >> it is limited to 26 events (assuming only lower case letters are used). The limit
> >> is low enough to be of concern.
> > 
> > The events which can be monitored by a single counter on ABMC and MPAM
> > so far are combinable, so 26 counters per group today means it limits
> > breaking down MBM traffic for each group 26 ways. If a user complained
> > that a 26-way breakdown of a group's MBM traffic was limiting their
> > investigation, I would question whether they know what they're looking
> > for.
> 
> Based on the discussion so far, it felt like it is not a group level
> breakdown. It is kind of global level breakdown. I could be wrong here.
> 
> My understanding so far, MPAM has a number of global counters. It can be
> assigned to any domain in the system and monitor events.
> 
> They also have a way to configure the events (read, write or both).
> 
> Both these feature are inline with current resctrl implementation and can
> be easily adapted.
> 
> One thing I am not clear why MPAM implementation plans to create separate
> files(dynamically) in /sys/fs/resctrl/info/L3_MON/ directory to read the
> events. We already have files in each group to read the events.
> 
> # ls -l /sys/fs/resctrl/mon_data/mon_L3_00/
> total 0
> -r--r--r--. 1 root root 0 Feb 17 08:16 llc_occupancy
> -r--r--r--. 1 root root 0 Feb 17 08:16 mbm_local_bytes
> -r--r--r--. 1 root root 0 Feb 17 08:16 mbm_total_bytes


To be clear, we have no current plan to do this from the Arm side.

My sketch was just a thought experiment to test whether we would have
difficulties _if_ a decision were made to extend the interface in that
direction.

But it looks OK to me: the interface proposed in this series seems to
leave enough possibilities for extension open that we could do
something like what I described later in if we decide to.


Overall, the interface proposed in this series seems a reasonable way
to support ABMC systems while keeping the consumer-side interface
(i.e., reading the mbm_total_bytes files etc.) as similar to the
classic / Intel RDT situation as possible.

MPAM can fit in with this approach, as demonstrated by James' past
branches porting the MPAM driver on top of previous versions of the
ABMC series.

As I understand it, he's almost done with porting onto this v11,
with no significant issues.

Cheers
---Dave