Re: [PATCH v2 2/2] x86/resctrl: Don't workqueue local event counter reads

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Tony,

On 11/7/24 2:14 PM, Luck, Tony wrote:
>> I think maybe the issue you are trying to address is a user assigning a counter
>> and then reading the cached data and getting cached data from a previous
>> configuration? Please note that in the current implementation the cached
>> data is reset directly on counter assignment [1]. If a user assigns a new
>> counter and then immediately read cached data then the cached data will
>> reflect the assignment even if the overflow worker thread did not get a chance
>> to run since the assignment.
> 
> The issue is that AMD's ABMC implementation resets counts when reassigning
> h/w counters to events in resctrl groups.  If the processes reading counters is
> not fully aware of h/w counter reassignment, insanity will occur.
> 
> E.g. read a counter:
> 
> $ cat mbm_local_bytes
> 123456789
> 
> H/w counter for this event/group assigned elsewhere.
> 
> H/w counter assigned back to this event/group
> 
> $ cat mbm_local_bytes
> 23456
> 
> Bandwidth calculation sees traffic amount:
> 	 (23456 - 123456789) = -123433333
> Oops. Negative!

As I understand this is already an issue today on AMD systems without assignable counters
that may run out of counters. On these systems, any RMID that is no longer being tracked will
be reset to zero. [1] 

The support for assignable counters give user space control over this unexpected reset of
counters.

The scenario you present seem to demonstrate how two independent user space systems
can trample on each other when interacting with the same resources. Is this something you expect
resctrl should protect against? I would expect that there would be a single user space system
doing something like above and it would reset history after unassigning a counter.

This does indeed highlight that if resctrl does start to dynamically assign counters (which
has only been speculated in this thread and is not part of current [1] design) then it may cause
problems on user space side.

Reinette

[1] https://lore.kernel.org/all/cover.1730244116.git.babu.moger@amd.com/