Re: Region aware RDT options for resctrl

["Luck, Tony" <tony.luck@intel.com>]

On Mon, Apr 14, 2025 at 10:30:18AM -0700, Reinette Chatre wrote:
> Hi Tony,
> 
> Could you please help clarify how memory regions should be viewed?
> 
> On 4/11/25 1:56 PM, Luck, Tony wrote:
> > On Fri, Apr 11, 2025 at 01:54:12PM -0700, Luck, Tony wrote:
> > 
> > Add Cc: lkml
> > 
> >> A future CPU from Intel will implement "region aware" memory bandwidth
> >> monitoring and bandwidth allocation. This will provide for more granular
> >> monitoring and control for heterogeneous memory configurations. BIOS
> >> will populate an ACPI table that describes which system physical address
> >> ranges belong to each region. E.g. for a two socket system with both
> >> DDR and CXL memory regions could be assigned like this:
> >>
> >> Region 0: Local DDR
> >> Region 1: Remote DDR
> >> Region 2: Local CXL
> >> Region 3: Remote CXL
> 
> If considering an assignment like above ...
> 
> ...
> 
> >> Option 1: Per-memory regions might be described individually like this:
> >>
> >> 	$ cat schemata
> >> 	RMB0:0=100;1=100
> >> 	RMB1:0=75;1=75
> >> 	RMB2:0=25;1=25
> >>
> 
> ... I assume "RMB0" represents "Region 0" and so forth. In this case, what do
> the "domain IDs" used in above option represent?

Measurement and control is still done at the scope of each L3 cache. So
the domain ids in the schemata file and in the names of the directories
under "mon_data" are the Linux L3 cache ids.

Here's a different example of a schemata file (with different throttle
values in all positions to make the explanation below easier):

	$ cat schemata
	RMB0:0=100;1=75
	RMB1:0=50;1=25

This is a two socket system with just two regions (DDR local and remote)

0=100	CPUs in the domain of L3 instance 0 are not throttled for
	accesses to their local DDR
1=75	CPUs in the domain of L3 instance 1 are throttled to 75% for
	accesses to their local DDR
0=50	CPUS in the domain of L3 instance 0 are throttled to 50% for
	accesses to remote DDR
1=25	CPUs in the domain of L3 instance 1 are throttled to 25% for
	accesses to remote DDR

The ACPI MRRM table describes the memory ranges in each region. Each
range has two region number associated with it. One for local access,
the other for remote access. A dump of one entry looks like this:

[0002]                       Memory Range : 0000
[0002]                             Length : 0020
[0004]                           Reserved : 00000000
[0008]                System Address Base : 0000000000000000
[0008]              System Address Length : 00000000E0000000
[0002]                 Region Valid Flags : 0003
[0001]             Static Local Region ID : 00
[0001]            Static Remote Region ID : 01
[0004]                           Reserved : 00000000

It shows that the range from 0 GB to 3.5 GB will be counted/controlled
in region 0 when accessed by a CPU where this is local memory, but as
region 1 when accessed by a CPU where this range is remote.

> 
> Thank you.
> 
> Reinette

-Tony