Re: [PATCH v7 00/10] x86,fs/resctrl: Support L3 Smart Data Cache Injection Allocation Enforcement (SDCIAE)

[Reinette Chatre <reinette.chatre@intel.com>]

Hi Babu,

On 7/10/25 10:16 AM, Babu Moger wrote:
> 
> This series adds the support for L3 Smart Data Cache Injection Allocation
> Enforcement (SDCIAE) to resctrl infrastructure. It is referred to as
> "io_alloc" in resctrl subsystem.
> 
> Upcoming AMD hardware implements Smart Data Cache Injection (SDCI).
> Smart Data Cache Injection (SDCI) is a mechanism that enables direct
> insertion of data from I/O devices into the L3 cache. By directly caching
> data from I/O devices rather than first storing the I/O data in DRAM, SDCI
> reduces demands on DRAM bandwidth and reduces latency to the processor
> consuming the I/O data.
> 
> The SDCIAE (SDCI Allocation Enforcement) PQE feature allows system software
> to control the portion of the L3 cache used for SDCI devices.
> 
> When enabled, SDCIAE forces all SDCI lines to be placed into the L3 cache
> partitions identified by the highest-supported L3_MASK_n register, where n
> is the maximum supported CLOSID.

Even though this CLOSID use is unique to AMD this implementation makes it part
of resctrl fs's support of io_alloc. It is confusing to have architectural
specific features be handled by resctrl fs so I think it will be useful to
add a snippet here to help folks trying to decipher this work. Consider
for example a snippet like:

  Since CLOSIDs are managed by resctrl fs it is least invasive to make
  the "io_alloc is supported by maximum supported CLOSID" part of the
  initial resctrl fs support for io_alloc. Take care not to expose this
  use of CLOSID for io_alloc to user space so that this is not required from
  other architectures that may support io_alloc differently in the future.

> 
> The feature details are documented in the APM listed below [1].
> [1] AMD64 Architecture Programmer's Manual Volume 2: System Programming
> Publication # 24593 Revision 3.41 section 19.4.7 L3 Smart Data Cache
> Injection Allocation Enforcement (SDCIAE)
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537
> 
> The feature requires linux support of TPH (TLP Processing Hints).
> The support is available in linux kernel after the commit
> 48d0fd2b903e3 ("PCI/TPH: Add TPH documentation")
> 
> The patches are based on top of commit (6.16.0-rc5)
> commit b4ec95e3bc3f ("Merge x86/microcode into tip/master")
> 
> # Linux Implementation
> 
> Feature adds following interface files when the resctrl "io_alloc" feature
> is supported on the resource:
> 
> /sys/fs/resctrl/info/L3/io_alloc: Report the feature status. Enable/disable the
> 				  feature by writing to the interface.
> 
> /sys/fs/resctrl/info/L3/io_alloc_cbm:  List the Capacity Bit Masks (CBMs) available
> 				       for I/O devices when io_alloc feature is enabled.
> 				       Configure the CBM by writing to the interface.
> 
> When CDP is enabled, these files will be created both in L3CODE and L3DATA.

"will be" -> "are"

> 
> # Examples:
> 
> a. Check if io_alloc feature is available
> 	# mount -t resctrl resctrl /sys/fs/resctrl/
> 
> 	# cat /sys/fs/resctrl/info/L3/io_alloc
> 	disabled
> 
> b. Enable the io_alloc feature. 
> 
> 	# echo 1 > /sys/fs/resctrl/info/L3/io_alloc 
> 	# cat /sys/fs/resctrl/info/L3/io_alloc
> 	enabled
> 
> c. Check the CBM values for the io_alloc feature.
> 
> 	# cat /sys/fs/resctrl/info/L3/io_alloc_cbm 
> 	0=ffff;1=ffff
> 
> d. Change the CBM value for the domain 1:

"for the domain 1" -> "of domain 1"?

(nit: inconsistent use of "." vs. ":" in items)

> 	# echo 1=FF > /sys/fs/resctrl/info/L3/io_alloc_cbm
> 
> 	# cat /sys/fs/resctrl/info/L3/io_alloc_cbm 
> 	0=ffff;1=00ff
> 
> d. Disable io_alloc feature and exit.
> 
> 	# echo 0 > /sys/fs/resctrl/info/L3/io_alloc 
> 	# cat /sys/fs/resctrl/info/L3/io_alloc
> 	disabled
> 
> 	# umount /sys/fs/resctrl/
> 
> ---

Reinette