Re: [PATCH v6 12/12] Documentation/x86: Update resctrl_ui.rst for new features

["Moger, Babu" <babu.moger@amd.com>]

Hi Sanjaya,

On 10/11/22 04:06, Bagas Sanjaya wrote:
> On Mon, Oct 10, 2022 at 03:30:40PM -0500, Babu Moger wrote:
>> diff --git a/Documentation/x86/resctrl.rst b/Documentation/x86/resctrl.rst
>> index 71a531061e4e..e2a59249d183 100644
>> --- a/Documentation/x86/resctrl.rst
>> +++ b/Documentation/x86/resctrl.rst
>> @@ -17,14 +17,16 @@ AMD refers to this feature as AMD Platform Quality of Service(AMD QoS).
>>  This feature is enabled by the CONFIG_X86_CPU_RESCTRL and the x86 /proc/cpuinfo
>>  flag bits:
>>  
>> -=============================================	================================
>> +===============================================	================================
>>  RDT (Resource Director Technology) Allocation	"rdt_a"
>>  CAT (Cache Allocation Technology)		"cat_l3", "cat_l2"
>>  CDP (Code and Data Prioritization)		"cdp_l3", "cdp_l2"
>>  CQM (Cache QoS Monitoring)			"cqm_llc", "cqm_occup_llc"
>>  MBM (Memory Bandwidth Monitoring)		"cqm_mbm_total", "cqm_mbm_local"
>>  MBA (Memory Bandwidth Allocation)		"mba"
>> -=============================================	================================
>> +SMBA (Slow Memory Bandwidth Allocation)         "smba"
>> +BMEC (Bandwidth Monitoring Event Configuration) "bmec"
>> +===============================================	================================
>>  
>>  To use the feature mount the file system::
>>  
>> @@ -161,6 +163,79 @@ with the following files:
>>  "mon_features":
>>  		Lists the monitoring events if
>>  		monitoring is enabled for the resource.
>> +                Example::
>> +
>> +                   # cat /sys/fs/resctrl/info/L3_MON/mon_features
>> +                   llc_occupancy
>> +                   mbm_total_bytes
>> +                   mbm_local_bytes
>> +
>> +                If the system supports Bandwidth Monitoring Event
>> +                Configuration (BMEC), then the bandwidth events will
>> +                be configurable. The output will be::
>> +
>> +                   # cat /sys/fs/resctrl/info/L3_MON/mon_features
>> +                   llc_occupancy
>> +                   mbm_total_bytes
>> +                   mbm_total_config
>> +                   mbm_local_bytes
>> +                   mbm_local_config
>> +
>> +"mbm_total_config", "mbm_local_config":
>> +        These files contain the current event configuration for the events
>> +        mbm_total_bytes and mbm_local_bytes, respectively, when the
>> +        Bandwidth Monitoring Event Configuration (BMEC) feature is supported.
>> +        The event configuration settings are domain specific and will affect
>> +        all the CPUs in the domain.
>> +
>> +        Following are the types of events supported:
>> +
>> +        ====    ========================================================
>> +        Bits    Description
>> +        ====    ========================================================
>> +        6       Dirty Victims from the QOS domain to all types of memory
>> +        5       Reads to slow memory in the non-local NUMA domain
>> +        4       Reads to slow memory in the local NUMA domain
>> +        3       Non-temporal writes to non-local NUMA domain
>> +        2       Non-temporal writes to local NUMA domain
>> +        1       Reads to memory in the non-local NUMA domain
>> +        0       Reads to memory in the local NUMA domain
>> +        ====    ========================================================
>> +
>> +        By default, the mbm_total_bytes configuration is set to 0x7f to count
>> +        all the event types and the mbm_local_bytes configuration is set to
>> +        0x15 to count all the local memory events.
>> +
>> +        Examples:
>> +
>> +        * To view the current configuration::
>> +          ::
>> +
>> +            # cat /sys/fs/resctrl/info/L3_MON/mbm_total_config
>> +            0=0x7f;1=0x7f;2=0x7f;3=0x7f
>> +
>> +            # cat /sys/fs/resctrl/info/L3_MON/mbm_local_config
>> +            0=0x15;1=0x15;3=0x15;4=0x15
>> +
>> +        * To change the mbm_total_bytes to count only reads on domain 0.
>> +          To achieve this, the bits 0, 1, 4 and 5 needs to be set which is
>> +          110011b (in hex 0x33).
>> +          ::
>> +
>> +            # echo  "0=0x33" > /sys/fs/resctrl/info/L3_MON/mbm_total_config
>> +
>> +            # cat /sys/fs/resctrl/info/L3_MON/mbm_total_config
>> +            0=0x33;1=0x7f;2=0x7f;3=0x7f
>> +
>> +        * To change the mbm_local_bytes to count all the slow memory reads
>> +          on domain 1. To achieve this, the bits 4 and 5 needs to be set
>> +          which is 110000b (in hex 0x30).
>> +          ::
>> +
>> +            # echo  "1=0x30" > /sys/fs/resctrl/info/L3_MON/mbm_local_config
>> +
>> +            # cat /sys/fs/resctrl/info/L3_MON/mbm_local_config
>> +            0=0x15;1=0x30;3=0x15;4=0x15
>>  
>>  "max_threshold_occupancy":
>>  		Read/write file provides the largest value (in
>> @@ -464,6 +539,25 @@ Memory bandwidth domain is L3 cache.
>>  
>>  	MB:<cache_id0>=bw_MBps0;<cache_id1>=bw_MBps1;...
>>  
>> +Slow Memory bandwidth Allocation (SMBA)
>> +-------------------------------------------------
>> +AMD hardware can support slow Memory bandwidth Allocation feature.
>> +Currently, CXL.memory is the only supported "slow" memory device.
>> +With the support of SMBA feature the hardware enables bandwidth
>> +allocation on the slow memory devices. If there are multiple slow
>> +memory devices in the system, then the throttling logic groups all
>> +the slow sources together and applies the limit on them as a whole.
>> +
>> +The presence of the SMBA feature(with CXL.memory) is independent
>> +of whether slow memory device is actually present in the system.
>> +If there is no slow memory in the system, then setting a SMBA limit
>> +will have no impact on the performance of the system.
>> +
>> +Slow Memory bandwidth domain is L3 cache.
>> +::
>> +
>> +	SMBA:<cache_id0>=bandwidth0;<cache_id1>=bandwidth1;...
>> +
>>  Reading/writing the schemata file
>>  ---------------------------------
>>  Reading the schemata file will show the state of all resources
>> @@ -479,6 +573,44 @@ which you wish to change.  E.g.
>>    L3DATA:0=fffff;1=fffff;2=3c0;3=fffff
>>    L3CODE:0=fffff;1=fffff;2=fffff;3=fffff
>>  
>> +Reading/writing the schemata file (on AMD systems)
>> +--------------------------------------------------
>> +Reading the schemata file will show the state of all resources
>> +on all domains. When writing the memory bandwidth allocation you
>> +only need to specify those values in an absolute number expressed
>> +in 1/8 GB/s increments. To allocate bandwidth limit of 2GB, you
>> +need to specify the value 16 (16 * 1/8 = 2). For example:
>> +::
>> +
>> +  # cat schemata
>> +    MB:0=2048;1=2048;2=2048;3=2048
>> +    L3:0=ffff;1=ffff;2=ffff;3=ffff
>> +
>> +  # echo "MB:1=16" > schemata
>> +  # cat schemata
>> +    MB:0=2048;1=  16;2=2048;3=2048
>> +    L3:0=ffff;1=ffff;2=ffff;3=ffff
>> +
>> +Reading/writing the schemata file (on AMD systems) with SMBA feature
>> +-------------------------------------------------------------------
> The heading above produces htmldocs warnings:
>
> Documentation/x86/resctrl.rst:595: WARNING: Title underline too short.
>
> Reading/writing the schemata file (on AMD systems) with SMBA feature
> -------------------------------------------------------------------
> Documentation/x86/resctrl.rst:595: WARNING: Title underline too short.
>
> Reading/writing the schemata file (on AMD systems) with SMBA feature
> -------------------------------------------------------------------
>
> I have applied the fixup:
Thanks
>
> ---- >8 ----
>
> diff --git a/Documentation/x86/resctrl.rst b/Documentation/x86/resctrl.rst
> index e2a59249d18322..145916828f2bae 100644
> --- a/Documentation/x86/resctrl.rst
> +++ b/Documentation/x86/resctrl.rst
> @@ -592,7 +592,7 @@ need to specify the value 16 (16 * 1/8 = 2). For example:
>      L3:0=ffff;1=ffff;2=ffff;3=ffff
>  
>  Reading/writing the schemata file (on AMD systems) with SMBA feature
> --------------------------------------------------------------------
> +--------------------------------------------------------------------
>  Reading the schemata file will show the state of all resources
>  on all domains. When writing the memory bandwidth allocation you
>  only need to specify those values in an absolute number expressed
>
>> +Reading the schemata file will show the state of all resources
>> +on all domains. When writing the memory bandwidth allocation you
>> +only need to specify those values in an absolute number expressed
>> +in 1/8 GB/s increments. To allocate bandwidth limit of 8GB, you
>> +need to specify the value 64 (64 * 1/8 = 8).  E.g.
>> +::
>> +
>> +  # cat schemata
>> +    SMBA:0=2048;1=2048;2=2048;3=2048
>> +      MB:0=2048;1=2048;2=2048;3=2048
>> +      L3:0=ffff;1=ffff;2=ffff;3=ffff
>> +
>> +  # echo "SMBA:1=64" > schemata
>> +  # cat schemata
>> +    SMBA:0=2048;1=  64;2=2048;3=2048
>> +      MB:0=2048;1=2048;2=2048;3=2048
>> +      L3:0=ffff;1=ffff;2=ffff;3=ffff
>> +
>>  Cache Pseudo-Locking
>>  ====================
>>  CAT enables a user to specify the amount of cache space that an
>>
>>
> The rest of prose can be improved:
>
> ---- >8 ----
>
> diff --git a/Documentation/x86/resctrl.rst b/Documentation/x86/resctrl.rst
> index 145916828f2bae..92b2c4e03a4a26 100644
> --- a/Documentation/x86/resctrl.rst
> +++ b/Documentation/x86/resctrl.rst
> @@ -217,9 +217,9 @@ with the following files:
>              # cat /sys/fs/resctrl/info/L3_MON/mbm_local_config
>              0=0x15;1=0x15;3=0x15;4=0x15
>  
> -        * To change the mbm_total_bytes to count only reads on domain 0.
> -          To achieve this, the bits 0, 1, 4 and 5 needs to be set which is
> -          110011b (in hex 0x33).
> +        * To change the mbm_total_bytes to count only reads on domain 0
> +          (the bits 0, 1, 4 and 5 needs to be set, which means 110011b
> +          {in hex 0x33}):
>            ::
>  
>              # echo  "0=0x33" > /sys/fs/resctrl/info/L3_MON/mbm_total_config
> @@ -228,8 +228,8 @@ with the following files:
>              0=0x33;1=0x7f;2=0x7f;3=0x7f
>  
>          * To change the mbm_local_bytes to count all the slow memory reads
> -          on domain 1. To achieve this, the bits 4 and 5 needs to be set
> -          which is 110000b (in hex 0x30).
> +          on domain 1 (the bits 4 and 5 needs to be set, which means 110000b
> +          {in hex 0x30}):
>            ::
>  
>              # echo  "1=0x30" > /sys/fs/resctrl/info/L3_MON/mbm_local_config
> @@ -540,20 +540,21 @@ Memory bandwidth domain is L3 cache.
>  	MB:<cache_id0>=bw_MBps0;<cache_id1>=bw_MBps1;...
>  
>  Slow Memory bandwidth Allocation (SMBA)
> --------------------------------------------------
> -AMD hardware can support slow Memory bandwidth Allocation feature.
> +---------------------------------------
> +AMD hardwares support Slow Memory bandwidth Allocation (SMBA) feature.
>  Currently, CXL.memory is the only supported "slow" memory device.
> -With the support of SMBA feature the hardware enables bandwidth
> -allocation on the slow memory devices. If there are multiple slow
> -memory devices in the system, then the throttling logic groups all
> -the slow sources together and applies the limit on them as a whole.
> +With the support of SMBA, the hardware enables bandwidth allocation
> +on the slow memory devices. If there are multiple such devices in the
> +system, the throttling logic groups all the slow sources together
> +and applies the limit on them as a whole.
>  
> -The presence of the SMBA feature(with CXL.memory) is independent
> -of whether slow memory device is actually present in the system.
> -If there is no slow memory in the system, then setting a SMBA limit
> -will have no impact on the performance of the system.
> +The presence of SMBA (with CXL.memory) is independent of slow memory
> +devices presence. If there is no such devices on the system, then
> +setting the configuring SMBA will have no impact on the performance
> +of the system.
>  
> -Slow Memory bandwidth domain is L3 cache.
> +The bandwidth domain for slow memory is L3 cache. Its schemata file
> +is formatted as:
>  ::
>  
>  	SMBA:<cache_id0>=bandwidth0;<cache_id1>=bandwidth1;...
> @@ -575,11 +576,13 @@ which you wish to change.  E.g.
>  
>  Reading/writing the schemata file (on AMD systems)
>  --------------------------------------------------
> -Reading the schemata file will show the state of all resources
> -on all domains. When writing the memory bandwidth allocation you
> -only need to specify those values in an absolute number expressed
> -in 1/8 GB/s increments. To allocate bandwidth limit of 2GB, you
> -need to specify the value 16 (16 * 1/8 = 2). For example:
> +Reading the schemata file will show the current bandwidth limit on all
> +domains. The allocated resources are in multiples of one eighth GB/s.
> +When writing to the file, you need to specify what cache id you wish to
> +configure the bandwidth limit.
> +
> +For example, to allocate 2GB/s limit on the first cache id:
> +
>  ::
>  
>    # cat schemata
> @@ -593,11 +596,11 @@ need to specify the value 16 (16 * 1/8 = 2). For example:
>  
>  Reading/writing the schemata file (on AMD systems) with SMBA feature
>  --------------------------------------------------------------------
> -Reading the schemata file will show the state of all resources
> -on all domains. When writing the memory bandwidth allocation you
> -only need to specify those values in an absolute number expressed
> -in 1/8 GB/s increments. To allocate bandwidth limit of 8GB, you
> -need to specify the value 64 (64 * 1/8 = 8).  E.g.
> +Reading and writing the schemata file is the same as without SMBA in
> +above section.
> +
> +For example, to allocate 8GB/s limit on the first cache id:
> +
>  ::
>  
>    # cat schemata
>
Thanks

Babu Moger