Re: [PATCH v4 4/4] docs: add document to introduce CDP command

[Ian Campbell <ian.campbell@citrix.com>]

On Thu, 2015-09-17 at 17:35 +0800, He Chen wrote:
> Add new CDP options with CAT commands in xl interface man page.
> Add description of CDP in xl-psr.markdown.

It would have been fine to include this in the previous patch by the way.

> 
> Signed-off-by: He Chen <he.chen@linux.intel.com>
> ---
>  docs/man/xl.pod.1         | 14 ++++++++++++++
>  docs/misc/xl-psr.markdown | 44 +++++++++++++++++++++++++++++++++++++++--
> ---
>  2 files changed, 53 insertions(+), 5 deletions(-)
> 
> diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> index f22c3f3..3c7107d 100644
> --- a/docs/man/xl.pod.1
> +++ b/docs/man/xl.pod.1
> @@ -1530,6 +1530,12 @@ applications. In the Xen implementation, CAT is
> used to control cache allocation
>  on VM basis. To enforce cache on a specific domain, just set capacity
> bitmasks
>  (CBM) for the domain.
>  
> +Intel Broadwell and later server platforms also offer Code/Data Prioritization
> +(CDP) for cache allocation, which support

                   allocations,

>  specifying code or data cache for

It seems like a word is missing, perhaps "size" as in "code or data case
size"? Or perhaps "priority"?

> +applications. CDP is used on VM basis in the Xen implementation. To specify

"used on a per VM basis"

> +code or data CBM for the domain, CDP feature must be enabled and CBM type
> +options need to be specified when setting CBM.

I asked on patch 3 whether these options were mutually exclusive or not,
the answer should be reflected in the documentation too please.

> diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown
> index 3545912..5eb97cc 100644
> --- a/docs/misc/xl-psr.markdown
> +++ b/docs/misc/xl-psr.markdown
> @@ -14,7 +14,7 @@ tracks cache utilization of memory accesses according
> to the RMID and reports
>  monitored data via a counter register.
>  
>  For more detailed information please refer to Intel SDM chapter
> -"17.14 - Platform Shared Resource Monitoring: Cache Monitoring Technology".
> +"17.15 - Platform Shared Resource Monitoring: Cache Monitoring Technology".

Was the chapter number just wrong, or are these things prone to changing?

If the latter then we should either omit them or we need to refer to a
specific revision of the SDM as well.

>  
>  In Xen's implementation, each domain in the system can be assigned a
> RMID
>  independently, while RMID=0 is reserved for monitoring domains that
> don't
> @@ -91,17 +91,42 @@ For example, assuming a system with 8 portions and 3
> domains:
>     first domain exclusive access to half the cache, and the other two
> exclusive
>     access to one quarter each.
>  
> -For more detailed information please refer to Intel SDM chapter
> -"17.15 - Platform Shared Resource Control: Cache Allocation Technology".
> -


>  In Xen's implementation, CBM can be configured with libxl/xl interfaces but
>  COS is maintained in hypervisor only. The cache partition granularity is per
>  domain, each domain has COS=0 assigned by default, the corresponding CBM is
>  all-ones, which means all the cache resource can be used by default.
>  
> +Code/Data Prioritization (CDP) Technology is an extension of CAT, which is
> +available on Intel Broadwell and later server platforms. CDP enables isolation
> +and separate prioritization of code and data fetches to the L3 cache in a
> +software configurable manner, which can enable workload prioritization and
> +tuning of cache capacity to the characteristics of the workload. CDP extends
> +Cache Allocation Technology (CAT) by providing separate code and data masks
> +per Class of Service (COS).

Looks good.

> +
> +CDP can be enabled by adding `psr=cdp` to Xen bootline.

"bootline" => "command line"

> +
> +When CDP is enabled,
> +
> + * the CAT masks are re-mapped into interleaved pairs of masks for data or code
> +   fetches.
> +
> + * the range of COS for CAT is re-indexed, with the lower-half of the COS
> +   range available for CDP.
> +
> +CDP allows OS or Hypervisor to partition cache allocation more fine-grained,

             ^the                                           ^in a          

> +code cache and data cache can be specified respectively. With CDP enabled,

End fine-grained with a full-stop and then:

    The code cache and data cache can be specified separately

or s/separately/independently/?

(I think, I'm not 100% sure what you meant by "respectively", so maybe the
suggestion is wrong)

> +one COS corresponds to two CBMs (code CBM & data CBM), since the sum of CBMs
> +is fixed, that means the number of available COSes will reduce to half when

                                                           reduce by half

Ian.