RE: [PATCH v2] cxl/region: use region (not root decoder) granularity for calculations

[Dan Williams <dan.j.williams@intel.com>]

Jim Harris wrote:
> Root decoder granularity must match value from CFWMS, which may not
> be the region's granularity for non-interleaved root decoders.
> 
> So when calculating granularities for host bridge decoders, use the
> region's granularity instead of the root decoder's granularity to ensure
> the correct granularities are set for the host bridge decoders and any
> downstream switch decoders.
> 
> Test configuration is 1 host bridge * 2 switches * 2 endpoints per switch.
> 
> Region created with 2048 granularity using following command line:
> 
> cxl create-region -m -d decoder0.0 -w 4 mem0 mem2 mem1 mem3 \
> 		  -g 2048 -s 2048M
> 
> Use "cxl list -PDE | grep granularity" to get a view of the granularity
> set at each level of the topology.
> 
> Before this patch:
>         "interleave_granularity":2048,
>         "interleave_granularity":2048,
>     "interleave_granularity":512,
>         "interleave_granularity":2048,
>         "interleave_granularity":2048,
>     "interleave_granularity":512,
> "interleave_granularity":256,
> 
> After:
>         "interleave_granularity":2048,
>         "interleave_granularity":2048,
>     "interleave_granularity":4096,
>         "interleave_granularity":2048,
>         "interleave_granularity":2048,
>     "interleave_granularity":4096,
> "interleave_granularity":2048,
> 
> Signed-off-by: Jim Harris <jim.harris@samsung.com>
> ---
>  drivers/cxl/core/region.c |    9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
> index 6d63b8798c29..c25be26d0bdf 100644
> --- a/drivers/cxl/core/region.c
> +++ b/drivers/cxl/core/region.c
> @@ -1133,7 +1133,14 @@ static int cxl_port_setup_targets(struct cxl_port *port,
>  	}
>  
>  	if (is_cxl_root(parent_port)) {
> -		parent_ig = cxlrd->cxlsd.cxld.interleave_granularity;
> +		/*
> +		 * Root decoder IG is always set to value in CFMWS which
> +		 * may be different than this region's IG.  We can use the
> +		 * region's IG here since interleave_granularity_store()
> +		 * does not allow interleaved host-bridges with
> +		 * root IG != region IG.
> +		 */
> +		parent_ig = p->interleave_granularity;
>  		parent_iw = cxlrd->cxlsd.cxld.interleave_ways;

This causes the kernel to correctly detect that cxl_test was programming
an incorrect host-bridge level granularity value. Fixup the test.

diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c
index fb6ab9cef84f..b88546299902 100644
--- a/tools/testing/cxl/test/cxl.c
+++ b/tools/testing/cxl/test/cxl.c
@@ -831,7 +831,7 @@ static void mock_init_hdm_decoder(struct cxl_decoder *cxld)
                        cxld->interleave_ways = 2;
                else
                        cxld->interleave_ways = 1;
-               cxld->interleave_granularity = 256;
+               cxld->interleave_granularity = 4096;
                cxld->hpa_range = (struct range) {
                        .start = base,
                        .end = base + size - 1,