Re: [GIT preview] for-6.3/cxl-ram-region

[Gregory Price <gregory.price@memverge.com>]

On Tue, Jan 31, 2023 at 05:59:26PM +0000, Verma, Vishal L wrote:
> On Mon, 2023-01-30 at 21:00 -0500, Gregory Price wrote:
> > On Mon, Jan 30, 2023 at 03:18:38PM -0800, Dan Williams wrote:
> > 
> > 
> > This worked for me, though it took me a bit to figure out how to wire
> > everythign up - still not sure this is entirely correct but this is the
> > string of commands that were required to successfully attach an endpoint
> > decoder to the root decoder.
> > 
> > Note: The root decoder has interleave_(granularity=256, ways=1), and the
> > region code appears to require the same granularity?  Does that mean
> > we're stuck to 256b granularity? (unless i misread the code, which i'm
> > about 75% sure i am).
> > 
> > 
> > Command string:
> > 
> > # Program the endpoint decoder for ram of size 1GB
> > echo ram > /sys/bus/cxl/devices/decoder2.0/mode
> > echo 0x40000000 > /sys/bus/cxl/devices/decoder2.0/dpa_size
> > 
> > # Create a region in the root decoder
> > echo region0 > /sys/bus/cxl/devices/decoder0.0/create_ram_region
> > 
> > # Configure that region with the same interleave granularity
> > # and ways as the root and endpoint decoders
> > echo 256 > /sys/bus/cxl/devices/region0/interleave_granularity
> > echo 1 > /sys/bus/cxl/devices/region0/interleave_ways
> > echo 0x40000000 > /sys/bus/cxl/devices/region0/size
> > 
> > # Link the endpoint decoder as a target in the region
> > echo decoder2.0 > /sys/bus/cxl/devices/region0/target0
> > 
> > # Commit the changes
> > echo 1 > /sys/bus/cxl/devices/region0/commit
> 
> I've pushed a cxl-cli branch[1] that incorporates this flow, and allows
> for: cxl create-region -t ram <other options>
> 
> Feel free to give it a spin!
> 
> [1]: https://github.com/pmem/ndctl/commits/vv/volatile-regions

Right now I believe this is failing due to the interleave and size not
having default values

./cxl create-region -m -t ram -d decoder0.0 -w 1 -g 4096 mem0
cxl region: create_region: create_region: unable to determine region size
cxl region: cmd_create_region: created 0 regions


appears to be due to this code
static int create_region(struct cxl_ctx *ctx, int *count,
             struct parsed_params *p)
{
// ... snip ...
    rc = create_region_validate_config(ctx, p);
    if (rc)
        return rc;

    if (p->size) {
        size = p->size;
        default_size = false;
    } else if (p->ep_min_size) {
        size = p->ep_min_size * p->ways;
**    } else {
**        log_err(&rl, "%s: unable to determine region size\n", __func__);
**        return -ENXIO;
**    }

So both size and ep_min_size are 0 here

echo region0 > /sys/bus/cxl/devices/decoder0.0/create_ram_region
cat /sys/bus/cxl/devices/region0/interleave_ways
0
cat /sys/bus/cxl/devices/region0/interleave_granularity
0
cat /sys/bus/cxl/devices/region0/size
0