Re: [GIT preview] for-6.3/cxl-ram-region

[Gregory Price <gregory.price@memverge.com>]

On Mon, Jan 30, 2023 at 12:10:08PM -0800, Dan Williams wrote:
> Gregory Price wrote:
> [..]
> > I found the same results.
> > echo mem0 > /sys/bus/cxl/devices/region0/target0
> > 
> > Not sure if bug/missing feature, but after attaching a device to the
> > target, you get no output when reading target0
> > 
> > ```
> > [root@fedora ~]# cat /sys/bus/cxl/devices/region0/target0
> > 
> > [root@fedora ~]#
> 
> Hmm, did you not get:
> 
> "-bash: echo: write error: Invalid argument"
> 
> ...at that step? Because targetX expects an endpoint decoder, not a
> memdev.
> 

¯\_(ツ)_/¯ bug?

I went through the kernel code and thought it was looking for a memdev
but i guess i was wrong

// ... snip ...
    dev = bus_find_device_by_name(&cxl_bus_type, NULL, buf);
    if (!dev)
      return -ENODEV;

    if (!is_endpoint_decoder(dev)) {
      rc = -EINVAL;
      goto out;
    }
// ... snip ...

bool is_endpoint_decoder(struct device *dev)
{
  return dev->type == &cxl_decoder_endpoint_type;
}
EXPORT_SYMBOL_NS_GPL(is_endpoint_decoder, CXL);


notably, memdev's do not expose their devtype, maybe something overwrote
it to be a different type?  Seems unlikely, but i'd need to attach gdb.

[root@fedora mem0]# ls
dev     firmware_version    numa_node    pmem  serial     uevent
driver  label_storage_size  payload_max  ram   subsystem



> > ```
> > echo region0 > /sys/bus/cxl/devices/decoder0.0/create_ram_region
> > echo 1 > /sys/bus/cxl/devices/region0/interleave_ways
> > echo 256 > /sys/bus/cxl/devices/region0/interleave_granularity
> > echo 0x10000000000 > /sys/bus/cxl/devices/region0/size
> > echo mem0 > /sys/bus/cxl/devices/region0/target0
> 
> > and mem0 would get 4096 memory# blocks (presumably under region/devdax?)
> 
> At 1T of size, mem0 would be hosting 4294967296 256-byte blocks.
>

Ah i thought granularity was in MB, derp derp.  Note - that's what the
root decoder interleave granularity is set to, so i just mirrored that.

[root@fedora decoder0.0]# cat interleave_granularity
256