Re: dax behavior on CXL 1.1 hosts

[Dan Williams <dan.j.williams@intel.com>]

Yiannis Nikolakopoulos wrote:
> Hi,
> 
> I am trying to understand the dax behavior on CXL 1.1 hosts running
> recent kernels and what is that I have probably misconfigured.
> To my understanding this series [1] introduced some different
> behavior, and I am trying to figure out what I am getting wrong here
> (but of course I might as well be looking in a completely wrong
> place).
[..]
> When running with Kernel 6.2.16 (or older) the dax device appears as
> expected, and I can either map it or configure it as system ram. No
> issues there.
[..]
> What is it that I am missing here?

Starting around v6.3 the CXL subsystem started attempting to takeover
dax device registration.

    09d09e04d2fc cxl/dax: Create dax devices for CXL RAM regions

I.e. instead of simply relying on memory-map information, "dax_hmem",
let the CXL subsystem assemble a cxl_region which outputs a dax-device
on the backend.

The rationale for this is that a CXL Region enables RAS flows that raw
memory map enumeration does not.

The problem though is what happens when the CXL subsytem fails to parse
the configuration. In that case you end up with neither a CXL Region, or
the the original "raw" dax_hmem device.

There has been a slow drip of fixes to get the CXL subsystem to
understand all the various platform quirks contributing to CXL Region
assembly failure. There is also efforts like this [1] in flight to
attempt to recover dax operation after a Region assembly failure.

Until then, the workaround is to disable the cxl_acpi driver from
loading. When cxl_acpi is disabled, dax_hmem proceeds to produce a raw
dax device. That configuration forfeits all the RAS support, but that is
to be expected when the CXL subsystem needs help to parse the system
topology.

[1]: http://lore.kernel.org/20250403183315.286710-1-terry.bowman@amd.com