Re: [PATCH 0/6] cxl: Initialization reworks in support Soft Reserve Recovery and Accelerator Memory

[Alejandro Lucero Palau <alucerop@amd.com>]

On 12/5/25 21:17, dan.j.williams@intel.com wrote:
> Alejandro Lucero Palau wrote:
> [..]
>>> For "Accelerator Memory", the driver is not cxl_pci, but any potential
>>> PCI driver that wants to use the devm_cxl_add_memdev() ABI to attach to
>>> the CXL memory domain. Those drivers want to know if the CXL link is
>>> live end-to-end (from endpoint, through switches, to the host bridge)
>>> and CXL memory operations are enabled. If not, a CXL accelerator may be
>>> able to fall back to PCI-only operation. Similar to the "Soft Reserve
>>> Memory" it needs to know that the CXL subsystem had a chance to probe
>>> the ancestor topology of the device and let that driver make a
>>> synchronous decision about CXL operation.
>>
>> IMO, this is not the problem with accelerators, because this can not be
>> dynamically done, or not easily.
> Hmm, what do you mean can not be dynamically done? The observation is
> that a CXL card and its driver have no idea if the card is going to be
> plugged into a PCIe only slot.


Right.


>
> At runtime the driver only finds out the CXL is not there from the
> result of devm_cxl_add_memdev().


If there is no CXL properly initialized, what also implies a PCI-only 
slot, the driver can know looking at the CXL.mem and CXL.cache status in 
the CXL control register. That is what sfc driver does now using Terry's 
patchset instead of only checking CXL DVSEC and trying further CXL 
initialization using the CXL core API for Type2. Neither call to create 
cxl dev state nor memdev is needed to figure out. Of course, those calls 
can point to another kind of problem, but the driver can find out 
without using them.


>> The HW will support CXL or PCI, and if
>> CXL mem is not enabled by the firmware, likely due to a
>> negotiation/linking problem, the driver can keep going with CXL.io.
> Right, I think we are in violent agreement.
>
>> Of course, this is from my experience with sfc driver/hardware. Note
>> sfc driver added the check for CXL availability based on Terry's v13.
> Note that Terry's check for CXL availabilty is purely a hardware
> detection, there are still software reasons why cxl_acpi and cxl_mem
> can prevent devm_cxl_add_memdev() success.
>
>> But this is useful for solving the problem of module removal which can
>> leave the type2 driver without the base for doing any unwinding. Once a
>> type2 uses code from those other cxl modules explicitly, the problem is
>> avoided. You seem to have forgotten about this problem, what I think it
>> is worth to describe.
> What problem exactly? If it needs to be captured in these changelogs or
> code comments, let me know.


It is a surprise you not remembering this ...


v17 tried to fix this problem which was pointed out in v16 by you in 
several patches.


v17:

https://lore.kernel.org/linux-cxl/6887b72724173_11968100cb@dwillia2-mobl4.notmuch/


Next my reply to another comment from you trying to clarify/enumerate 
different problems which were getting intertwined creating confusion (at 
least to me). Sadly none did comment further, likely none read my 
explanation ... even if I asked for it with another email and 
specifically in one community meeting:

https://lore.kernel.org/linux-cxl/836d06d6-a36f-4ba3-b7c9-ba8687ba2190@amd.com/


Next discussion about trying to solve the modules removal adding a 
callback by the driver which you did not like:

https://lore.kernel.org/linux-cxl/6892325deccdb_55f09100fb@dwillia2-xfh.jf.intel.com.notmuch/


Here, v16, you stated specifically about cxl kernel modules removed 
while a Type2 driver is using cxl:

https://lore.kernel.org/linux-cxl/682e2a0b9b15b_1626e10088@dwillia2-xfh.jf.intel.com.notmuch/

https://lore.kernel.org/linux-cxl/682e300371a0_1626e1003@dwillia2-xfh.jf.intel.com.notmuch/

https://lore.kernel.org/linux-cxl/682e3f3343977_1626e100b0@dwillia2-xfh.jf.intel.com.notmuch/