Re: [RFC PATCH v1] cxl: add support for cxl reset

[Alejandro Lucero Palau <alucerop@amd.com>]

On 12/23/24 23:49, Srirangan Madhavan wrote:
>>>> What happens if there are current cxl regions mapped to the device being
>>>> reset?  I don't think it is enough to flush the caches.  Section 9.7
>>>> talks about system software requirements for the HDMs.  How are those
>>>> requirements met with this patch?
>>> Considering different platforms might have specific operations to
>>> effectively clear the regions, some type of optional helper routines in the
>>> CXL core would be appropriate.  This way if required, a Type 2 device driver
>>> for example can choose to use it if required.
>> There are a couple of ways to go here.  For memory devices I think this is
>> on the user.  But I don't think that is a primary use case as I'm still
>> unclear of why a user would need to do any reset of the memory device.
>>
>> For type 2 devices I think the use case will be device dependant.  The
>> specific driver in that case will need to tear down their regions (with or
>> without user space coordination) and then issue the reset.
> That makes sense. So in either case, based on this logic, I am assuming
> it is out of scope for this patch to remove the the memory regions.
>
>> All that said I think this patch belongs with a series which implements
>> some device support.  Or is more general for type 2.  Which is what
>> Alejandro (cc'ed) is working on.
> Thanks for the feedback.
> @Alejandro, if you have any thoughts on this, kindly let me know.
> I can proceed based on that.


Sorry for the late reply.


I thought about this some time ago and shared my concerns about current 
CXL error management which should lead to a CXL reset in some situations.


Apart from that not being there yet, how to do the reset has the problem 
of the restoration of device HDM registers. I think the kernel CXL core 
should keep that information before the reset, and restore them back 
when the reset is done. Other possibility is to just unroll the HDM 
which implies involving all CXL switches in the path plus the CXL Root 
Complex. I do not know if there are other considerations for supporting 
restoration. For a, I hope, near future, CXL.cache should be included 
here, but it is not clear yet how to support it, so even less clear for 
a reset.


Of course, CXL errors can be more problematic than with PCIe, and a 
reset could imply the system (some cores) being stuck, so all this is 
quite tricky.


In the specific case I'm adding along with the generic CXL Type2 
support, if the system is stable after a detected CXL error, the device 
HDM should be configured after the reset, by implicitly some code linked 
to the reset triggered by the driver, what could be quite similar to the 
initial CXL probe, but if restoration is an option, the CXL core keeping 
the same ranges and somehow giving that back to the device.


I hope this helps somehow.

Thank you

Alejandro


> Regards,
> Srirangan.
>
>