Re: [PATCH v2 4/4] acpi/ghes, cxl/pci: Trace FW-First CXL Protocol Errors

[Smita Koralahalli <Smita.KoralahalliChannabasappa@amd.com>]

On 10/2/2024 5:16 PM, Dan Williams wrote:
> Smita Koralahalli wrote:
>> When PCIe AER is in FW-First, OS should process CXL Protocol errors from
>> CPER records.
>>
>> Reuse the existing work queue cxl_cper_work registered with GHES to notify
>> the CXL subsystem on a Protocol error.
>>
>> The defined trace events cxl_aer_uncorrectable_error and
>> cxl_aer_correctable_error currently trace native CXL AER errors. Reuse
>> them to trace FW-First Protocol Errors.
>>
>> Signed-off-by: Smita Koralahalli <Smita.KoralahalliChannabasappa@amd.com>
>> ---
>> v2:
>> 	Removed pr_warn for serial number.
>> 	p_err -> rec/p_rec.
>> ---
>>   drivers/acpi/apei/ghes.c | 14 ++++++++++++++
>>   drivers/cxl/core/pci.c   | 24 ++++++++++++++++++++++++
>>   drivers/cxl/cxlpci.h     |  3 +++
>>   drivers/cxl/pci.c        | 20 ++++++++++++++++++--
>>   include/cxl/event.h      |  1 +
>>   5 files changed, 60 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c
>> index 9dcf0f78458f..5082885e1f2c 100644
>> --- a/drivers/acpi/apei/ghes.c
>> +++ b/drivers/acpi/apei/ghes.c
>> @@ -723,6 +723,20 @@ static void cxl_cper_handle_prot_err(struct acpi_hest_generic_data *gdata)
>>   
>>   	if (cxl_cper_handle_prot_err_info(gdata, &wd.p_rec))
>>   		return;
>> +
>> +	guard(spinlock_irqsave)(&cxl_cper_work_lock);
>> +
>> +	if (!cxl_cper_work)
>> +		return;
>> +
>> +	wd.event_type = CXL_CPER_EVENT_PROT_ERR;
>> +
>> +	if (!kfifo_put(&cxl_cper_fifo, wd)) {
>> +		pr_err_ratelimited("CXL CPER kfifo overflow\n");
>> +		return;
>> +	}
>> +
>> +	schedule_work(cxl_cper_work);
> 
> The cxl_cper_work item is only for cases where the cxl_pci driver might care
> about annotating an error report with driver specific details like the
> impacted kernel object name, 'struct cxl_memdev', or address translation
> for DPA data.
> 
> Protocol errors that are not endpoint errors should never be placed in
> the cxl_cper_fifo. That is exclusively for errors that cxl_pci needs to
> consume.
> 
> My expectation is that similar to aer_recover_queue for PCIe protocol
> errors CXL needs to grow a cxl_recover_queue that at a minimum triggers
> new trace events to dump these records to RAS daemon.
> 
> I am struggling to think what useful information cxl_pci could ever
> append to a protocol error event.
> 
> What is more likely is that later when Terry adds port error handling a
> CPER protocol error record could trigger a new cxl_do_recovery() to
> react to CXL topology errors that might impact downstream CXL devices.
> In that case the notification will come through something like a new
> 'struct cxl_error_handlers *' hanging off 'struct pci_driver' since
> accelerator drivers are going to have distinct error handling from
> generic memory expanders.

Hmm, I agree handling device and protocol errors separately. Unless 
Terry has something to add here, I can start working on a minimal 
version of cxl_recover_queue as of now.

Thanks
Smita
>