Prototyping on CXL hardware

Prototyping on CXL hardware

[Julien BERAUD]

Hello Everyone,

I'm currently trying to prototype with a simple use-case on a CXL type 3 device
directly connected to a CXL 2.0 capable server.

Since I have seen many patches being integrated in the mainline kernel, I am 
running a 6.0.1 release.

The init phase of the drivers fails with the following trace:

cxl_mem mem0: at mem0 no parent for dport: pci0000:16

It occurs at the first iteration of the loop in devm_cxl_enumerate_ports 

The previous steps have been succeding as far as I know.

---------------------------------------------------------------------------------
So I have a few questions that I hope someone can answer here, I am sorry if they
sound naive :

- What is supposed to fill the device's ancestry, and what does it means if there
is none ?

- More generally, is a CXL type 3 device directly connected to a CXL capable
server supposed to initialize correctly on a 6.0.1 kernel or should I use a more
advanced branch ?


Thanks in advance for your answers,
Julien

Re: Prototyping on CXL hardware

[Jonathan Cameron]

On Tue, 22 Nov 2022 10:54:52 +0100
Julien BERAUD <jberaud@sasfermat.com> wrote:

> Hello Everyone,
> 

Hi Julien, Nice to 'meet' you.

> I'm currently trying to prototype with a simple use-case on a CXL type 3 device
> directly connected to a CXL 2.0 capable server.
> 
> Since I have seen many patches being integrated in the mainline kernel, I am 
> running a 6.0.1 release.
> 
> The init phase of the drivers fails with the following trace:
> 
> cxl_mem mem0: at mem0 no parent for dport: pci0000:16
> 
> It occurs at the first iteration of the loop in devm_cxl_enumerate_ports 
> 
> The previous steps have been succeding as far as I know.
> 
> ---------------------------------------------------------------------------------
> So I have a few questions that I hope someone can answer here, I am sorry if they
> sound naive :
> 
> - What is supposed to fill the device's ancestry, and what does it means if there
> is none ?
> 
> - More generally, is a CXL type 3 device directly connected to a CXL capable
> server supposed to initialize correctly on a 6.0.1 kernel or should I use a more
> advanced branch ?

To take a lazy punt in the dark as it end of day here.

Do you have an ACPI CEDT table and required ACPI stuff such as ACPI0017 in DSDT?
Otherwise the root of the created CXL topology will be missing - I haven't checked
the code paths, but that might well give the above result.

There are a few fixes you will want on top of 6.0.1 - but those are all around
region creation I think, so you aren't getting that far.
I've not been tracking progress of those. Dan, what's the current status?

For reference you could take a look at the upstream QEMU (very recent needed as we
had a fix going through on x86).  That should work and gives you something to compare
with when bringing up real hardware.

Jonathan

> 
> 
> Thanks in advance for your answers,
> Julien

Re: Prototyping on CXL hardware

[Julien BERAUD]

On 22/11/2022 17:53, Jonathan Cameron wrote:
> On Tue, 22 Nov 2022 10:54:52 +0100
> Julien BERAUD <jberaud@sasfermat.com> wrote:
> 
>> Hello Everyone,
>>
> 
> Hi Julien, Nice to 'meet' you.

Nice to meet you too.

> 
>> I'm currently trying to prototype with a simple use-case on a CXL type 3 device
>> directly connected to a CXL 2.0 capable server.
>>
>> Since I have seen many patches being integrated in the mainline kernel, I am 
>> running a 6.0.1 release.
>>
>> The init phase of the drivers fails with the following trace:
>>
>> cxl_mem mem0: at mem0 no parent for dport: pci0000:16
>>
>> It occurs at the first iteration of the loop in devm_cxl_enumerate_ports 
>>
>> The previous steps have been succeding as far as I know.
>>
>> ---------------------------------------------------------------------------------
>> So I have a few questions that I hope someone can answer here, I am sorry if they
>> sound naive :
>>
>> - What is supposed to fill the device's ancestry, and what does it means if there
>> is none ?
>>
>> - More generally, is a CXL type 3 device directly connected to a CXL capable
>> server supposed to initialize correctly on a 6.0.1 kernel or should I use a more
>> advanced branch ?
> 
> To take a lazy punt in the dark as it end of day here.
> 
> Do you have an ACPI CEDT table and required ACPI stuff such as ACPI0017 in DSDT?
> Otherwise the root of the created CXL topology will be missing - I haven't checked
> the code paths, but that might well give the above result.

Thank you very much, I was kind of suspecting this would be the issue. I'm far from
an expert on ACPI and other firmware related topics and I have to understand what
it implies on the device (we are developing the type 3 device) and possibly on the
host to add the CEDT table and "ACPI0017 in DSDT". So I'm back to my homework here.

> 
> There are a few fixes you will want on top of 6.0.1 - but those are all around
> region creation I think, so you aren't getting that far.
> I've not been tracking progress of those. Dan, what's the current status?
> 
> For reference you could take a look at the upstream QEMU (very recent needed as we
> had a fix going through on x86).  That should work and gives you something to compare
> with when bringing up real hardware.
> 

It is exactly what I am currently trying to do. I'm having trouble finding out the
proper set of options on the master branch or 7.0.1 of qemu but I have been able to
boot with an emulated type 3 device on the (rather old) cxl-2.0v4 on
https://gitlab.com/bwidawsk/qemu.git.
Again I'm back to my homework here, glad to know I'm kind of looking in the right
direction.

> Jonathan
> 
>>
>>
>> Thanks in advance for your answers,
>> Julien
> 

Cheers,
Julien