Access to CXL memory

Access to CXL memory

[Dengcheng Zhu]

Hi,

I learned that CXL memory could be exposed to the system as a NUMA
node as well as an mmap()able device.

Two questions are:

- Can these 2 forms coexist at the same time?
- If yes, how does memory management work between these 2 interfaces?

Please shed some light upon this issue.

Thanks,
Dengcheng

RE: Access to CXL memory

[Dai WeiBin (David Dai)]

I understand BIOS will allocate system physical region for your CXL device when system enumeration.
Afterward BIOS writes the SPA to CXL base address register, meanwhile it also creates SRAT to Linux OS.
After OS handoff, OS sees a SRAT and a PCIe device, the PCIe device has CXL function.
So Linux kernel can parse SRAT table to get memory information and put it into Linux mem block as system RAM.
if the SRAT set the memory reserved, that means Linux buddy system doesn't manage it, so CXL device driver may read cxl base address register to get HPA and size, then map it as private managed memory.

Anyway, you must ensure only one master manages the memory, as well as only CXL type2/type3 has volatile memory which can be managed by host system or driver.

Thanks,
David  

-----Original Message-----
From: Dengcheng Zhu (dengcheng.zhu@gmail.com) [mailto:dengcheng.zhu@gmail.com] 
Sent: 2021年5月18日 2:51
To: ben.widawsky@intel.com; dan.j.williams@intel.com
Cc: linux-cxl@vger.kernel.org
Subject: Access to CXL memory


Hi,

I learned that CXL memory could be exposed to the system as a NUMA
node as well as an mmap()able device.

Two questions are:

- Can these 2 forms coexist at the same time?
- If yes, how does memory management work between these 2 interfaces?

Please shed some light upon this issue.

Thanks,
Dengcheng