Re: [LSF/MM] CXL Boot to Bash - Section 0: ACPI and Linux Resources

[Yuquan Wang <wangyuquan1236@phytium.com.cn>]

On Thu, Mar 06, 2025 at 12:08:49PM -0500, Gregory Price wrote:
> On Thu, Mar 06, 2025 at 09:37:49AM +0800, Yuquan Wang wrote:
> > On Wed, Mar 05, 2025 at 05:20:52PM -0500, Gregory Price wrote:
> 
> First, thank you for bringing this up, this is exactly the type of
> ambiguiuty i was hoping others would contribute.  It's difficult to
> figure out if the ACPI tables are "Correct", if there's unimplemented
> features, or we're doing something wrong - because some of this is
> undocumented theory of operation.
> 
Thank you for your patience in replying my questions. :)
> > > ==================
> > > NUMA node creation
> > > ===================
> > > NUMA nodes are *NOT* hot-pluggable.  All *POSSIBLE* NUMA nodes are
> > > identified at `__init` time, more specifically during `mm_init`.
> > > 
> > > What this means is that the CEDT and SRAT must contain sufficient
> > > `proximity domain` information for linux to identify how many NUMA
> > > nodes are required (and what memory regions to associate with them).
> > > 
> > Condition:
> > 1) A UMA/NUMA system that SRAT is absence, but it keeps CEDT.CFMWS
> > 2）Enable CONFIG_ACPI_NUMA
> > 
> > Results:
> > 1) acpi_numa_init: the fake_pxm will be 0 and send to acpi_parse_cfmws()
> > 2）If dynamically create cxl ram region, the cxl memory would be assigned
> > to node0 rather than a fake new node.
> >
> 
> This is very interesting.  Can I ask a few questions:
> 
> 1) is this real hardware or a VM?
Qemu VM (arm64 virt).
> 2) By `dynamic creation` you mean leveraging cxl-cli (ndctl)?
Yes. After boot, I used "cxl create-region". 
> 2a) Is the BIOS programming decoders, or are you programming the
>     decoder after boot?
Program the decoder after boot. It seems like currently bios for qemu could
not programm cxl both on x86(q35) and arm64(virt). I am trying to find a
cxl-enable bios for qemu virt to do some test.
> 
> 
> > Confusions:
> > 1) Does CXL memory usage require a numa system with SRAT? As you
> > mentioned in SRAT section: 
> > 
> > "This table is technically optional, but for performance information
> > to be enumerated by linux it must be present."
> > 
> > Hence, as I understand it, it seems a bug in kernel.
> >
> 
> It's hard to say if this is a bug yet.  It's either a bug, or your
> system should have an SRAT to describe what the BIOS has done.
> 
> > 2) If it is a bug, could  we forbid this situation by adding fake_pxm
> > check and returning error in acpi_numa_init()?
> > 
> 
> > 3）If not,  maybe we can add some kernel logic to allow create these fake
> > nodes on a system without SRAT?
> > 
> 
> I think we should at least provide a warning (if the SRAT is expected
> but missing) - but lets get some more information first.
> 
> ~Gregory