Re: [LSF/MM] CXL Boot to Bash - Section 0: ACPI and Linux Resources

[Jonathan Cameron <Jonathan.Cameron@huawei.com>]

On Fri, 7 Mar 2025 10:12:57 -0500
Gregory Price <gourry@gourry.net> wrote:

> On Fri, Mar 07, 2025 at 10:20:31AM +0800, Yuquan Wang wrote:
> > > 2a) Is the BIOS programming decoders, or are you programming the
> > >     decoder after boot?  
> > Program the decoder after boot. It seems like currently bios for qemu could
> > not programm cxl both on x86(q35) and arm64(virt). I am trying to find a
> > cxl-enable bios for qemu virt to do some test.  
> 
> What's likely happening here then is that QEMU is not emitting an SRAT
> (either because the logic is missing or by design). 
> 
> From other discussions, this may be the intention of the GenPort work,
> which is intended to have placeholders in the SRAT for the Proximity
> Domains for devices to be initialized later (i.e. dynamically).
> 

For QEMU you need to provide a whole bunch of config to get SRAT / HMAT
etc ( and BIOS never configures the stuff, it's all OS first.

I wrote a slightly pathological test case that should give the general idea
https://elixir.bootlin.com/qemu/v9.2.2/source/tests/qtest/bios-tables-test.c#L1940
It flushed out a few bugs :)

    test_acpi_one(" -machine hmat=on,cxl=on"
                  " -smp 3,sockets=3"
                  " -m 128M,maxmem=384M,slots=2"
                  " -device pcie-root-port,chassis=1,id=pci.1"
                  " -device pci-testdev,bus=pci.1,"
                  "multifunction=on,addr=00.0"
                  " -device pci-testdev,bus=pci.1,addr=00.1"
                  " -device pci-testdev,bus=pci.1,id=gidev,addr=00.2"
                  " -device pxb-cxl,bus_nr=64,bus=pcie.0,id=cxl.1"
                  " -object memory-backend-ram,size=64M,id=ram0"
                  " -object memory-backend-ram,size=64M,id=ram1"
                  " -numa node,nodeid=0,cpus=0,memdev=ram0"
                  " -numa node,nodeid=1"
                  " -object acpi-generic-initiator,id=gi0,pci-dev=gidev,node=1"
                  " -numa node,nodeid=2"
                  " -object acpi-generic-port,id=gp0,pci-bus=cxl.1,node=2"
                  " -numa node,nodeid=3,cpus=1"
                  " -numa node,nodeid=4,memdev=ram1"
                  " -numa node,nodeid=5,cpus=2"
                  " -numa hmat-lb,initiator=0,target=0,hierarchy=memory,"
                  "data-type=access-latency,latency=10"
                  " -numa hmat-lb,initiator=0,target=0,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=800M"
                  " -numa hmat-lb,initiator=0,target=2,hierarchy=memory,"
                  "data-type=access-latency,latency=100"
                  " -numa hmat-lb,initiator=0,target=2,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=200M"
                  " -numa hmat-lb,initiator=0,target=4,hierarchy=memory,"
                  "data-type=access-latency,latency=100"
                  " -numa hmat-lb,initiator=0,target=4,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=200M"
                  " -numa hmat-lb,initiator=0,target=5,hierarchy=memory,"
                  "data-type=access-latency,latency=200"
                  " -numa hmat-lb,initiator=0,target=5,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=400M"
                  " -numa hmat-lb,initiator=1,target=0,hierarchy=memory,"
                  "data-type=access-latency,latency=500"
                  " -numa hmat-lb,initiator=1,target=0,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=100M"
                  " -numa hmat-lb,initiator=1,target=2,hierarchy=memory,"
                  "data-type=access-latency,latency=50"
                  " -numa hmat-lb,initiator=1,target=2,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=400M"
                  " -numa hmat-lb,initiator=1,target=4,hierarchy=memory,"
                  "data-type=access-latency,latency=50"
                  " -numa hmat-lb,initiator=1,target=4,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=800M"
                  " -numa hmat-lb,initiator=1,target=5,hierarchy=memory,"
                  "data-type=access-latency,latency=500"
                  " -numa hmat-lb,initiator=1,target=5,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=100M"
                  " -numa hmat-lb,initiator=3,target=0,hierarchy=memory,"
                  "data-type=access-latency,latency=20"
                  " -numa hmat-lb,initiator=3,target=0,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=400M"
                  " -numa hmat-lb,initiator=3,target=2,hierarchy=memory,"
                  "data-type=access-latency,latency=80"
                  " -numa hmat-lb,initiator=3,target=2,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=200M"
                  " -numa hmat-lb,initiator=3,target=4,hierarchy=memory,"
                  "data-type=access-latency,latency=80"
                  " -numa hmat-lb,initiator=3,target=4,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=200M"
                  " -numa hmat-lb,initiator=3,target=5,hierarchy=memory,"
                  "data-type=access-latency,latency=20"
                  " -numa hmat-lb,initiator=3,target=5,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=400M"
                  " -numa hmat-lb,initiator=5,target=0,hierarchy=memory,"
                  "data-type=access-latency,latency=20"
                  " -numa hmat-lb,initiator=5,target=0,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=400M"
                  " -numa hmat-lb,initiator=5,target=2,hierarchy=memory,"
                  "data-type=access-latency,latency=80"
                  " -numa hmat-lb,initiator=5,target=4,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=200M"
                  " -numa hmat-lb,initiator=5,target=4,hierarchy=memory,"
                  "data-type=access-latency,latency=80"
                  " -numa hmat-lb,initiator=5,target=2,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=200M"
                  " -numa hmat-lb,initiator=5,target=5,hierarchy=memory,"
                  "data-type=access-latency,latency=10"
                  " -numa hmat-lb,initiator=5,target=5,hierarchy=memory,"
                  "data-type=access-bandwidth,bandwidth=800M",
                  &data);

Jonathan


> ~Gregory
>