* [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
@ 2026-10-26 10:03 ` Yukiya Takaki via
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Ani Sinha, Anisa Su,
Igor Mammedov, Jonathan Cameron, Junjie Cao, Paolo Bonzini,
Peter Maydell, Richard Henderson, linux-cxl, qemu-arm,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
Hello everyone,
I’m a student intern, and this is my first time sending patches to an open source mailing list.
This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
Background and motivation
Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
allowing the Linux CXL driver to discover and create regions during boot.
This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
Some parts of this implementation are based on Li, Zhijian’s work in:
https://github.com/zhijianli88/qemu/tree/program-decoder-v2
I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
Approach
This patch first adds a program_hdm_decoder option to cxl-type3.
When program-hdm-decoder is enabled on cxl-type3, QEMU will:
1) program endpoint/host-bridge HDM decoder state,
2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
3) create memory-only NUMA nodes for those ranges,
4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
into SeaBIOS would introduce additional complexity and risk inconsistencies.
Testing
We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
and E820_SOFT_RESERVED entries appeared in the E820 table:
- BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
- CXL memory: 4 GB
- CFMWS: 4 GB
- Host bridge: 1
The following is the startup script.
sudo /path/to/qemu/build/qemu-system-x86_64 \
-name guest=cxl \
-nographic \
-machine q35,accel=tcg,nvdimm=on,cxl=on \
-cpu max -smp 4,sockets=4,cores=1,threads=1 \
-m size=4G,slots=10,maxmem=20G \
-drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
-bios /path/to/seabios/out/bios.bin \
-nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
-object memory-backend-ram,size=2G,id=m0 \
-object memory-backend-ram,size=2G,id=m1 \
-numa node,nodeid=0,cpus=0-1,memdev=m0 \
-numa node,nodeid=1,cpus=2-3,memdev=m1 \
-device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
-object memory-backend-ram,size=4G,share=on,id=vmem0 \
-device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
Furthermore, we conducted the experiment in which we added two CXL Type3.
- CXL memory: 4 GB × 2
- CFMWS: 4 GB × 2
The following is the startup script.
sudo /path/to/qemu/build/qemu-system-x86_64 \
-name guest=cxl \
-nographic \
-machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
-cpu max -smp 4,sockets=4,cores=1,threads=1 \
-m size=4G,slots=10,maxmem=20G \
-drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
-bios /path/to/seabios/out/bios.bin \
-nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
-object memory-backend-ram,size=2G,id=m0 \
-object memory-backend-ram,size=2G,id=m1 \
-numa node,nodeid=0,cpus=0-1,memdev=m0 \
-numa node,nodeid=1,cpus=2-3,memdev=m1 \
-device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
-device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
-device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
-object memory-backend-ram,size=4G,share=on,id=vmem0 \
-object memory-backend-ram,size=4G,share=on,id=vmem1 \
-device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
-device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
How to use
Specify the following parameters when starting QEMU:
- bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
- Add hmat=on to the -machine option [1]
- Add program-hdm-decoder=true to each -device cxl-type3 option
[1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
so HMAT must be enabled.
QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
TODO
With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
As a result, automatic creation of CXL regions in the guest OS may fail:
- A CXL persistent memory device is included
- Interleaved configurations
- Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
- Configurations where CXL memory is attached via a switch
- Multi-target CFMWS environments
- ARM environments
- HMAT does not yet include information about CXL memory devices
The following behaviors are also provisional implementations and may need further refinement:
- A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
or to allow the NUMA node IDs to be specified by the user.
- Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
adding an explicit option to associate each Type 3 device with a CFMWS entry,
or generating CFMWS entries automatically from the configured CXL topology and devices.
- HDM decoder reset sequencing and state reconstruction.
The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
by moving the common initialization of the host bridge component registers from the reset path to the realization path.
However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
automatically programmed by QEMU should be reconstructed as part of reset processing.
Thanks,
------
Yukiya Takaki (7):
QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
add CXL Type 3 memory ranges to SRAT
hw/acpi/cxl_type3.c | 30 +++++
hw/acpi/meson.build | 1 +
hw/arm/cxl.c | 10 ++
hw/arm/meson.build | 2 +
hw/cxl/cxl-host.c | 63 ++++++++++
hw/i386/acpi-build.c | 7 ++
hw/i386/cxl.c | 11 ++
hw/i386/meson.build | 1 +
hw/i386/pc.c | 2 +
hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
hw/pci-bridge/pci_expander_bridge.c | 13 +-
include/hw/cxl/cxl.h | 2 +
include/hw/cxl/cxl_device.h | 6 +
include/hw/cxl/cxl_host.h | 3 +
include/hw/mem/cxl_type3.h | 8 ++
15 files changed, 334 insertions(+), 2 deletions(-)
create mode 100644 hw/acpi/cxl_type3.c
create mode 100644 hw/arm/cxl.c
create mode 100644 hw/i386/cxl.c
create mode 100644 include/hw/mem/cxl_type3.h
--
2.52.0
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
@ 2026-10-26 10:03 ` Yukiya Takaki via
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Ani Sinha, Anisa Su,
Igor Mammedov, Jonathan Cameron, Junjie Cao, Paolo Bonzini,
Peter Maydell, Richard Henderson, linux-cxl, qemu-arm,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
Hello everyone,
I’m a student intern, and this is my first time sending patches to an open source mailing list.
This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
Background and motivation
Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
allowing the Linux CXL driver to discover and create regions during boot.
This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
Some parts of this implementation are based on Li, Zhijian’s work in:
https://github.com/zhijianli88/qemu/tree/program-decoder-v2
I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
Approach
This patch first adds a program_hdm_decoder option to cxl-type3.
When program-hdm-decoder is enabled on cxl-type3, QEMU will:
1) program endpoint/host-bridge HDM decoder state,
2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
3) create memory-only NUMA nodes for those ranges,
4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
into SeaBIOS would introduce additional complexity and risk inconsistencies.
Testing
We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
and E820_SOFT_RESERVED entries appeared in the E820 table:
- BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
- CXL memory: 4 GB
- CFMWS: 4 GB
- Host bridge: 1
The following is the startup script.
sudo /path/to/qemu/build/qemu-system-x86_64 \
-name guest=cxl \
-nographic \
-machine q35,accel=tcg,nvdimm=on,cxl=on \
-cpu max -smp 4,sockets=4,cores=1,threads=1 \
-m size=4G,slots=10,maxmem=20G \
-drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
-bios /path/to/seabios/out/bios.bin \
-nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
-object memory-backend-ram,size=2G,id=m0 \
-object memory-backend-ram,size=2G,id=m1 \
-numa node,nodeid=0,cpus=0-1,memdev=m0 \
-numa node,nodeid=1,cpus=2-3,memdev=m1 \
-device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
-object memory-backend-ram,size=4G,share=on,id=vmem0 \
-device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
Furthermore, we conducted the experiment in which we added two CXL Type3.
- CXL memory: 4 GB × 2
- CFMWS: 4 GB × 2
The following is the startup script.
sudo /path/to/qemu/build/qemu-system-x86_64 \
-name guest=cxl \
-nographic \
-machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
-cpu max -smp 4,sockets=4,cores=1,threads=1 \
-m size=4G,slots=10,maxmem=20G \
-drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
-bios /path/to/seabios/out/bios.bin \
-nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
-object memory-backend-ram,size=2G,id=m0 \
-object memory-backend-ram,size=2G,id=m1 \
-numa node,nodeid=0,cpus=0-1,memdev=m0 \
-numa node,nodeid=1,cpus=2-3,memdev=m1 \
-device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
-device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
-device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
-device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
-object memory-backend-ram,size=4G,share=on,id=vmem0 \
-object memory-backend-ram,size=4G,share=on,id=vmem1 \
-device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
-device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
-M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
How to use
Specify the following parameters when starting QEMU:
- bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
- Add hmat=on to the -machine option [1]
- Add program-hdm-decoder=true to each -device cxl-type3 option
[1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
so HMAT must be enabled.
QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
TODO
With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
As a result, automatic creation of CXL regions in the guest OS may fail:
- A CXL persistent memory device is included
- Interleaved configurations
- Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
- Configurations where CXL memory is attached via a switch
- Multi-target CFMWS environments
- ARM environments
- HMAT does not yet include information about CXL memory devices
The following behaviors are also provisional implementations and may need further refinement:
- A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
or to allow the NUMA node IDs to be specified by the user.
- Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
adding an explicit option to associate each Type 3 device with a CFMWS entry,
or generating CFMWS entries automatically from the configured CXL topology and devices.
- HDM decoder reset sequencing and state reconstruction.
The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
by moving the common initialization of the host bridge component registers from the reset path to the realization path.
However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
automatically programmed by QEMU should be reconstructed as part of reset processing.
Thanks,
------
Yukiya Takaki (7):
QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
add CXL Type 3 memory ranges to SRAT
hw/acpi/cxl_type3.c | 30 +++++
hw/acpi/meson.build | 1 +
hw/arm/cxl.c | 10 ++
hw/arm/meson.build | 2 +
hw/cxl/cxl-host.c | 63 ++++++++++
hw/i386/acpi-build.c | 7 ++
hw/i386/cxl.c | 11 ++
hw/i386/meson.build | 1 +
hw/i386/pc.c | 2 +
hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
hw/pci-bridge/pci_expander_bridge.c | 13 +-
include/hw/cxl/cxl.h | 2 +
include/hw/cxl/cxl_device.h | 6 +
include/hw/cxl/cxl_host.h | 3 +
include/hw/mem/cxl_type3.h | 8 ++
15 files changed, 334 insertions(+), 2 deletions(-)
create mode 100644 hw/acpi/cxl_type3.c
create mode 100644 hw/arm/cxl.c
create mode 100644 hw/i386/cxl.c
create mode 100644 include/hw/mem/cxl_type3.h
--
2.52.0
^ permalink raw reply [flat|nested] 24+ messages in thread* [PATCH RFC 1/7] QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
2026-10-26 10:03 ` Yukiya Takaki via
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Anisa Su, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
Introduce a `program-hdm-decoder` property for CXL Type 3 devices.
When this property is enabled, QEMU will program the HDM decoder so that
the guest can automatically create CXL regions for Type 3 memory at boot.
The property is a boolean and is exposed as `program-hdm-decoder` on the
`cxl-type3` device. It defaults to `false`, so existing configurations are
not affected unless the user explicitly enables it.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 2 ++
include/hw/cxl/cxl_device.h | 2 ++
2 files changed, 4 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index a04dc74ecc..b65ddd264d 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1360,6 +1360,8 @@ static const Property ct3_props[] = {
DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND,
HostMemoryBackend *),
DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
+ DEFINE_PROP_BOOL("program-hdm-decoder", CXLType3Dev,
+ program_hdm_decoder, false),
DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0),
DEFINE_PROP_LINK("volatile-dc-memdev", CXLType3Dev, dc.host_dc,
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index ba551fa5f9..a2c830d181 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -777,6 +777,8 @@ struct CXLType3Dev {
/* BI flows */
bool hdmdb;
+ bool program_hdm_decoder;
+
struct dynamic_capacity {
HostMemoryBackend *host_dc;
AddressSpace host_dc_as;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 1/7] QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Anisa Su, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
Introduce a `program-hdm-decoder` property for CXL Type 3 devices.
When this property is enabled, QEMU will program the HDM decoder so that
the guest can automatically create CXL regions for Type 3 memory at boot.
The property is a boolean and is exposed as `program-hdm-decoder` on the
`cxl-type3` device. It defaults to `false`, so existing configurations are
not affected unless the user explicitly enables it.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 2 ++
include/hw/cxl/cxl_device.h | 2 ++
2 files changed, 4 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index a04dc74ecc..b65ddd264d 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1360,6 +1360,8 @@ static const Property ct3_props[] = {
DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND,
HostMemoryBackend *),
DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
+ DEFINE_PROP_BOOL("program-hdm-decoder", CXLType3Dev,
+ program_hdm_decoder, false),
DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0),
DEFINE_PROP_LINK("volatile-dc-memdev", CXLType3Dev, dc.host_dc,
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index ba551fa5f9..a2c830d181 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -777,6 +777,8 @@ struct CXLType3Dev {
/* BI flows */
bool hdmdb;
+ bool program_hdm_decoder;
+
struct dynamic_capacity {
HostMemoryBackend *host_dc;
AddressSpace host_dc_as;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH RFC 2/7] QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
2026-10-26 10:03 ` Yukiya Takaki via
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Anisa Su, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
E820, SRAT and HDM decoder programming in this series all need per‑device
host physical address (HPA) information for CXL Type 3 memory. Today,
CXLType3Dev does not expose this in a convenient way.
This patch adds minimal state to CXLHost and CXLType3Dev, plus a helper
to walk from a Type 3 device back to its host, so that each CXL Type 3
device can be assigned a stable HPA offset within the CFMW window and an
index that later code can rely on when building firmware tables and
regions.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 37 +++++++++++++++++++++++++++++++++++++
include/hw/cxl/cxl.h | 2 ++
include/hw/cxl/cxl_device.h | 2 ++
3 files changed, 41 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index b65ddd264d..d08a4255c9 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -732,9 +732,39 @@ static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
}
}
+
+static CXLHost *ct3d_to_cxl_pxb(DeviceState *dev)
+{
+ BusState *bus = dev->parent_bus;
+
+ DeviceState *cxl_rp_dev = NULL;
+ if (bus && bus->parent) {
+ cxl_rp_dev = bus->parent;
+ }
+ assert(cxl_rp_dev);
+
+ DeviceState *cxl_host_dev = NULL;
+ if (cxl_rp_dev) {
+ BusState *rp_bus = cxl_rp_dev->parent_bus;
+ if (rp_bus && rp_bus->parent) {
+ cxl_host_dev = rp_bus->parent;
+ }
+ }
+ assert(cxl_host_dev);
+
+ if (cxl_host_dev &&
+ object_dynamic_cast(OBJECT(cxl_host_dev), TYPE_PXB_CXL_HOST)) {
+ CXLHost *host = PXB_CXL_HOST(cxl_host_dev);
+ return host;
+ }
+
+ return NULL;
+}
+
static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
{
DeviceState *ds = DEVICE(ct3d);
+ CXLHost *host = ct3d_to_cxl_pxb(ds);
if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem
&& !ct3d->dc.num_regions) {
@@ -816,6 +846,13 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
g_free(p_name);
}
+ if (ct3d->program_hdm_decoder) {
+ ct3d->hpa_offset = host->unused_hpa_offset;
+ ct3d->idx = host->unused_dev_num;
+ host->unused_hpa_offset += ct3d->cxl_dstate.static_mem_size;
+ host->unused_dev_num++;
+ }
+
ct3d->dc.total_capacity = 0;
if (ct3d->dc.num_regions > 0) {
MemoryRegion *dc_mr;
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index 7cf3dc622c..b8b3071f81 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -67,6 +67,8 @@ struct CXLHost {
CXLComponentState cxl_cstate;
bool passthrough;
+ hwaddr unused_hpa_offset;
+ int unused_dev_num;
};
#define TYPE_PXB_CXL_HOST "pxb-cxl-host"
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index a2c830d181..c3a729f8c5 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -778,6 +778,8 @@ struct CXLType3Dev {
bool hdmdb;
bool program_hdm_decoder;
+ hwaddr hpa_offset;
+ int idx;
struct dynamic_capacity {
HostMemoryBackend *host_dc;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 2/7] QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Anisa Su, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
E820, SRAT and HDM decoder programming in this series all need per‑device
host physical address (HPA) information for CXL Type 3 memory. Today,
CXLType3Dev does not expose this in a convenient way.
This patch adds minimal state to CXLHost and CXLType3Dev, plus a helper
to walk from a Type 3 device back to its host, so that each CXL Type 3
device can be assigned a stable HPA offset within the CFMW window and an
index that later code can rely on when building firmware tables and
regions.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 37 +++++++++++++++++++++++++++++++++++++
include/hw/cxl/cxl.h | 2 ++
include/hw/cxl/cxl_device.h | 2 ++
3 files changed, 41 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index b65ddd264d..d08a4255c9 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -732,9 +732,39 @@ static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
}
}
+
+static CXLHost *ct3d_to_cxl_pxb(DeviceState *dev)
+{
+ BusState *bus = dev->parent_bus;
+
+ DeviceState *cxl_rp_dev = NULL;
+ if (bus && bus->parent) {
+ cxl_rp_dev = bus->parent;
+ }
+ assert(cxl_rp_dev);
+
+ DeviceState *cxl_host_dev = NULL;
+ if (cxl_rp_dev) {
+ BusState *rp_bus = cxl_rp_dev->parent_bus;
+ if (rp_bus && rp_bus->parent) {
+ cxl_host_dev = rp_bus->parent;
+ }
+ }
+ assert(cxl_host_dev);
+
+ if (cxl_host_dev &&
+ object_dynamic_cast(OBJECT(cxl_host_dev), TYPE_PXB_CXL_HOST)) {
+ CXLHost *host = PXB_CXL_HOST(cxl_host_dev);
+ return host;
+ }
+
+ return NULL;
+}
+
static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
{
DeviceState *ds = DEVICE(ct3d);
+ CXLHost *host = ct3d_to_cxl_pxb(ds);
if (!ct3d->hostmem && !ct3d->hostvmem && !ct3d->hostpmem
&& !ct3d->dc.num_regions) {
@@ -816,6 +846,13 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
g_free(p_name);
}
+ if (ct3d->program_hdm_decoder) {
+ ct3d->hpa_offset = host->unused_hpa_offset;
+ ct3d->idx = host->unused_dev_num;
+ host->unused_hpa_offset += ct3d->cxl_dstate.static_mem_size;
+ host->unused_dev_num++;
+ }
+
ct3d->dc.total_capacity = 0;
if (ct3d->dc.num_regions > 0) {
MemoryRegion *dc_mr;
diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
index 7cf3dc622c..b8b3071f81 100644
--- a/include/hw/cxl/cxl.h
+++ b/include/hw/cxl/cxl.h
@@ -67,6 +67,8 @@ struct CXLHost {
CXLComponentState cxl_cstate;
bool passthrough;
+ hwaddr unused_hpa_offset;
+ int unused_dev_num;
};
#define TYPE_PXB_CXL_HOST "pxb-cxl-host"
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index a2c830d181..c3a729f8c5 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -778,6 +778,8 @@ struct CXLType3Dev {
bool hdmdb;
bool program_hdm_decoder;
+ hwaddr hpa_offset;
+ int idx;
struct dynamic_capacity {
HostMemoryBackend *host_dc;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH RFC 3/7] QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
2026-10-26 10:03 ` Yukiya Takaki via
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Anisa Su, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
The previously added `hpa_offset` on CXLType3Dev records the offset of a
device within its CFMW window, but there is no simple helper to obtain the
full host physical address (HPA) range for a given Type 3 device.
This patch adds `ct3_type3_get_hpm_range()`, which returns the HPA base and
size for a CXLType3Dev by combining the CFMW base, the per‑device offset
and the device size. The helper is declared in the header so that other
code can use it when building tables and regions.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 40 +++++++++++++++++++++++++++++++++++++
include/hw/cxl/cxl_device.h | 1 +
2 files changed, 41 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index d08a4255c9..811f8e247d 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1358,6 +1358,46 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
return address_space_write(as, dpa_offset, attrs, &data, size);
}
+static uint64_t c3t_addr_size(DeviceState *dev)
+{
+ CXLType3Dev *ct3d = CXL_TYPE3(dev);
+ MemoryRegion *vmr = host_memory_backend_get_memory(ct3d->hostvmem);
+
+ return memory_region_size(vmr);
+}
+
+static uint64_t c3t_hpa_base(DeviceState *dev)
+{
+ GSList *cfmws_list, *iter;
+
+ cfmws_list = cxl_fmws_get_all_sorted();
+ for (iter = cfmws_list; iter; iter = iter->next) {
+ struct CXLFixedWindow *fw;
+
+ if (!object_dynamic_cast(iter->data, TYPE_CXL_FMW)) {
+ continue;
+ }
+ fw = CXL_FMW(iter->data);
+
+ PXBCXLDev *pxb = fw->target_hbs[0];
+ if (ct3d_to_cxl_pxb(dev) != pxb->cxl_host_bridge) {
+ continue;
+ }
+
+ g_slist_free(cfmws_list);
+ return fw->base;
+ }
+ g_slist_free(cfmws_list);
+
+ return 0;
+}
+
+void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size)
+{
+ *base = c3t_hpa_base(DEVICE(dev)) + dev->hpa_offset;
+ *size = c3t_addr_size(DEVICE(dev));
+}
+
static void ct3d_reset(DeviceState *dev)
{
CXLType3Dev *ct3d = CXL_TYPE3(dev);
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index c3a729f8c5..07654f4b62 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -878,6 +878,7 @@ void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
uint64_t len);
bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
uint64_t len);
+void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size);
void cxl_assign_event_header(CXLEventRecordHdr *hdr,
const QemuUUID *uuid, uint32_t flags,
uint8_t length, uint64_t timestamp,
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 3/7] QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Anisa Su, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
The previously added `hpa_offset` on CXLType3Dev records the offset of a
device within its CFMW window, but there is no simple helper to obtain the
full host physical address (HPA) range for a given Type 3 device.
This patch adds `ct3_type3_get_hpm_range()`, which returns the HPA base and
size for a CXLType3Dev by combining the CFMW base, the per‑device offset
and the device size. The helper is declared in the header so that other
code can use it when building tables and regions.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 40 +++++++++++++++++++++++++++++++++++++
include/hw/cxl/cxl_device.h | 1 +
2 files changed, 41 insertions(+)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index d08a4255c9..811f8e247d 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1358,6 +1358,46 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
return address_space_write(as, dpa_offset, attrs, &data, size);
}
+static uint64_t c3t_addr_size(DeviceState *dev)
+{
+ CXLType3Dev *ct3d = CXL_TYPE3(dev);
+ MemoryRegion *vmr = host_memory_backend_get_memory(ct3d->hostvmem);
+
+ return memory_region_size(vmr);
+}
+
+static uint64_t c3t_hpa_base(DeviceState *dev)
+{
+ GSList *cfmws_list, *iter;
+
+ cfmws_list = cxl_fmws_get_all_sorted();
+ for (iter = cfmws_list; iter; iter = iter->next) {
+ struct CXLFixedWindow *fw;
+
+ if (!object_dynamic_cast(iter->data, TYPE_CXL_FMW)) {
+ continue;
+ }
+ fw = CXL_FMW(iter->data);
+
+ PXBCXLDev *pxb = fw->target_hbs[0];
+ if (ct3d_to_cxl_pxb(dev) != pxb->cxl_host_bridge) {
+ continue;
+ }
+
+ g_slist_free(cfmws_list);
+ return fw->base;
+ }
+ g_slist_free(cfmws_list);
+
+ return 0;
+}
+
+void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size)
+{
+ *base = c3t_hpa_base(DEVICE(dev)) + dev->hpa_offset;
+ *size = c3t_addr_size(DEVICE(dev));
+}
+
static void ct3d_reset(DeviceState *dev)
{
CXLType3Dev *ct3d = CXL_TYPE3(dev);
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index c3a729f8c5..07654f4b62 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -878,6 +878,7 @@ void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
uint64_t len);
bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
uint64_t len);
+void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size);
void cxl_assign_event_header(CXLEventRecordHdr *hdr,
const QemuUUID *uuid, uint32_t flags,
uint8_t length, uint64_t timestamp,
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH RFC 3/7] QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
2026-10-26 10:03 ` Yukiya Takaki via qemu development
(?)
@ 2026-10-28 1:09 ` Anisa Su
2026-10-29 0:17 ` Yukiya Takaki (Fujitsu)
-1 siblings, 1 reply; 24+ messages in thread
From: Anisa Su @ 2026-10-28 1:09 UTC (permalink / raw)
To: Yukiya Takaki
Cc: qemu-devel, Jonathan Cameron, Junjie Cao, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
On Mon, Oct 26, 2026 at 07:03:24PM +0900, Yukiya Takaki wrote:
> The previously added `hpa_offset` on CXLType3Dev records the offset of a
> device within its CFMW window, but there is no simple helper to obtain the
> full host physical address (HPA) range for a given Type 3 device.
>
> This patch adds `ct3_type3_get_hpm_range()`, which returns the HPA base and
> size for a CXLType3Dev by combining the CFMW base, the per‑device offset
> and the device size. The helper is declared in the header so that other
> code can use it when building tables and regions.
>
> Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
> Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
> ---
> hw/mem/cxl_type3.c | 40 +++++++++++++++++++++++++++++++++++++
> include/hw/cxl/cxl_device.h | 1 +
> 2 files changed, 41 insertions(+)
>
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index d08a4255c9..811f8e247d 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -1358,6 +1358,46 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
> return address_space_write(as, dpa_offset, attrs, &data, size);
> }
>
> +static uint64_t c3t_addr_size(DeviceState *dev)
> +{
> + CXLType3Dev *ct3d = CXL_TYPE3(dev);
> + MemoryRegion *vmr = host_memory_backend_get_memory(ct3d->hostvmem);
> +
I think the reason program-hdm-decoder=true fails on CXL PMEM
topologies as you mentioned in the cover letter is because this function only
calculates size from ct3d->hostvmem, which will pass NULL to
memory_region_size() here.
RAM and PMEM size is already calculated in cxl_setup_memory() in
ct3d->cxl_dstate.static_mem_size
I think this helper can be deleted by directly using static_mem_size in
ct3_type3_get_hpm_range().
Or if you prefer not to support PMEM for this feature at all, you can
reject it in patch 2:
static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) {
...
if (ct3d->program_hdm_decoder &&
(!ct3d->hostvmem || ct3d->hostpmem || ct3d->dc.num_regions)) {
error_setg(errp, "program-hdm-decoder requires volatile-memdev "
"and no persistent or dynamic capacity");
return false;
}
}
> + return memory_region_size(vmr);
> +}
> +
> +static uint64_t c3t_hpa_base(DeviceState *dev)
> +{
> + GSList *cfmws_list, *iter;
> +
> + cfmws_list = cxl_fmws_get_all_sorted();
> + for (iter = cfmws_list; iter; iter = iter->next) {
> + struct CXLFixedWindow *fw;
> +
> + if (!object_dynamic_cast(iter->data, TYPE_CXL_FMW)) {
> + continue;
> + }
> + fw = CXL_FMW(iter->data);
> +
> + PXBCXLDev *pxb = fw->target_hbs[0];
> + if (ct3d_to_cxl_pxb(dev) != pxb->cxl_host_bridge) {
> + continue;
> + }
> +
> + g_slist_free(cfmws_list);
> + return fw->base;
> + }
> + g_slist_free(cfmws_list);
> +
> + return 0;
> +}
> +
> +void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size)
> +{
> + *base = c3t_hpa_base(DEVICE(dev)) + dev->hpa_offset;
> + *size = c3t_addr_size(DEVICE(dev));
*size = dev->cxl_dstate.static_mem_size;
Thanks,
Anisa
> +}
> +
> static void ct3d_reset(DeviceState *dev)
> {
> CXLType3Dev *ct3d = CXL_TYPE3(dev);
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index c3a729f8c5..07654f4b62 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -878,6 +878,7 @@ void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
> uint64_t len);
> bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
> uint64_t len);
> +void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size);
> void cxl_assign_event_header(CXLEventRecordHdr *hdr,
> const QemuUUID *uuid, uint32_t flags,
> uint8_t length, uint64_t timestamp,
> --
> 2.52.0
>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH RFC 3/7] QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
2026-10-28 1:09 ` Anisa Su
@ 2026-10-29 0:17 ` Yukiya Takaki (Fujitsu)
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki (Fujitsu) @ 2026-10-29 0:17 UTC (permalink / raw)
To: Anisa Su
Cc: qemu-devel@nongnu.org, Jonathan Cameron, Junjie Cao,
linux-cxl@vger.kernel.org, Tomasz Wolski (Fujitsu),
Zhijian Li (Fujitsu), Yasunori Goto (Fujitsu),
yukiya.takaki@gmail.com
Thank you for pointing this out.
My intention with this implementation was to support volatile memory only, since I have not been able to verify how PMEM is exposed to the OS by firmware on real hardware.
However, I agree that the implementation should explicitly reject unsupported PMEM configurations rather than dereferencing a NULL pointer.
I will address this issue in v2.
________________________________________
From: Anisa Su <anisa.su887@gmail.com>
Sent: Wednesday, October 28, 2026 10:09
To: Takaki, Yukiya/高木 優希矢 <fj2110jc@fujitsu.com>
Cc: qemu-devel@nongnu.org <qemu-devel@nongnu.org>; Jonathan Cameron <jic23@kernel.org>; Junjie Cao <junjie.cao@intel.com>; linux-cxl@vger.kernel.org <linux-cxl@vger.kernel.org>; Wolski, Tomasz <tomasz.wolski@fujitsu.com>; Li, Zhijian/李 智坚 <lizhijian@fujitsu.com>; Goto, Yasunori/五島 康文 <y-goto@fujitsu.com>; yukiya.takaki@gmail.com <yukiya.takaki@gmail.com>
Subject: Re: [PATCH RFC 3/7] QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
On Mon, Oct 26, 2026 at 07:03:24PM +0900, Yukiya Takaki wrote:
> The previously added `hpa_offset` on CXLType3Dev records the offset of a
> device within its CFMW window, but there is no simple helper to obtain the
> full host physical address (HPA) range for a given Type 3 device.
>
> This patch adds `ct3_type3_get_hpm_range()`, which returns the HPA base and
> size for a CXLType3Dev by combining the CFMW base, the per‑device offset
> and the device size. The helper is declared in the header so that other
> code can use it when building tables and regions.
>
> Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
> Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
> ---
> hw/mem/cxl_type3.c | 40 +++++++++++++++++++++++++++++++++++++
> include/hw/cxl/cxl_device.h | 1 +
> 2 files changed, 41 insertions(+)
>
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index d08a4255c9..811f8e247d 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -1358,6 +1358,46 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
> return address_space_write(as, dpa_offset, attrs, &data, size);
> }
>
> +static uint64_t c3t_addr_size(DeviceState *dev)
> +{
> + CXLType3Dev *ct3d = CXL_TYPE3(dev);
> + MemoryRegion *vmr = host_memory_backend_get_memory(ct3d->hostvmem);
> +
I think the reason program-hdm-decoder=true fails on CXL PMEM
topologies as you mentioned in the cover letter is because this function only
calculates size from ct3d->hostvmem, which will pass NULL to
memory_region_size() here.
RAM and PMEM size is already calculated in cxl_setup_memory() in
ct3d->cxl_dstate.static_mem_size
I think this helper can be deleted by directly using static_mem_size in
ct3_type3_get_hpm_range().
Or if you prefer not to support PMEM for this feature at all, you can
reject it in patch 2:
static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp) {
...
if (ct3d->program_hdm_decoder &&
(!ct3d->hostvmem || ct3d->hostpmem || ct3d->dc.num_regions)) {
error_setg(errp, "program-hdm-decoder requires volatile-memdev "
"and no persistent or dynamic capacity");
return false;
}
}
> + return memory_region_size(vmr);
> +}
> +
> +static uint64_t c3t_hpa_base(DeviceState *dev)
> +{
> + GSList *cfmws_list, *iter;
> +
> + cfmws_list = cxl_fmws_get_all_sorted();
> + for (iter = cfmws_list; iter; iter = iter->next) {
> + struct CXLFixedWindow *fw;
> +
> + if (!object_dynamic_cast(iter->data, TYPE_CXL_FMW)) {
> + continue;
> + }
> + fw = CXL_FMW(iter->data);
> +
> + PXBCXLDev *pxb = fw->target_hbs[0];
> + if (ct3d_to_cxl_pxb(dev) != pxb->cxl_host_bridge) {
> + continue;
> + }
> +
> + g_slist_free(cfmws_list);
> + return fw->base;
> + }
> + g_slist_free(cfmws_list);
> +
> + return 0;
> +}
> +
> +void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size)
> +{
> + *base = c3t_hpa_base(DEVICE(dev)) + dev->hpa_offset;
> + *size = c3t_addr_size(DEVICE(dev));
*size = dev->cxl_dstate.static_mem_size;
Thanks,
Anisa
> +}
> +
> static void ct3d_reset(DeviceState *dev)
> {
> CXLType3Dev *ct3d = CXL_TYPE3(dev);
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index c3a729f8c5..07654f4b62 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -878,6 +878,7 @@ void ct3_clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
> uint64_t len);
> bool ct3_test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
> uint64_t len);
> +void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size);
> void cxl_assign_event_header(CXLEventRecordHdr *hdr,
> const QemuUUID *uuid, uint32_t flags,
> uint8_t length, uint64_t timestamp,
> --
> 2.52.0
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH RFC 4/7] QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
2026-10-26 10:03 ` Yukiya Takaki via
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, linux-cxl, Wolski, Tomasz, lizhijian, y-goto,
yukiya.takaki
On some real systems, firmware programs the HDM decoders on both the
endpoint and the CXL host bridge so that the OS can discover CXL memory
ranges at boot and the CXL driver can automatically create regions.
This patch adds helpers to program the HDM decoders in QEMU using the
per‑device HPA range from `ct3_type3_get_hpm_range()`, and wires them
into the CXL Type 3 reset path. When `program_hdm_decoder` is enabled,
both the endpoint and host‑bridge HDM decoders are configured so that
the guest sees the same kind of automatically assigned CXL regions as
on real hardware.
The reset hold phase processes child devices before their parents.
Consequently, the Type 3 reset handler programs the host bridge decoder
before the pxb-cxl reset handler runs. Calling
cxl_component_register_init_common() from the pxb-cxl reset handler
would then clear the component register block and erase the decoder
configuration that was just programmed.
When the affected HPA ranges are advertised through E820 and SRAT,
which will be added in the following patches, Linux attempts to use them
as system memory. Without a valid host bridge decoder, accesses to
these ranges cannot be routed to the Type 3 devices. This corrupts
kernel memory-management data and eventually causes a kernel panic.
Initialize the common host bridge component registers when the PXB is
realized instead. Keep the reset handler limited to the
topology-dependent HDM capability setup so that the automatically
programmed decoder settings are not overwritten.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 98 +++++++++++++++++++++++++++++
hw/pci-bridge/pci_expander_bridge.c | 13 +++-
2 files changed, 109 insertions(+), 2 deletions(-)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 811f8e247d..60e4c0e20f 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -19,6 +19,9 @@
#include "hw/pci/pci.h"
#include "hw/core/qdev-properties.h"
#include "hw/core/qdev-properties-system.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/cxl/cxl_host.h"
#include "qapi/error.h"
#include "qemu/log.h"
#include "qemu/module.h"
@@ -1358,6 +1361,59 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
return address_space_write(as, dpa_offset, attrs, &data, size);
}
+static void __update_hdm_decoder_n(uint32_t *cache_mem, int which,
+ uint32_t ctrl, uint64_t base,
+ uint64_t size)
+{
+ int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
+
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + which * hdm_inc,
+ base & 0xffffffff);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + which * hdm_inc,
+ base >> 32);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + which * hdm_inc,
+ size & 0xffffffff);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + which * hdm_inc,
+ size >> 32);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl);
+}
+
+static void update_hdm_decoder_endpoint(uint32_t *cache_mem, int which,
+ uint32_t ctrl, uint64_t base,
+ uint64_t size)
+{
+ return __update_hdm_decoder_n(cache_mem, which, ctrl, base, size);
+}
+
+static void update_hdm_decoder_pxb(uint32_t *cache_mem, int which,
+ uint64_t tl, uint32_t ctrl,
+ uint64_t base, uint64_t size)
+{
+ int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
+
+ stl_le_p(cache_mem +
+ R_CXL_HDM_DECODER0_TARGET_LIST_LO + which * hdm_inc,
+ tl & 0xffffffff);
+ stl_le_p(cache_mem +
+ R_CXL_HDM_DECODER0_TARGET_LIST_HI + which * hdm_inc,
+ tl >> 32);
+
+ __update_hdm_decoder_n(cache_mem, which, ctrl, base, size);
+}
+
+static uint8_t ct3d_root_port_number(DeviceState *dev)
+{
+ BusState *bus = dev->parent_bus;
+ DeviceState *rp_dev;
+
+ assert(bus);
+ rp_dev = bus->parent;
+ assert(rp_dev);
+ assert(object_dynamic_cast(OBJECT(rp_dev), TYPE_PCIE_ROOT_PORT));
+
+ return PCIE_PORT(rp_dev)->port;
+}
+
static uint64_t c3t_addr_size(DeviceState *dev)
{
CXLType3Dev *ct3d = CXL_TYPE3(dev);
@@ -1398,6 +1454,42 @@ void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size)
*size = c3t_addr_size(DEVICE(dev));
}
+static void program_pxb_hdm_decoder(DeviceState *dev)
+{
+ CXLHost *cxl = ct3d_to_cxl_pxb(dev);
+ CXLType3Dev *ct3d = CXL_TYPE3(dev);
+ int hdm_num = ct3d->idx;
+ int order = ct3d_root_port_number(dev);
+ assert(cxl);
+ uint64_t base, size;
+ ct3_type3_get_hpm_range(ct3d, &base, &size);
+
+ CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
+ uint32_t *cache_mem = cxl_cstate->crb.cache_mem_registers;
+
+ uint32_t ctrl = R_CXL_HDM_DECODER0_CTRL_COMMIT_MASK |
+ R_CXL_HDM_DECODER0_CTRL_COMMITTED_MASK |
+ R_CXL_HDM_DECODER0_CTRL_TYPE_MASK;
+
+ update_hdm_decoder_pxb(cache_mem, hdm_num, order, ctrl, base, size);
+}
+
+static void program_hdm_decoder(DeviceState *dev)
+{
+ CXLType3Dev *ct3d = CXL_TYPE3(dev);
+ uint64_t base, size;
+ ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
+ uint32_t *cache_mem = cregs->cache_mem_registers;
+
+ ct3_type3_get_hpm_range(ct3d, &base, &size);
+
+ uint32_t ctrl = R_CXL_HDM_DECODER0_CTRL_COMMIT_MASK |
+ R_CXL_HDM_DECODER0_CTRL_COMMITTED_MASK |
+ R_CXL_HDM_DECODER0_CTRL_TYPE_MASK;
+
+ update_hdm_decoder_endpoint(cache_mem, 0, ctrl, base, size);
+}
+
static void ct3d_reset(DeviceState *dev)
{
CXLType3Dev *ct3d = CXL_TYPE3(dev);
@@ -1408,6 +1500,12 @@ static void ct3d_reset(DeviceState *dev)
ct3d->flitmode);
cxl_component_register_init_common(reg_state, write_msk,
CXL2_TYPE3_DEVICE, ct3d->hdmdb);
+
+ if (ct3d->program_hdm_decoder) {
+ program_hdm_decoder(dev);
+ program_pxb_hdm_decoder(dev);
+ }
+
cxl_device_register_init_t3(ct3d, CXL_T3_MSIX_MBOX);
/*
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 40ffbc4e08..e8fd7343d1 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -297,16 +297,24 @@ static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
return pin - PCI_SLOT(pxb->devfn);
}
+static void pxb_cxl_dev_init(DeviceState *dev)
+{
+ CXLHost *cxl = PXB_CXL_DEV(dev)->cxl_host_bridge;
+ CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
+ uint32_t *reg_state = cxl_cstate->crb.cache_mem_registers;
+ uint32_t *write_msk = cxl_cstate->crb.cache_mem_regs_write_mask;
+
+ cxl_component_register_init_common(reg_state, write_msk, CXL2_RC, false);
+}
+
static void pxb_cxl_dev_reset(DeviceState *dev)
{
CXLHost *cxl = PXB_CXL_DEV(dev)->cxl_host_bridge;
CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
PCIHostState *hb = PCI_HOST_BRIDGE(cxl);
uint32_t *reg_state = cxl_cstate->crb.cache_mem_registers;
- uint32_t *write_msk = cxl_cstate->crb.cache_mem_regs_write_mask;
int dsp_count = 0;
- cxl_component_register_init_common(reg_state, write_msk, CXL2_RC, false);
/*
* The CXL specification allows for host bridges with no HDM decoders
* if they only have a single root port.
@@ -507,6 +515,7 @@ static void pxb_cxl_dev_realize(PCIDevice *dev, Error **errp)
if (!pxb_dev_realize_common(dev, CXL, errp)) {
return;
}
+ pxb_cxl_dev_init(DEVICE(dev));
pxb_cxl_dev_reset(DEVICE(dev));
}
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 4/7] QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, linux-cxl, Wolski, Tomasz, lizhijian, y-goto,
yukiya.takaki
On some real systems, firmware programs the HDM decoders on both the
endpoint and the CXL host bridge so that the OS can discover CXL memory
ranges at boot and the CXL driver can automatically create regions.
This patch adds helpers to program the HDM decoders in QEMU using the
per‑device HPA range from `ct3_type3_get_hpm_range()`, and wires them
into the CXL Type 3 reset path. When `program_hdm_decoder` is enabled,
both the endpoint and host‑bridge HDM decoders are configured so that
the guest sees the same kind of automatically assigned CXL regions as
on real hardware.
The reset hold phase processes child devices before their parents.
Consequently, the Type 3 reset handler programs the host bridge decoder
before the pxb-cxl reset handler runs. Calling
cxl_component_register_init_common() from the pxb-cxl reset handler
would then clear the component register block and erase the decoder
configuration that was just programmed.
When the affected HPA ranges are advertised through E820 and SRAT,
which will be added in the following patches, Linux attempts to use them
as system memory. Without a valid host bridge decoder, accesses to
these ranges cannot be routed to the Type 3 devices. This corrupts
kernel memory-management data and eventually causes a kernel panic.
Initialize the common host bridge component registers when the PXB is
realized instead. Keep the reset handler limited to the
topology-dependent HDM capability setup so that the automatically
programmed decoder settings are not overwritten.
Co-developed-by: Li Zhijian <lizhijian@fujitsu.com>
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/mem/cxl_type3.c | 98 +++++++++++++++++++++++++++++
hw/pci-bridge/pci_expander_bridge.c | 13 +++-
2 files changed, 109 insertions(+), 2 deletions(-)
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 811f8e247d..60e4c0e20f 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -19,6 +19,9 @@
#include "hw/pci/pci.h"
#include "hw/core/qdev-properties.h"
#include "hw/core/qdev-properties-system.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/cxl/cxl_host.h"
#include "qapi/error.h"
#include "qemu/log.h"
#include "qemu/module.h"
@@ -1358,6 +1361,59 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
return address_space_write(as, dpa_offset, attrs, &data, size);
}
+static void __update_hdm_decoder_n(uint32_t *cache_mem, int which,
+ uint32_t ctrl, uint64_t base,
+ uint64_t size)
+{
+ int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
+
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + which * hdm_inc,
+ base & 0xffffffff);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + which * hdm_inc,
+ base >> 32);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + which * hdm_inc,
+ size & 0xffffffff);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + which * hdm_inc,
+ size >> 32);
+ stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl);
+}
+
+static void update_hdm_decoder_endpoint(uint32_t *cache_mem, int which,
+ uint32_t ctrl, uint64_t base,
+ uint64_t size)
+{
+ return __update_hdm_decoder_n(cache_mem, which, ctrl, base, size);
+}
+
+static void update_hdm_decoder_pxb(uint32_t *cache_mem, int which,
+ uint64_t tl, uint32_t ctrl,
+ uint64_t base, uint64_t size)
+{
+ int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
+
+ stl_le_p(cache_mem +
+ R_CXL_HDM_DECODER0_TARGET_LIST_LO + which * hdm_inc,
+ tl & 0xffffffff);
+ stl_le_p(cache_mem +
+ R_CXL_HDM_DECODER0_TARGET_LIST_HI + which * hdm_inc,
+ tl >> 32);
+
+ __update_hdm_decoder_n(cache_mem, which, ctrl, base, size);
+}
+
+static uint8_t ct3d_root_port_number(DeviceState *dev)
+{
+ BusState *bus = dev->parent_bus;
+ DeviceState *rp_dev;
+
+ assert(bus);
+ rp_dev = bus->parent;
+ assert(rp_dev);
+ assert(object_dynamic_cast(OBJECT(rp_dev), TYPE_PCIE_ROOT_PORT));
+
+ return PCIE_PORT(rp_dev)->port;
+}
+
static uint64_t c3t_addr_size(DeviceState *dev)
{
CXLType3Dev *ct3d = CXL_TYPE3(dev);
@@ -1398,6 +1454,42 @@ void ct3_type3_get_hpm_range(CXLType3Dev *dev, uint64_t *base, uint64_t *size)
*size = c3t_addr_size(DEVICE(dev));
}
+static void program_pxb_hdm_decoder(DeviceState *dev)
+{
+ CXLHost *cxl = ct3d_to_cxl_pxb(dev);
+ CXLType3Dev *ct3d = CXL_TYPE3(dev);
+ int hdm_num = ct3d->idx;
+ int order = ct3d_root_port_number(dev);
+ assert(cxl);
+ uint64_t base, size;
+ ct3_type3_get_hpm_range(ct3d, &base, &size);
+
+ CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
+ uint32_t *cache_mem = cxl_cstate->crb.cache_mem_registers;
+
+ uint32_t ctrl = R_CXL_HDM_DECODER0_CTRL_COMMIT_MASK |
+ R_CXL_HDM_DECODER0_CTRL_COMMITTED_MASK |
+ R_CXL_HDM_DECODER0_CTRL_TYPE_MASK;
+
+ update_hdm_decoder_pxb(cache_mem, hdm_num, order, ctrl, base, size);
+}
+
+static void program_hdm_decoder(DeviceState *dev)
+{
+ CXLType3Dev *ct3d = CXL_TYPE3(dev);
+ uint64_t base, size;
+ ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
+ uint32_t *cache_mem = cregs->cache_mem_registers;
+
+ ct3_type3_get_hpm_range(ct3d, &base, &size);
+
+ uint32_t ctrl = R_CXL_HDM_DECODER0_CTRL_COMMIT_MASK |
+ R_CXL_HDM_DECODER0_CTRL_COMMITTED_MASK |
+ R_CXL_HDM_DECODER0_CTRL_TYPE_MASK;
+
+ update_hdm_decoder_endpoint(cache_mem, 0, ctrl, base, size);
+}
+
static void ct3d_reset(DeviceState *dev)
{
CXLType3Dev *ct3d = CXL_TYPE3(dev);
@@ -1408,6 +1500,12 @@ static void ct3d_reset(DeviceState *dev)
ct3d->flitmode);
cxl_component_register_init_common(reg_state, write_msk,
CXL2_TYPE3_DEVICE, ct3d->hdmdb);
+
+ if (ct3d->program_hdm_decoder) {
+ program_hdm_decoder(dev);
+ program_pxb_hdm_decoder(dev);
+ }
+
cxl_device_register_init_t3(ct3d, CXL_T3_MSIX_MBOX);
/*
diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c
index 40ffbc4e08..e8fd7343d1 100644
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
@@ -297,16 +297,24 @@ static int pxb_map_irq_fn(PCIDevice *pci_dev, int pin)
return pin - PCI_SLOT(pxb->devfn);
}
+static void pxb_cxl_dev_init(DeviceState *dev)
+{
+ CXLHost *cxl = PXB_CXL_DEV(dev)->cxl_host_bridge;
+ CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
+ uint32_t *reg_state = cxl_cstate->crb.cache_mem_registers;
+ uint32_t *write_msk = cxl_cstate->crb.cache_mem_regs_write_mask;
+
+ cxl_component_register_init_common(reg_state, write_msk, CXL2_RC, false);
+}
+
static void pxb_cxl_dev_reset(DeviceState *dev)
{
CXLHost *cxl = PXB_CXL_DEV(dev)->cxl_host_bridge;
CXLComponentState *cxl_cstate = &cxl->cxl_cstate;
PCIHostState *hb = PCI_HOST_BRIDGE(cxl);
uint32_t *reg_state = cxl_cstate->crb.cache_mem_registers;
- uint32_t *write_msk = cxl_cstate->crb.cache_mem_regs_write_mask;
int dsp_count = 0;
- cxl_component_register_init_common(reg_state, write_msk, CXL2_RC, false);
/*
* The CXL specification allows for host bridges with no HDM decoders
* if they only have a single root port.
@@ -507,6 +515,7 @@ static void pxb_cxl_dev_realize(PCIDevice *dev, Error **errp)
if (!pxb_dev_realize_common(dev, CXL, errp)) {
return;
}
+ pxb_cxl_dev_init(DEVICE(dev));
pxb_cxl_dev_reset(DEVICE(dev));
}
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH RFC 5/7] QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
2026-10-26 10:03 ` Yukiya Takaki via
(?)
@ 2026-10-26 10:03 ` Yukiya Takaki via
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, Paolo Bonzini, Peter Maydell, Richard Henderson,
linux-cxl, qemu-arm, Wolski, Tomasz, lizhijian, y-goto,
yukiya.takaki
On real systems, CXL Type 3 memory is exposed to the OS via the E820
table as E820_SOFT_RESERVED, so the kernel can treat it as a dedicated
reserved range rather than normal RAM.
This patch adds a small helper and a recursive walker so that, on x86,
the HPA ranges assigned to CXL Type 3 devices are added to the E820 table
as E820_SOFT_RESERVED when `program_hdm_decoder` is enabled. An ARM stub
is also added as a placeholder for future support.
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/arm/cxl.c | 10 ++++++++++
hw/arm/meson.build | 2 ++
hw/cxl/cxl-host.c | 25 +++++++++++++++++++++++++
hw/i386/cxl.c | 11 +++++++++++
hw/i386/meson.build | 1 +
hw/i386/pc.c | 1 +
include/hw/cxl/cxl_host.h | 2 ++
7 files changed, 52 insertions(+)
create mode 100644 hw/arm/cxl.c
create mode 100644 hw/i386/cxl.c
diff --git a/hw/arm/cxl.c b/hw/arm/cxl.c
new file mode 100644
index 0000000000..1cbcf2d8e5
--- /dev/null
+++ b/hw/arm/cxl.c
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "exec/hwaddr.h"
+#include "hw/cxl/cxl_host.h"
+
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size)
+{
+ /* TODO: Implement memory map addition for CXL Type 3 on ARM */
+}
diff --git a/hw/arm/meson.build b/hw/arm/meson.build
index bcc068de04..2a4f09fbef 100644
--- a/hw/arm/meson.build
+++ b/hw/arm/meson.build
@@ -116,6 +116,8 @@ arm_common_ss.add(when: ['CONFIG_AXIADO_EVK', 'TARGET_AARCH64'], if_true: files(
'ax3000-boards.c',
'ax3000-evk.c'))
+arm_common_ss.add(when: 'CONFIG_CXL', if_true: files('cxl.c'))
+
arm_common_ss.add(files('boot.c'))
hw_common_arch += {'arm': arm_common_ss}
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
index e279ce5fe3..1967c9a9c3 100644
--- a/hw/cxl/cxl-host.c
+++ b/hw/cxl/cxl-host.c
@@ -689,12 +689,37 @@ static int cxl_fmws_mmio_map(Object *obj, void *opaque)
return 0;
}
+static int cxl_c3_mmio_map(Object *obj, void *opaque)
+{
+ uint64_t base, size;
+ CXLType3Dev *c3d;
+
+ if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+ return 0;
+ }
+ c3d = CXL_TYPE3(obj);
+
+ if (!c3d->program_hdm_decoder) {
+ return 0;
+ }
+
+ ct3_type3_get_hpm_range(c3d, &base, &size);
+ cxl_c3_add_memory_map(base, size);
+
+ return 0;
+}
+
void cxl_fmws_update_mmio(void)
{
/* Ordering is not required for this */
object_child_foreach_recursive(object_get_root(), cxl_fmws_mmio_map, NULL);
}
+void cxl_c3_update_mmio(void)
+{
+ object_child_foreach_recursive(object_get_root(), cxl_c3_mmio_map, NULL);
+}
+
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr)
{
GSList *cfmws_list, *iter;
diff --git a/hw/i386/cxl.c b/hw/i386/cxl.c
new file mode 100644
index 0000000000..02dbe4d3f0
--- /dev/null
+++ b/hw/i386/cxl.c
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "exec/hwaddr.h"
+#include "hw/cxl/cxl_host.h"
+#include "hw/i386/e820_memory_layout.h"
+
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size)
+{
+ e820_add_entry(base, size, E820_SOFT_RESERVED);
+}
diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index e16930c431..b212b3f6fa 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -24,6 +24,7 @@ i386_ss.add(when: 'CONFIG_VTD', if_true: files('intel_iommu.c'))
i386_ss.add(when: 'CONFIG_VTD_ACCEL', if_true: files('intel_iommu_accel.c'))
i386_ss.add(when: 'CONFIG_SGX', if_true: files('sgx-epc.c','sgx.c'),
if_false: files('sgx-stub.c'))
+i386_ss.add(when: 'CONFIG_CXL', if_true: files('cxl.c'))
if have_hmp
stub_ss.add(files('sgx-hmp-stub.c'))
endif
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2d8c87b0e3..f843629543 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -534,6 +534,7 @@ void pc_machine_done(Notifier *notifier, void *data)
if (pcms->cxl_devices_state.is_enabled) {
cxl_fmws_link_targets(&error_fatal);
+ cxl_c3_update_mmio();
}
/* set the number of CPUs */
diff --git a/include/hw/cxl/cxl_host.h b/include/hw/cxl/cxl_host.h
index 21619bb748..6c142a5b89 100644
--- a/include/hw/cxl/cxl_host.h
+++ b/include/hw/cxl/cxl_host.h
@@ -18,6 +18,8 @@ void cxl_fmws_link_targets(Error **errp);
void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp);
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr);
void cxl_fmws_update_mmio(void);
+void cxl_c3_update_mmio(void);
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size);
GSList *cxl_fmws_get_all_sorted(void);
extern const MemoryRegionOps cfmws_ops;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 5/7] QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
@ 2026-10-26 10:03 ` Yukiya Takaki via
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, Paolo Bonzini, Peter Maydell, Richard Henderson,
linux-cxl, qemu-arm, Wolski, Tomasz, lizhijian, y-goto,
yukiya.takaki
On real systems, CXL Type 3 memory is exposed to the OS via the E820
table as E820_SOFT_RESERVED, so the kernel can treat it as a dedicated
reserved range rather than normal RAM.
This patch adds a small helper and a recursive walker so that, on x86,
the HPA ranges assigned to CXL Type 3 devices are added to the E820 table
as E820_SOFT_RESERVED when `program_hdm_decoder` is enabled. An ARM stub
is also added as a placeholder for future support.
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/arm/cxl.c | 10 ++++++++++
hw/arm/meson.build | 2 ++
hw/cxl/cxl-host.c | 25 +++++++++++++++++++++++++
hw/i386/cxl.c | 11 +++++++++++
hw/i386/meson.build | 1 +
hw/i386/pc.c | 1 +
include/hw/cxl/cxl_host.h | 2 ++
7 files changed, 52 insertions(+)
create mode 100644 hw/arm/cxl.c
create mode 100644 hw/i386/cxl.c
diff --git a/hw/arm/cxl.c b/hw/arm/cxl.c
new file mode 100644
index 0000000000..1cbcf2d8e5
--- /dev/null
+++ b/hw/arm/cxl.c
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "exec/hwaddr.h"
+#include "hw/cxl/cxl_host.h"
+
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size)
+{
+ /* TODO: Implement memory map addition for CXL Type 3 on ARM */
+}
diff --git a/hw/arm/meson.build b/hw/arm/meson.build
index bcc068de04..2a4f09fbef 100644
--- a/hw/arm/meson.build
+++ b/hw/arm/meson.build
@@ -116,6 +116,8 @@ arm_common_ss.add(when: ['CONFIG_AXIADO_EVK', 'TARGET_AARCH64'], if_true: files(
'ax3000-boards.c',
'ax3000-evk.c'))
+arm_common_ss.add(when: 'CONFIG_CXL', if_true: files('cxl.c'))
+
arm_common_ss.add(files('boot.c'))
hw_common_arch += {'arm': arm_common_ss}
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
index e279ce5fe3..1967c9a9c3 100644
--- a/hw/cxl/cxl-host.c
+++ b/hw/cxl/cxl-host.c
@@ -689,12 +689,37 @@ static int cxl_fmws_mmio_map(Object *obj, void *opaque)
return 0;
}
+static int cxl_c3_mmio_map(Object *obj, void *opaque)
+{
+ uint64_t base, size;
+ CXLType3Dev *c3d;
+
+ if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+ return 0;
+ }
+ c3d = CXL_TYPE3(obj);
+
+ if (!c3d->program_hdm_decoder) {
+ return 0;
+ }
+
+ ct3_type3_get_hpm_range(c3d, &base, &size);
+ cxl_c3_add_memory_map(base, size);
+
+ return 0;
+}
+
void cxl_fmws_update_mmio(void)
{
/* Ordering is not required for this */
object_child_foreach_recursive(object_get_root(), cxl_fmws_mmio_map, NULL);
}
+void cxl_c3_update_mmio(void)
+{
+ object_child_foreach_recursive(object_get_root(), cxl_c3_mmio_map, NULL);
+}
+
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr)
{
GSList *cfmws_list, *iter;
diff --git a/hw/i386/cxl.c b/hw/i386/cxl.c
new file mode 100644
index 0000000000..02dbe4d3f0
--- /dev/null
+++ b/hw/i386/cxl.c
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "exec/hwaddr.h"
+#include "hw/cxl/cxl_host.h"
+#include "hw/i386/e820_memory_layout.h"
+
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size)
+{
+ e820_add_entry(base, size, E820_SOFT_RESERVED);
+}
diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index e16930c431..b212b3f6fa 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -24,6 +24,7 @@ i386_ss.add(when: 'CONFIG_VTD', if_true: files('intel_iommu.c'))
i386_ss.add(when: 'CONFIG_VTD_ACCEL', if_true: files('intel_iommu_accel.c'))
i386_ss.add(when: 'CONFIG_SGX', if_true: files('sgx-epc.c','sgx.c'),
if_false: files('sgx-stub.c'))
+i386_ss.add(when: 'CONFIG_CXL', if_true: files('cxl.c'))
if have_hmp
stub_ss.add(files('sgx-hmp-stub.c'))
endif
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2d8c87b0e3..f843629543 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -534,6 +534,7 @@ void pc_machine_done(Notifier *notifier, void *data)
if (pcms->cxl_devices_state.is_enabled) {
cxl_fmws_link_targets(&error_fatal);
+ cxl_c3_update_mmio();
}
/* set the number of CPUs */
diff --git a/include/hw/cxl/cxl_host.h b/include/hw/cxl/cxl_host.h
index 21619bb748..6c142a5b89 100644
--- a/include/hw/cxl/cxl_host.h
+++ b/include/hw/cxl/cxl_host.h
@@ -18,6 +18,8 @@ void cxl_fmws_link_targets(Error **errp);
void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp);
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr);
void cxl_fmws_update_mmio(void);
+void cxl_c3_update_mmio(void);
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size);
GSList *cxl_fmws_get_all_sorted(void);
extern const MemoryRegionOps cfmws_ops;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 5/7] QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
@ 2026-10-26 10:03 ` Yukiya Takaki via
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, Paolo Bonzini, Peter Maydell, Richard Henderson,
linux-cxl, qemu-arm, Wolski, Tomasz, lizhijian, y-goto,
yukiya.takaki
On real systems, CXL Type 3 memory is exposed to the OS via the E820
table as E820_SOFT_RESERVED, so the kernel can treat it as a dedicated
reserved range rather than normal RAM.
This patch adds a small helper and a recursive walker so that, on x86,
the HPA ranges assigned to CXL Type 3 devices are added to the E820 table
as E820_SOFT_RESERVED when `program_hdm_decoder` is enabled. An ARM stub
is also added as a placeholder for future support.
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/arm/cxl.c | 10 ++++++++++
hw/arm/meson.build | 2 ++
hw/cxl/cxl-host.c | 25 +++++++++++++++++++++++++
hw/i386/cxl.c | 11 +++++++++++
hw/i386/meson.build | 1 +
hw/i386/pc.c | 1 +
include/hw/cxl/cxl_host.h | 2 ++
7 files changed, 52 insertions(+)
create mode 100644 hw/arm/cxl.c
create mode 100644 hw/i386/cxl.c
diff --git a/hw/arm/cxl.c b/hw/arm/cxl.c
new file mode 100644
index 0000000000..1cbcf2d8e5
--- /dev/null
+++ b/hw/arm/cxl.c
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "exec/hwaddr.h"
+#include "hw/cxl/cxl_host.h"
+
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size)
+{
+ /* TODO: Implement memory map addition for CXL Type 3 on ARM */
+}
diff --git a/hw/arm/meson.build b/hw/arm/meson.build
index bcc068de04..2a4f09fbef 100644
--- a/hw/arm/meson.build
+++ b/hw/arm/meson.build
@@ -116,6 +116,8 @@ arm_common_ss.add(when: ['CONFIG_AXIADO_EVK', 'TARGET_AARCH64'], if_true: files(
'ax3000-boards.c',
'ax3000-evk.c'))
+arm_common_ss.add(when: 'CONFIG_CXL', if_true: files('cxl.c'))
+
arm_common_ss.add(files('boot.c'))
hw_common_arch += {'arm': arm_common_ss}
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
index e279ce5fe3..1967c9a9c3 100644
--- a/hw/cxl/cxl-host.c
+++ b/hw/cxl/cxl-host.c
@@ -689,12 +689,37 @@ static int cxl_fmws_mmio_map(Object *obj, void *opaque)
return 0;
}
+static int cxl_c3_mmio_map(Object *obj, void *opaque)
+{
+ uint64_t base, size;
+ CXLType3Dev *c3d;
+
+ if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+ return 0;
+ }
+ c3d = CXL_TYPE3(obj);
+
+ if (!c3d->program_hdm_decoder) {
+ return 0;
+ }
+
+ ct3_type3_get_hpm_range(c3d, &base, &size);
+ cxl_c3_add_memory_map(base, size);
+
+ return 0;
+}
+
void cxl_fmws_update_mmio(void)
{
/* Ordering is not required for this */
object_child_foreach_recursive(object_get_root(), cxl_fmws_mmio_map, NULL);
}
+void cxl_c3_update_mmio(void)
+{
+ object_child_foreach_recursive(object_get_root(), cxl_c3_mmio_map, NULL);
+}
+
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr)
{
GSList *cfmws_list, *iter;
diff --git a/hw/i386/cxl.c b/hw/i386/cxl.c
new file mode 100644
index 0000000000..02dbe4d3f0
--- /dev/null
+++ b/hw/i386/cxl.c
@@ -0,0 +1,11 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "exec/hwaddr.h"
+#include "hw/cxl/cxl_host.h"
+#include "hw/i386/e820_memory_layout.h"
+
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size)
+{
+ e820_add_entry(base, size, E820_SOFT_RESERVED);
+}
diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index e16930c431..b212b3f6fa 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -24,6 +24,7 @@ i386_ss.add(when: 'CONFIG_VTD', if_true: files('intel_iommu.c'))
i386_ss.add(when: 'CONFIG_VTD_ACCEL', if_true: files('intel_iommu_accel.c'))
i386_ss.add(when: 'CONFIG_SGX', if_true: files('sgx-epc.c','sgx.c'),
if_false: files('sgx-stub.c'))
+i386_ss.add(when: 'CONFIG_CXL', if_true: files('cxl.c'))
if have_hmp
stub_ss.add(files('sgx-hmp-stub.c'))
endif
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2d8c87b0e3..f843629543 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -534,6 +534,7 @@ void pc_machine_done(Notifier *notifier, void *data)
if (pcms->cxl_devices_state.is_enabled) {
cxl_fmws_link_targets(&error_fatal);
+ cxl_c3_update_mmio();
}
/* set the number of CPUs */
diff --git a/include/hw/cxl/cxl_host.h b/include/hw/cxl/cxl_host.h
index 21619bb748..6c142a5b89 100644
--- a/include/hw/cxl/cxl_host.h
+++ b/include/hw/cxl/cxl_host.h
@@ -18,6 +18,8 @@ void cxl_fmws_link_targets(Error **errp);
void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp);
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr);
void cxl_fmws_update_mmio(void);
+void cxl_c3_update_mmio(void);
+void cxl_c3_add_memory_map(hwaddr base, hwaddr size);
GSList *cxl_fmws_get_all_sorted(void);
extern const MemoryRegionOps cfmws_ops;
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH RFC 6/7] QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
2026-10-26 10:03 ` Yukiya Takaki via
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, Paolo Bonzini, Richard Henderson, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
On the actual hardware, CXL memory is exposed to the OS as a "Memory Only Node."
This allows the OS to recognize that accessing CXL memory takes time.
As a preliminary step, a function was created to automatically add a NUMA node for each instance of CXL memory.
This function generates one NUMA node per instance of CXL memory.
This function is executed when program_hdm_decoder is set.
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/cxl/cxl-host.c | 38 +++++++++++++++++++++++++++++++++++++
hw/i386/pc.c | 1 +
include/hw/cxl/cxl_device.h | 1 +
include/hw/cxl/cxl_host.h | 1 +
4 files changed, 41 insertions(+)
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
index 1967c9a9c3..ba321841dc 100644
--- a/hw/cxl/cxl-host.c
+++ b/hw/cxl/cxl-host.c
@@ -720,6 +720,44 @@ void cxl_c3_update_mmio(void)
object_child_foreach_recursive(object_get_root(), cxl_c3_mmio_map, NULL);
}
+static int setup_cxl_numa_node_inner(Object *obj, void *opaque)
+{
+ MachineState *machine = opaque;
+ DeviceState *dev;
+ CXLType3Dev *c3d;
+ int node_id;
+
+ if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+ return 0;
+ }
+
+ dev = DEVICE(obj);
+ c3d = CXL_TYPE3(dev);
+ node_id = machine->numa_state->num_nodes;
+
+ if (!c3d->program_hdm_decoder) {
+ return 0;
+ }
+
+ NumaOptions option = {
+ .type = NUMA_OPTIONS_TYPE_NODE,
+ .u.node = {
+ .has_nodeid = true,
+ .nodeid = node_id,
+ },
+ };
+
+ set_numa_options(machine, &option, &error_fatal);
+ c3d->node = node_id;
+ return 0;
+}
+
+void setup_cxl_numa_node(MachineState *machine)
+{
+ object_child_foreach_recursive(object_get_root(),
+ setup_cxl_numa_node_inner, machine);
+}
+
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr)
{
GSList *cfmws_list, *iter;
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index f843629543..b1d1f03468 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -534,6 +534,7 @@ void pc_machine_done(Notifier *notifier, void *data)
if (pcms->cxl_devices_state.is_enabled) {
cxl_fmws_link_targets(&error_fatal);
+ setup_cxl_numa_node(MACHINE(pcms));
cxl_c3_update_mmio();
}
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 07654f4b62..9001aacf0e 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -780,6 +780,7 @@ struct CXLType3Dev {
bool program_hdm_decoder;
hwaddr hpa_offset;
int idx;
+ int node;
struct dynamic_capacity {
HostMemoryBackend *host_dc;
diff --git a/include/hw/cxl/cxl_host.h b/include/hw/cxl/cxl_host.h
index 6c142a5b89..70285328c4 100644
--- a/include/hw/cxl/cxl_host.h
+++ b/include/hw/cxl/cxl_host.h
@@ -19,6 +19,7 @@ void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp);
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr);
void cxl_fmws_update_mmio(void);
void cxl_c3_update_mmio(void);
+void setup_cxl_numa_node(MachineState *machine);
void cxl_c3_add_memory_map(hwaddr base, hwaddr size);
GSList *cxl_fmws_get_all_sorted(void);
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* [PATCH RFC 6/7] QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Anisa Su, Jonathan Cameron,
Junjie Cao, Paolo Bonzini, Richard Henderson, linux-cxl,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
On the actual hardware, CXL memory is exposed to the OS as a "Memory Only Node."
This allows the OS to recognize that accessing CXL memory takes time.
As a preliminary step, a function was created to automatically add a NUMA node for each instance of CXL memory.
This function generates one NUMA node per instance of CXL memory.
This function is executed when program_hdm_decoder is set.
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/cxl/cxl-host.c | 38 +++++++++++++++++++++++++++++++++++++
hw/i386/pc.c | 1 +
include/hw/cxl/cxl_device.h | 1 +
include/hw/cxl/cxl_host.h | 1 +
4 files changed, 41 insertions(+)
diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
index 1967c9a9c3..ba321841dc 100644
--- a/hw/cxl/cxl-host.c
+++ b/hw/cxl/cxl-host.c
@@ -720,6 +720,44 @@ void cxl_c3_update_mmio(void)
object_child_foreach_recursive(object_get_root(), cxl_c3_mmio_map, NULL);
}
+static int setup_cxl_numa_node_inner(Object *obj, void *opaque)
+{
+ MachineState *machine = opaque;
+ DeviceState *dev;
+ CXLType3Dev *c3d;
+ int node_id;
+
+ if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+ return 0;
+ }
+
+ dev = DEVICE(obj);
+ c3d = CXL_TYPE3(dev);
+ node_id = machine->numa_state->num_nodes;
+
+ if (!c3d->program_hdm_decoder) {
+ return 0;
+ }
+
+ NumaOptions option = {
+ .type = NUMA_OPTIONS_TYPE_NODE,
+ .u.node = {
+ .has_nodeid = true,
+ .nodeid = node_id,
+ },
+ };
+
+ set_numa_options(machine, &option, &error_fatal);
+ c3d->node = node_id;
+ return 0;
+}
+
+void setup_cxl_numa_node(MachineState *machine)
+{
+ object_child_foreach_recursive(object_get_root(),
+ setup_cxl_numa_node_inner, machine);
+}
+
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr)
{
GSList *cfmws_list, *iter;
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index f843629543..b1d1f03468 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -534,6 +534,7 @@ void pc_machine_done(Notifier *notifier, void *data)
if (pcms->cxl_devices_state.is_enabled) {
cxl_fmws_link_targets(&error_fatal);
+ setup_cxl_numa_node(MACHINE(pcms));
cxl_c3_update_mmio();
}
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 07654f4b62..9001aacf0e 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -780,6 +780,7 @@ struct CXLType3Dev {
bool program_hdm_decoder;
hwaddr hpa_offset;
int idx;
+ int node;
struct dynamic_capacity {
HostMemoryBackend *host_dc;
diff --git a/include/hw/cxl/cxl_host.h b/include/hw/cxl/cxl_host.h
index 6c142a5b89..70285328c4 100644
--- a/include/hw/cxl/cxl_host.h
+++ b/include/hw/cxl/cxl_host.h
@@ -19,6 +19,7 @@ void cxl_hook_up_pxb_registers(PCIBus *bus, CXLState *state, Error **errp);
hwaddr cxl_fmws_set_memmap(hwaddr base, hwaddr max_addr);
void cxl_fmws_update_mmio(void);
void cxl_c3_update_mmio(void);
+void setup_cxl_numa_node(MachineState *machine);
void cxl_c3_add_memory_map(hwaddr base, hwaddr size);
GSList *cxl_fmws_get_all_sorted(void);
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH RFC 7/7] add CXL Type 3 memory ranges to SRAT
2026-10-26 10:03 ` Yukiya Takaki via
` (7 preceding siblings ...)
(?)
@ 2026-10-26 10:03 ` Yukiya Takaki via qemu development
-1 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki via qemu development @ 2026-10-26 10:03 UTC (permalink / raw)
To: qemu-devel
Cc: Yukiya Takaki, Michael S. Tsirkin, Ani Sinha, Igor Mammedov,
Paolo Bonzini, Richard Henderson, Wolski, Tomasz, lizhijian,
y-goto, yukiya.takaki
On Linux, the kernel first checks E820_SOFT_RESERVED ranges, and when
multiple CXL Type 3 devices share a contiguous HPA range those ranges
tend to be merged into a single region. In that case, the CXL driver
cannot reliably initialize per‑device regions based on E820 alone.
To let the OS distinguish CXL Type 3 memory from normal RAM and associate
it with the “memory only” NUMA nodes created earlier, SRAT needs entries
for the host physical address ranges assigned to CXL Type 3 devices.
This patch adds a helper that walks CXL Type 3 devices, obtains their HPA
ranges via `ct3_type3_get_hpm_range()`, and emits SRAT memory affinity
entries using the NUMA node IDs stored on each device. This runs only when
CXL is enabled and `program_hdm_decoder` is set.
Signed-off-by: Yukiya Takaki <fj2110jc@fujitsu.com>
---
hw/acpi/cxl_type3.c | 30 ++++++++++++++++++++++++++++++
hw/acpi/meson.build | 1 +
hw/i386/acpi-build.c | 7 +++++++
include/hw/mem/cxl_type3.h | 8 ++++++++
4 files changed, 46 insertions(+)
create mode 100644 hw/acpi/cxl_type3.c
create mode 100644 include/hw/mem/cxl_type3.h
diff --git a/hw/acpi/cxl_type3.c b/hw/acpi/cxl_type3.c
new file mode 100644
index 0000000000..4cad270257
--- /dev/null
+++ b/hw/acpi/cxl_type3.c
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+
+#include "hw/acpi/aml-build.h"
+#include "hw/cxl/cxl_device.h"
+#include "hw/mem/cxl_type3.h"
+
+int cxl_type3_build_srat(Object *obj, void *opaque)
+{
+ GArray *table_data = opaque;
+ DeviceState *dev;
+ CXLType3Dev *c3d;
+ uint64_t addr, size;
+ if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+ return 0;
+ }
+ dev = DEVICE(obj);
+ c3d = CXL_TYPE3(dev);
+
+ if (!c3d->program_hdm_decoder) {
+ return 0;
+ }
+
+ ct3_type3_get_hpm_range(c3d, &addr, &size);
+
+ build_srat_memory(table_data, addr, size, c3d->node,
+ MEM_AFFINITY_ENABLED | MEM_AFFINITY_HOTPLUGGABLE);
+ return 0;
+}
diff --git a/hw/acpi/meson.build b/hw/acpi/meson.build
index 09ad488a04..b1def538bd 100644
--- a/hw/acpi/meson.build
+++ b/hw/acpi/meson.build
@@ -13,6 +13,7 @@ acpi_ss.add(when: 'CONFIG_ACPI_NVDIMM', if_true: files('nvdimm.c'))
stub_ss.add(files('acpi-nvdimm-stub.c'))
acpi_ss.add(when: 'CONFIG_ACPI_PCI', if_true: files('pci.c'))
acpi_ss.add(when: 'CONFIG_ACPI_CXL', if_true: files('cxl.c'))
+acpi_ss.add(when: 'CONFIG_CXL_MEM_DEVICE', if_true: files('cxl_type3.c'))
stub_ss.add(files('cxl-stub.c'))
acpi_ss.add(when: 'CONFIG_ACPI_VMGENID', if_true: files('vmgenid.c'))
acpi_ss.add(when: 'CONFIG_ACPI_VMCLOCK', if_true: files('vmclock.c'))
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 6d7308b4e4..2f607f3ec9 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -51,6 +51,7 @@
#include "hw/rtc/mc146818rtc_regs.h"
#include "migration/vmstate.h"
#include "hw/mem/memory-device.h"
+#include "hw/mem/cxl_type3.h"
#include "hw/mem/nvdimm.h"
#include "hw/mem/sp-mem.h"
#include "system/numa.h"
@@ -1460,6 +1461,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
uint64_t mem_len, mem_base, next_base;
MachineClass *mc = MACHINE_GET_CLASS(machine);
X86MachineState *x86ms = X86_MACHINE(machine);
+ PCMachineState *pcms = PC_MACHINE(machine);
const CPUArchIdList *apic_ids = mc->possible_cpu_arch_ids(machine);
int nb_numa_nodes = machine->numa_state->num_nodes;
NodeInfo *numa_info = machine->numa_state->nodes;
@@ -1584,6 +1586,11 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
build_srat_device_memory(table_data, machine);
}
+ if (pcms->cxl_devices_state.is_enabled) {
+ object_child_foreach_recursive(object_get_root(),
+ cxl_type3_build_srat, table_data);
+ }
+
acpi_table_end(linker, &table);
}
diff --git a/include/hw/mem/cxl_type3.h b/include/hw/mem/cxl_type3.h
new file mode 100644
index 0000000000..15a204d66a
--- /dev/null
+++ b/include/hw/mem/cxl_type3.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_CXL_TYPE3_H
+#define QEMU_CXL_TYPE3_H
+
+int cxl_type3_build_srat(Object *obj, void *opaque);
+
+#endif
--
2.52.0
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
2026-10-26 10:03 ` Yukiya Takaki via
` (8 preceding siblings ...)
(?)
@ 2026-11-05 2:38 ` Anisa Su
2026-11-05 22:41 ` Anisa Su
-1 siblings, 1 reply; 24+ messages in thread
From: Anisa Su @ 2026-11-05 2:38 UTC (permalink / raw)
To: Yukiya Takaki
Cc: qemu-devel, Michael S. Tsirkin, Ani Sinha, Igor Mammedov,
Jonathan Cameron, Junjie Cao, Paolo Bonzini, Peter Maydell,
Richard Henderson, linux-cxl, qemu-arm, Wolski, Tomasz, lizhijian,
y-goto, yukiya.takaki
On Mon, Oct 26, 2026 at 07:03:21PM +0900, Yukiya Takaki wrote:
> Hello everyone,
>
> I’m a student intern, and this is my first time sending patches to an open source mailing list.
>
> This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
> and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
>
> Background and motivation
> Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
> to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
> By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
> allowing the Linux CXL driver to discover and create regions during boot.
> This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
>
> Some parts of this implementation are based on Li, Zhijian’s work in:
> https://github.com/zhijianli88/qemu/tree/program-decoder-v2
>
> I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
>
> Approach
> This patch first adds a program_hdm_decoder option to cxl-type3.
> When program-hdm-decoder is enabled on cxl-type3, QEMU will:
> 1) program endpoint/host-bridge HDM decoder state,
> 2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
> 3) create memory-only NUMA nodes for those ranges,
> 4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
>
> On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
> However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
> into SeaBIOS would introduce additional complexity and risk inconsistencies.
>
Hi again,
Thank you for your contribution. I think this series can be valuable for
helping test autodiscovery in the kernel driver. I spent more time looking over this
patchset and in my view, the main blockers for making this ready for upstream are:
1.The HPA state is kept in the wrong place.
Patch 2 adds hpa_offset, idx and node to CXLType3Dev but and endpoint should
not track host state.
I also think it makes more sense to make program-dhm-decoder a machine
option (-M program-hdm-decoder...) instead of having it on cxl-type3
endpoint
2. Host bridge register init moved from reset to realize.
As you mentioned below, "The current implementation prevents the host
bridge HDM decoder configuration from being overwritten later in the same reset
by moving the common initialization of the host bridge component registers from
the reset path to the realization path."
To more accurately emulate what firmware does, maybe you can try
something like the following?
1. Add program-hdm option to machine (hw/cxl/cxl-host.c, include/hw/cxl/cxl.h)
typedef struct CXLState {
bool is_enabled;
+ bool program_hdm_decoders;
...
} CXLState;
+static void machine_set_hdm_auto(Object *obj, Visitor *v, const char *name,
+ void *opaque, Error **errp)
+{
+ CXLState *cxl_state = opaque;
+ bool value;
+
+ if (!visit_type_bool(v, name, &value, errp)) {
+ return;
+ }
+ cxl_state->program_hdm_decoders = value;
+}
void cxl_machine_init(Object *obj, CXLState *state)
{
...
+ object_property_add(obj, "cxl-hdm-auto", "bool", machine_get_hdm_auto,
+ machine_set_hdm_auto, NULL, state);
+ object_property_set_description(obj, "cxl-hdm-auto",
+ "Set on/off to have HDM decoders "
+ "programmed at reset as firmware would");
2. Add cxl_fmws_program_decoders() to pxb_cxl reset exit.
Every device has 3 reset phase: enter, hold, and exit. Currently,
pxb-cxl has:
- enter = none
- hold = pxb_cxL-dev_reset(), which zeroes registers
- exit = none
Register cxl_fmws_program_decoders() to pxb-cxl reset exit.
cxl-host.c:
+void cxl_fmws_for_each_auto_range(PXBCXLDev *pxb)
{
+ /* use existing helper cxl_fmws_get_all_sorted to iterate */
+ GSList *fws = cxl_fmws_get_all_sorted(), *it;
+
+for (it = fws; it; it = it->next) {
+ CXLFixedWindow *fw = CXL_FMW(it->data);
/* program decoders here */
+ }
}
...
+/* Call from the pxb-cxl reset exit phase, see pxb_cxl_dev_reset_exit() */
+void cxl_fmws_program_decoders(PXBCXLDev *pxb)
+{
+ cxl_fmws_for_each_auto_range(pxb);
+}
--- a/hw/pci-bridge/pci_expander_bridge.c
+++ b/hw/pci-bridge/pci_expander_bridge.c
#include "hw/cxl/cxl.h"
+#include "hw/cxl/cxl_host.h"
+static void pxb_cxl_dev_reset_exit(Object *obj, ResetType type)
+{
+ PXBCXLDev *pxb = PXB_CXL_DEV(obj);
+ PXBCXLDevClass *pc = PXB_CXL_DEV_GET_CLASS(obj);
+
+ if (pc->parent_phases.exit) {
+ pc->parent_phases.exit(obj, type);
+ }
+ if (pxb->cxl_state && pxb->cxl_state->program_hdm_decoders) {
+ cxl_fmws_program_decoders(pxb);
+ }
+}
+
static void pxb_cxl_dev_class_init(ObjectClass *klass, const void *data)
{
+ ResettableClass *rc = RESETTABLE_CLASS(klass);
...
device_class_set_legacy_reset(dc, pxb_cxl_dev_reset);
+ rc->phases.exit = pxb_cxl_dev_reset_exit;
}
This way, we shouldn't need to store the extra state (hpa_offset,etc.)
in the cxl-type3 endpoints.
I'm also not that familiar with the machine initialization and reset
stages, so maybe this approach doesn't work, but this is my best guess.
Thanks,
Anisa
> Testing
> We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
> and E820_SOFT_RESERVED entries appeared in the E820 table:
>
> - BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
> - CXL memory: 4 GB
> - CFMWS: 4 GB
> - Host bridge: 1
>
> The following is the startup script.
>
> sudo /path/to/qemu/build/qemu-system-x86_64 \
> -name guest=cxl \
> -nographic \
> -machine q35,accel=tcg,nvdimm=on,cxl=on \
> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
> -m size=4G,slots=10,maxmem=20G \
> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
> -bios /path/to/seabios/out/bios.bin \
> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
> -object memory-backend-ram,size=2G,id=m0 \
> -object memory-backend-ram,size=2G,id=m1 \
> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
>
> Furthermore, we conducted the experiment in which we added two CXL Type3.
>
> - CXL memory: 4 GB × 2
> - CFMWS: 4 GB × 2
>
> The following is the startup script.
>
> sudo /path/to/qemu/build/qemu-system-x86_64 \
> -name guest=cxl \
> -nographic \
> -machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
> -m size=4G,slots=10,maxmem=20G \
> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
> -bios /path/to/seabios/out/bios.bin \
> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
> -object memory-backend-ram,size=2G,id=m0 \
> -object memory-backend-ram,size=2G,id=m1 \
> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
> -device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
> -device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
> -object memory-backend-ram,size=4G,share=on,id=vmem1 \
> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
> -device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
>
>
> How to use
> Specify the following parameters when starting QEMU:
>
> - bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
> - Add hmat=on to the -machine option [1]
> - Add program-hdm-decoder=true to each -device cxl-type3 option
> [1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
> these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
> SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
> so HMAT must be enabled.
>
> QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
>
> TODO
> With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
> As a result, automatic creation of CXL regions in the guest OS may fail:
>
> - A CXL persistent memory device is included
> - Interleaved configurations
> - Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
> - Configurations where CXL memory is attached via a switch
> - Multi-target CFMWS environments
> - ARM environments
> - HMAT does not yet include information about CXL memory devices
>
> The following behaviors are also provisional implementations and may need further refinement:
> - A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
> or to allow the NUMA node IDs to be specified by the user.
> - Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
> based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
> through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
> This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
> adding an explicit option to associate each Type 3 device with a CFMWS entry,
> or generating CFMWS entries automatically from the configured CXL topology and devices.
> - HDM decoder reset sequencing and state reconstruction.
> The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
> by moving the common initialization of the host bridge component registers from the reset path to the realization path.
> However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
> automatically programmed by QEMU should be reconstructed as part of reset processing.
>
> Thanks,
>
> ------
>
> Yukiya Takaki (7):
> QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
> QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
> QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
> QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
> QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
> QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
> add CXL Type 3 memory ranges to SRAT
>
> hw/acpi/cxl_type3.c | 30 +++++
> hw/acpi/meson.build | 1 +
> hw/arm/cxl.c | 10 ++
> hw/arm/meson.build | 2 +
> hw/cxl/cxl-host.c | 63 ++++++++++
> hw/i386/acpi-build.c | 7 ++
> hw/i386/cxl.c | 11 ++
> hw/i386/meson.build | 1 +
> hw/i386/pc.c | 2 +
> hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
> hw/pci-bridge/pci_expander_bridge.c | 13 +-
> include/hw/cxl/cxl.h | 2 +
> include/hw/cxl/cxl_device.h | 6 +
> include/hw/cxl/cxl_host.h | 3 +
> include/hw/mem/cxl_type3.h | 8 ++
> 15 files changed, 334 insertions(+), 2 deletions(-)
> create mode 100644 hw/acpi/cxl_type3.c
> create mode 100644 hw/arm/cxl.c
> create mode 100644 hw/i386/cxl.c
> create mode 100644 include/hw/mem/cxl_type3.h
>
> --
> 2.52.0
>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
2026-11-05 2:38 ` [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot Anisa Su
@ 2026-11-05 22:41 ` Anisa Su
2026-11-07 18:38 ` Yukiya Takaki
0 siblings, 1 reply; 24+ messages in thread
From: Anisa Su @ 2026-11-05 22:41 UTC (permalink / raw)
To: Anisa Su
Cc: Yukiya Takaki, qemu-devel, Michael S. Tsirkin, Ani Sinha,
Igor Mammedov, Jonathan Cameron, Junjie Cao, Paolo Bonzini,
Peter Maydell, Richard Henderson, linux-cxl, qemu-arm,
Wolski, Tomasz, lizhijian, y-goto, yukiya.takaki
On Thu, Nov 05, 2026 at 11:38:48AM +0900, Anisa Su wrote:
> On Mon, Oct 26, 2026 at 07:03:21PM +0900, Yukiya Takaki wrote:
> > Hello everyone,
> >
> > I’m a student intern, and this is my first time sending patches to an open source mailing list.
> >
> > This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
> > and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
> >
> > Background and motivation
> > Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
> > to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
> > By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
> > allowing the Linux CXL driver to discover and create regions during boot.
> > This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
> >
> > Some parts of this implementation are based on Li, Zhijian’s work in:
> > https://github.com/zhijianli88/qemu/tree/program-decoder-v2
> >
> > I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
> >
> > Approach
> > This patch first adds a program_hdm_decoder option to cxl-type3.
> > When program-hdm-decoder is enabled on cxl-type3, QEMU will:
> > 1) program endpoint/host-bridge HDM decoder state,
> > 2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
> > 3) create memory-only NUMA nodes for those ranges,
> > 4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
> >
> > On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
> > However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
> > into SeaBIOS would introduce additional complexity and risk inconsistencies.
> >
Ignore my previous suggestion and listen to Jonathan :)
Thanks,
Anisa
> Hi again,
>
> Thank you for your contribution. I think this series can be valuable for
> helping test autodiscovery in the kernel driver. I spent more time looking over this
> patchset and in my view, the main blockers for making this ready for upstream are:
>
> 1.The HPA state is kept in the wrong place.
> Patch 2 adds hpa_offset, idx and node to CXLType3Dev but and endpoint should
> not track host state.
>
> I also think it makes more sense to make program-dhm-decoder a machine
> option (-M program-hdm-decoder...) instead of having it on cxl-type3
> endpoint
>
> 2. Host bridge register init moved from reset to realize.
> As you mentioned below, "The current implementation prevents the host
> bridge HDM decoder configuration from being overwritten later in the same reset
> by moving the common initialization of the host bridge component registers from
> the reset path to the realization path."
>
> To more accurately emulate what firmware does, maybe you can try
> something like the following?
>
> 1. Add program-hdm option to machine (hw/cxl/cxl-host.c, include/hw/cxl/cxl.h)
>
> typedef struct CXLState {
> bool is_enabled;
> + bool program_hdm_decoders;
> ...
> } CXLState;
>
> +static void machine_set_hdm_auto(Object *obj, Visitor *v, const char *name,
> + void *opaque, Error **errp)
> +{
> + CXLState *cxl_state = opaque;
> + bool value;
> +
> + if (!visit_type_bool(v, name, &value, errp)) {
> + return;
> + }
> + cxl_state->program_hdm_decoders = value;
> +}
>
> void cxl_machine_init(Object *obj, CXLState *state)
> {
> ...
> + object_property_add(obj, "cxl-hdm-auto", "bool", machine_get_hdm_auto,
> + machine_set_hdm_auto, NULL, state);
> + object_property_set_description(obj, "cxl-hdm-auto",
> + "Set on/off to have HDM decoders "
> + "programmed at reset as firmware would");
>
> 2. Add cxl_fmws_program_decoders() to pxb_cxl reset exit.
> Every device has 3 reset phase: enter, hold, and exit. Currently,
> pxb-cxl has:
> - enter = none
> - hold = pxb_cxL-dev_reset(), which zeroes registers
> - exit = none
>
> Register cxl_fmws_program_decoders() to pxb-cxl reset exit.
>
> cxl-host.c:
> +void cxl_fmws_for_each_auto_range(PXBCXLDev *pxb)
> {
>
> + /* use existing helper cxl_fmws_get_all_sorted to iterate */
> + GSList *fws = cxl_fmws_get_all_sorted(), *it;
> +
> +for (it = fws; it; it = it->next) {
> + CXLFixedWindow *fw = CXL_FMW(it->data);
> /* program decoders here */
>
> + }
> }
> ...
>
> +/* Call from the pxb-cxl reset exit phase, see pxb_cxl_dev_reset_exit() */
> +void cxl_fmws_program_decoders(PXBCXLDev *pxb)
> +{
> + cxl_fmws_for_each_auto_range(pxb);
> +}
>
> --- a/hw/pci-bridge/pci_expander_bridge.c
> +++ b/hw/pci-bridge/pci_expander_bridge.c
> #include "hw/cxl/cxl.h"
> +#include "hw/cxl/cxl_host.h"
>
> +static void pxb_cxl_dev_reset_exit(Object *obj, ResetType type)
> +{
> + PXBCXLDev *pxb = PXB_CXL_DEV(obj);
> + PXBCXLDevClass *pc = PXB_CXL_DEV_GET_CLASS(obj);
> +
> + if (pc->parent_phases.exit) {
> + pc->parent_phases.exit(obj, type);
> + }
> + if (pxb->cxl_state && pxb->cxl_state->program_hdm_decoders) {
> + cxl_fmws_program_decoders(pxb);
> + }
> +}
> +
> static void pxb_cxl_dev_class_init(ObjectClass *klass, const void *data)
> {
> + ResettableClass *rc = RESETTABLE_CLASS(klass);
> ...
> device_class_set_legacy_reset(dc, pxb_cxl_dev_reset);
> + rc->phases.exit = pxb_cxl_dev_reset_exit;
> }
>
> This way, we shouldn't need to store the extra state (hpa_offset,etc.)
> in the cxl-type3 endpoints.
>
> I'm also not that familiar with the machine initialization and reset
> stages, so maybe this approach doesn't work, but this is my best guess.
>
> Thanks,
> Anisa
>
> > Testing
> > We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
> > and E820_SOFT_RESERVED entries appeared in the E820 table:
> >
> > - BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
> > - CXL memory: 4 GB
> > - CFMWS: 4 GB
> > - Host bridge: 1
> >
> > The following is the startup script.
> >
> > sudo /path/to/qemu/build/qemu-system-x86_64 \
> > -name guest=cxl \
> > -nographic \
> > -machine q35,accel=tcg,nvdimm=on,cxl=on \
> > -cpu max -smp 4,sockets=4,cores=1,threads=1 \
> > -m size=4G,slots=10,maxmem=20G \
> > -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
> > -bios /path/to/seabios/out/bios.bin \
> > -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
> > -object memory-backend-ram,size=2G,id=m0 \
> > -object memory-backend-ram,size=2G,id=m1 \
> > -numa node,nodeid=0,cpus=0-1,memdev=m0 \
> > -numa node,nodeid=1,cpus=2-3,memdev=m1 \
> > -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
> > -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> > -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
> > -object memory-backend-ram,size=4G,share=on,id=vmem0 \
> > -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
> > -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
> >
> > Furthermore, we conducted the experiment in which we added two CXL Type3.
> >
> > - CXL memory: 4 GB × 2
> > - CFMWS: 4 GB × 2
> >
> > The following is the startup script.
> >
> > sudo /path/to/qemu/build/qemu-system-x86_64 \
> > -name guest=cxl \
> > -nographic \
> > -machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
> > -cpu max -smp 4,sockets=4,cores=1,threads=1 \
> > -m size=4G,slots=10,maxmem=20G \
> > -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
> > -bios /path/to/seabios/out/bios.bin \
> > -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
> > -object memory-backend-ram,size=2G,id=m0 \
> > -object memory-backend-ram,size=2G,id=m1 \
> > -numa node,nodeid=0,cpus=0-1,memdev=m0 \
> > -numa node,nodeid=1,cpus=2-3,memdev=m1 \
> > -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
> > -device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
> > -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> > -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
> > -device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
> > -object memory-backend-ram,size=4G,share=on,id=vmem0 \
> > -object memory-backend-ram,size=4G,share=on,id=vmem1 \
> > -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
> > -device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
> > -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
> >
> >
> > How to use
> > Specify the following parameters when starting QEMU:
> >
> > - bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
> > - Add hmat=on to the -machine option [1]
> > - Add program-hdm-decoder=true to each -device cxl-type3 option
> > [1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
> > these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
> > SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
> > so HMAT must be enabled.
> >
> > QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
> >
> > TODO
> > With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
> > As a result, automatic creation of CXL regions in the guest OS may fail:
> >
> > - A CXL persistent memory device is included
> > - Interleaved configurations
> > - Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
> > - Configurations where CXL memory is attached via a switch
> > - Multi-target CFMWS environments
> > - ARM environments
> > - HMAT does not yet include information about CXL memory devices
> >
> > The following behaviors are also provisional implementations and may need further refinement:
> > - A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
> > or to allow the NUMA node IDs to be specified by the user.
> > - Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
> > based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
> > through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
> > This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
> > adding an explicit option to associate each Type 3 device with a CFMWS entry,
> > or generating CFMWS entries automatically from the configured CXL topology and devices.
> > - HDM decoder reset sequencing and state reconstruction.
> > The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
> > by moving the common initialization of the host bridge component registers from the reset path to the realization path.
> > However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
> > automatically programmed by QEMU should be reconstructed as part of reset processing.
> >
> > Thanks,
> >
> > ------
> >
> > Yukiya Takaki (7):
> > QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
> > QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
> > QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
> > QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
> > QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
> > QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
> > add CXL Type 3 memory ranges to SRAT
> >
> > hw/acpi/cxl_type3.c | 30 +++++
> > hw/acpi/meson.build | 1 +
> > hw/arm/cxl.c | 10 ++
> > hw/arm/meson.build | 2 +
> > hw/cxl/cxl-host.c | 63 ++++++++++
> > hw/i386/acpi-build.c | 7 ++
> > hw/i386/cxl.c | 11 ++
> > hw/i386/meson.build | 1 +
> > hw/i386/pc.c | 2 +
> > hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
> > hw/pci-bridge/pci_expander_bridge.c | 13 +-
> > include/hw/cxl/cxl.h | 2 +
> > include/hw/cxl/cxl_device.h | 6 +
> > include/hw/cxl/cxl_host.h | 3 +
> > include/hw/mem/cxl_type3.h | 8 ++
> > 15 files changed, 334 insertions(+), 2 deletions(-)
> > create mode 100644 hw/acpi/cxl_type3.c
> > create mode 100644 hw/arm/cxl.c
> > create mode 100644 hw/i386/cxl.c
> > create mode 100644 include/hw/mem/cxl_type3.h
> >
> > --
> > 2.52.0
> >
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
2026-11-05 22:41 ` Anisa Su
@ 2026-11-07 18:38 ` Yukiya Takaki
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-11-07 18:38 UTC (permalink / raw)
To: Anisa Su
Cc: Yukiya Takaki, qemu-devel, Michael S. Tsirkin, Ani Sinha,
Igor Mammedov, Jonathan Cameron, Junjie Cao, Paolo Bonzini,
Peter Maydell, Richard Henderson, linux-cxl, qemu-arm,
Wolski, Tomasz, lizhijian, y-goto
Hi Anisa,
> On Thu, Nov 05, 2026 at 11:38:48AM +0900, Anisa Su wrote:
>> On Mon, Oct 26, 2026 at 07:03:21PM +0900, Yukiya Takaki wrote:
>>> Hello everyone,
>>>
>>> I’m a student intern, and this is my first time sending patches to an open source mailing list.
>>>
>>> This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
>>> and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
>>>
>>> Background and motivation
>>> Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
>>> to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
>>> By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
>>> allowing the Linux CXL driver to discover and create regions during boot.
>>> This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
>>>
>>> Some parts of this implementation are based on Li, Zhijian’s work in:
>>> https://github.com/zhijianli88/qemu/tree/program-decoder-v2
>>>
>>> I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
>>>
>>> Approach
>>> This patch first adds a program_hdm_decoder option to cxl-type3.
>>> When program-hdm-decoder is enabled on cxl-type3, QEMU will:
>>> 1) program endpoint/host-bridge HDM decoder state,
>>> 2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
>>> 3) create memory-only NUMA nodes for those ranges,
>>> 4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
>>>
>>> On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
>>> However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
>>> into SeaBIOS would introduce additional complexity and risk inconsistencies.
>>>
> Ignore my previous suggestion and listen to Jonathan :)
Thank you very much for taking the time to review my patches!
I will follow Jonathan's suggestion and explore moving the HDM decoder
programming and E820 updates to SeaBIOS, while updating the SRAT in QEMU
via acpi_build_update() for v2.
Best regards,
Yukiya
>
> Thanks,
> Anisa
>
>> Hi again,
>>
>> Thank you for your contribution. I think this series can be valuable for
>> helping test autodiscovery in the kernel driver. I spent more time looking over this
>> patchset and in my view, the main blockers for making this ready for upstream are:
>>
>> 1.The HPA state is kept in the wrong place.
>> Patch 2 adds hpa_offset, idx and node to CXLType3Dev but and endpoint should
>> not track host state.
>>
>> I also think it makes more sense to make program-dhm-decoder a machine
>> option (-M program-hdm-decoder...) instead of having it on cxl-type3
>> endpoint
>>
>> 2. Host bridge register init moved from reset to realize.
>> As you mentioned below, "The current implementation prevents the host
>> bridge HDM decoder configuration from being overwritten later in the same reset
>> by moving the common initialization of the host bridge component registers from
>> the reset path to the realization path."
>>
>> To more accurately emulate what firmware does, maybe you can try
>> something like the following?
>>
>> 1. Add program-hdm option to machine (hw/cxl/cxl-host.c, include/hw/cxl/cxl.h)
>>
>> typedef struct CXLState {
>> bool is_enabled;
>> + bool program_hdm_decoders;
>> ...
>> } CXLState;
>>
>> +static void machine_set_hdm_auto(Object *obj, Visitor *v, const char *name,
>> + void *opaque, Error **errp)
>> +{
>> + CXLState *cxl_state = opaque;
>> + bool value;
>> +
>> + if (!visit_type_bool(v, name, &value, errp)) {
>> + return;
>> + }
>> + cxl_state->program_hdm_decoders = value;
>> +}
>>
>> void cxl_machine_init(Object *obj, CXLState *state)
>> {
>> ...
>> + object_property_add(obj, "cxl-hdm-auto", "bool", machine_get_hdm_auto,
>> + machine_set_hdm_auto, NULL, state);
>> + object_property_set_description(obj, "cxl-hdm-auto",
>> + "Set on/off to have HDM decoders "
>> + "programmed at reset as firmware would");
>>
>> 2. Add cxl_fmws_program_decoders() to pxb_cxl reset exit.
>> Every device has 3 reset phase: enter, hold, and exit. Currently,
>> pxb-cxl has:
>> - enter = none
>> - hold = pxb_cxL-dev_reset(), which zeroes registers
>> - exit = none
>>
>> Register cxl_fmws_program_decoders() to pxb-cxl reset exit.
>>
>> cxl-host.c:
>> +void cxl_fmws_for_each_auto_range(PXBCXLDev *pxb)
>> {
>>
>> + /* use existing helper cxl_fmws_get_all_sorted to iterate */
>> + GSList *fws = cxl_fmws_get_all_sorted(), *it;
>> +
>> +for (it = fws; it; it = it->next) {
>> + CXLFixedWindow *fw = CXL_FMW(it->data);
>> /* program decoders here */
>>
>> + }
>> }
>> ...
>>
>> +/* Call from the pxb-cxl reset exit phase, see pxb_cxl_dev_reset_exit() */
>> +void cxl_fmws_program_decoders(PXBCXLDev *pxb)
>> +{
>> + cxl_fmws_for_each_auto_range(pxb);
>> +}
>>
>> --- a/hw/pci-bridge/pci_expander_bridge.c
>> +++ b/hw/pci-bridge/pci_expander_bridge.c
>> #include "hw/cxl/cxl.h"
>> +#include "hw/cxl/cxl_host.h"
>>
>> +static void pxb_cxl_dev_reset_exit(Object *obj, ResetType type)
>> +{
>> + PXBCXLDev *pxb = PXB_CXL_DEV(obj);
>> + PXBCXLDevClass *pc = PXB_CXL_DEV_GET_CLASS(obj);
>> +
>> + if (pc->parent_phases.exit) {
>> + pc->parent_phases.exit(obj, type);
>> + }
>> + if (pxb->cxl_state && pxb->cxl_state->program_hdm_decoders) {
>> + cxl_fmws_program_decoders(pxb);
>> + }
>> +}
>> +
>> static void pxb_cxl_dev_class_init(ObjectClass *klass, const void *data)
>> {
>> + ResettableClass *rc = RESETTABLE_CLASS(klass);
>> ...
>> device_class_set_legacy_reset(dc, pxb_cxl_dev_reset);
>> + rc->phases.exit = pxb_cxl_dev_reset_exit;
>> }
>>
>> This way, we shouldn't need to store the extra state (hpa_offset,etc.)
>> in the cxl-type3 endpoints.
>>
>> I'm also not that familiar with the machine initialization and reset
>> stages, so maybe this approach doesn't work, but this is my best guess.
>>
>> Thanks,
>> Anisa
>>
>>> Testing
>>> We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
>>> and E820_SOFT_RESERVED entries appeared in the E820 table:
>>>
>>> - BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
>>> - CXL memory: 4 GB
>>> - CFMWS: 4 GB
>>> - Host bridge: 1
>>>
>>> The following is the startup script.
>>>
>>> sudo /path/to/qemu/build/qemu-system-x86_64 \
>>> -name guest=cxl \
>>> -nographic \
>>> -machine q35,accel=tcg,nvdimm=on,cxl=on \
>>> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
>>> -m size=4G,slots=10,maxmem=20G \
>>> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
>>> -bios /path/to/seabios/out/bios.bin \
>>> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
>>> -object memory-backend-ram,size=2G,id=m0 \
>>> -object memory-backend-ram,size=2G,id=m1 \
>>> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
>>> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
>>> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
>>> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
>>> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
>>> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
>>> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
>>> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
>>>
>>> Furthermore, we conducted the experiment in which we added two CXL Type3.
>>>
>>> - CXL memory: 4 GB × 2
>>> - CFMWS: 4 GB × 2
>>>
>>> The following is the startup script.
>>>
>>> sudo /path/to/qemu/build/qemu-system-x86_64 \
>>> -name guest=cxl \
>>> -nographic \
>>> -machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
>>> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
>>> -m size=4G,slots=10,maxmem=20G \
>>> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
>>> -bios /path/to/seabios/out/bios.bin \
>>> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
>>> -object memory-backend-ram,size=2G,id=m0 \
>>> -object memory-backend-ram,size=2G,id=m1 \
>>> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
>>> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
>>> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
>>> -device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
>>> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
>>> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
>>> -device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
>>> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
>>> -object memory-backend-ram,size=4G,share=on,id=vmem1 \
>>> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
>>> -device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
>>> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
>>>
>>>
>>> How to use
>>> Specify the following parameters when starting QEMU:
>>>
>>> - bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
>>> - Add hmat=on to the -machine option [1]
>>> - Add program-hdm-decoder=true to each -device cxl-type3 option
>>> [1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
>>> these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
>>> SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
>>> so HMAT must be enabled.
>>>
>>> QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
>>>
>>> TODO
>>> With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
>>> As a result, automatic creation of CXL regions in the guest OS may fail:
>>>
>>> - A CXL persistent memory device is included
>>> - Interleaved configurations
>>> - Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
>>> - Configurations where CXL memory is attached via a switch
>>> - Multi-target CFMWS environments
>>> - ARM environments
>>> - HMAT does not yet include information about CXL memory devices
>>>
>>> The following behaviors are also provisional implementations and may need further refinement:
>>> - A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
>>> or to allow the NUMA node IDs to be specified by the user.
>>> - Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
>>> based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
>>> through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
>>> This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
>>> adding an explicit option to associate each Type 3 device with a CFMWS entry,
>>> or generating CFMWS entries automatically from the configured CXL topology and devices.
>>> - HDM decoder reset sequencing and state reconstruction.
>>> The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
>>> by moving the common initialization of the host bridge component registers from the reset path to the realization path.
>>> However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
>>> automatically programmed by QEMU should be reconstructed as part of reset processing.
>>>
>>> Thanks,
>>>
>>> ------
>>>
>>> Yukiya Takaki (7):
>>> QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
>>> QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
>>> QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
>>> QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
>>> QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
>>> QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
>>> add CXL Type 3 memory ranges to SRAT
>>>
>>> hw/acpi/cxl_type3.c | 30 +++++
>>> hw/acpi/meson.build | 1 +
>>> hw/arm/cxl.c | 10 ++
>>> hw/arm/meson.build | 2 +
>>> hw/cxl/cxl-host.c | 63 ++++++++++
>>> hw/i386/acpi-build.c | 7 ++
>>> hw/i386/cxl.c | 11 ++
>>> hw/i386/meson.build | 1 +
>>> hw/i386/pc.c | 2 +
>>> hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
>>> hw/pci-bridge/pci_expander_bridge.c | 13 +-
>>> include/hw/cxl/cxl.h | 2 +
>>> include/hw/cxl/cxl_device.h | 6 +
>>> include/hw/cxl/cxl_host.h | 3 +
>>> include/hw/mem/cxl_type3.h | 8 ++
>>> 15 files changed, 334 insertions(+), 2 deletions(-)
>>> create mode 100644 hw/acpi/cxl_type3.c
>>> create mode 100644 hw/arm/cxl.c
>>> create mode 100644 hw/i386/cxl.c
>>> create mode 100644 include/hw/mem/cxl_type3.h
>>>
>>> --
>>> 2.52.0
>>>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
2026-10-26 10:03 ` Yukiya Takaki via
` (9 preceding siblings ...)
(?)
@ 2026-11-05 21:54 ` Jonathan Cameron
2026-11-07 18:39 ` Yukiya Takaki
-1 siblings, 1 reply; 24+ messages in thread
From: Jonathan Cameron @ 2026-11-05 21:54 UTC (permalink / raw)
To: Yukiya Takaki
Cc: qemu-devel, Michael S. Tsirkin, Ani Sinha, Anisa Su,
Igor Mammedov, Junjie Cao, Paolo Bonzini, Peter Maydell,
Richard Henderson, linux-cxl, qemu-arm, Wolski, Tomasz, lizhijian,
y-goto, yukiya.takaki
On Mon, 26 Oct 2026 19:03:21 +0900
Yukiya Takaki <fj2110jc@fujitsu.com> wrote:
> Hello everyone,
>
> I’m a student intern, and this is my first time sending patches to an open source mailing list.
>
> This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
> and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
Hi Yukiya,
Interesting project.
>
> Background and motivation
> Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
> to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
> By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
> allowing the Linux CXL driver to discover and create regions during boot.
> This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
>
> Some parts of this implementation are based on Li, Zhijian’s work in:
> https://github.com/zhijianli88/qemu/tree/program-decoder-v2
>
> I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
>
> Approach
> This patch first adds a program_hdm_decoder option to cxl-type3.
> When program-hdm-decoder is enabled on cxl-type3, QEMU will:
> 1) program endpoint/host-bridge HDM decoder state,
> 2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
> 3) create memory-only NUMA nodes for those ranges,
> 4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
>
> On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
> However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
> into SeaBIOS would introduce additional complexity and risk inconsistencies.
So, did you explore doing this in a similar fashion to PCI enumeration?
That's done the bios and is very similar in that there is a mix of
policy and rules in how it is configured that we really don't want
in qemu. With that ACPI tables are then built in QEMU that contain
values read back from what was setup. The main thing I'm thinking of
here is the PXB handling that relies on the bios having done the
PCI enumeration first rather than doing it in qemu.
The arm code is here
https://elixir.bootlin.com/qemu/v11.1.3/source/hw/pci-host/gpex-acpi.c#L103
x86 probably this:
https://elixir.bootlin.com/qemu/v11.1.3/source/hw/i386/acpi-build.c#L1038
My immediate instinct is that would look more like a conventional bios
flow. It would also have the significant benefit of putting the enumerating
and configuration code into an open source bios, something I hoped would come out
early ARM work on this (see linux plumbers - I think this is right session)
https://lpc.events/event/16/contributions/1254/attachments/1113/2136/CXL%20Type-3%20device%20discovery,%20configuration%20in%20firmware%20and%20prepare%20ACPI%20tables%20for%20kernel%20usage.pdf
Then we'd regenerate the SRAT, HMAT etc before handling to the OS.
(I'm not sure this works for e820 having never touched that flow).
There is a lot of policy involved in CXL configuration and I'd not
want to replicate that part in QEMU. At best we'd want to pass
some hints to the bios.
Flow wise that would mean adding to acpi_build_update() some code
to do the necessary srat and other table updates. That would
definitely work for the actual ACPI tables, but I gather e820 is
rather different... However, e820 real table is I think
actually constructed by seabios so can that just make the
necessary updates after programming the CXL decoders etc?
Now if you did this on SBSA-ref and arm you'd have to do it
all in the bios (as that generates everything) and the whole
problem would be much more standard.
Given we have that option on arm/sbsa-ref I don't currently
see a reason to explore the bios flows on arm/virt.
I also gather (from asking an LLM so might be wrong) that there
are options that boot qemu on x86 with tables generated
by the bios, not qemu. Legacy path in seabios, coreboot and
and ovmf path used with xen. Maybe working with them avoids
the extra complexity of bounce backwards and forwards between
qemu and the firmware (though I think I'd probably avoid these).
Anyhow, interested to see where you go with this.
Definitely useful for those of us working with platforms that
never do the bios programmed flow but have to test it when
making changes!
Jonathan
>
> Testing
> We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
> and E820_SOFT_RESERVED entries appeared in the E820 table:
>
> - BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
> - CXL memory: 4 GB
> - CFMWS: 4 GB
> - Host bridge: 1
>
> The following is the startup script.
>
> sudo /path/to/qemu/build/qemu-system-x86_64 \
> -name guest=cxl \
> -nographic \
> -machine q35,accel=tcg,nvdimm=on,cxl=on \
> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
> -m size=4G,slots=10,maxmem=20G \
> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
> -bios /path/to/seabios/out/bios.bin \
> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
> -object memory-backend-ram,size=2G,id=m0 \
> -object memory-backend-ram,size=2G,id=m1 \
> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
>
> Furthermore, we conducted the experiment in which we added two CXL Type3.
>
> - CXL memory: 4 GB × 2
> - CFMWS: 4 GB × 2
>
> The following is the startup script.
>
> sudo /path/to/qemu/build/qemu-system-x86_64 \
> -name guest=cxl \
> -nographic \
> -machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
> -m size=4G,slots=10,maxmem=20G \
> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
> -bios /path/to/seabios/out/bios.bin \
> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
> -object memory-backend-ram,size=2G,id=m0 \
> -object memory-backend-ram,size=2G,id=m1 \
> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
> -device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
> -device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
> -object memory-backend-ram,size=4G,share=on,id=vmem1 \
> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
> -device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
>
>
> How to use
> Specify the following parameters when starting QEMU:
>
> - bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
> - Add hmat=on to the -machine option [1]
> - Add program-hdm-decoder=true to each -device cxl-type3 option
> [1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
> these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
> SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
> so HMAT must be enabled.
>
> QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
>
> TODO
> With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
> As a result, automatic creation of CXL regions in the guest OS may fail:
>
> - A CXL persistent memory device is included
> - Interleaved configurations
> - Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
> - Configurations where CXL memory is attached via a switch
> - Multi-target CFMWS environments
> - ARM environments
> - HMAT does not yet include information about CXL memory devices
>
> The following behaviors are also provisional implementations and may need further refinement:
> - A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
> or to allow the NUMA node IDs to be specified by the user.
> - Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
> based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
> through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
> This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
> adding an explicit option to associate each Type 3 device with a CFMWS entry,
> or generating CFMWS entries automatically from the configured CXL topology and devices.
> - HDM decoder reset sequencing and state reconstruction.
> The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
> by moving the common initialization of the host bridge component registers from the reset path to the realization path.
> However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
> automatically programmed by QEMU should be reconstructed as part of reset processing.
>
> Thanks,
>
> ------
>
> Yukiya Takaki (7):
> QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
> QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
> QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
> QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
> QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
> QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
> add CXL Type 3 memory ranges to SRAT
>
> hw/acpi/cxl_type3.c | 30 +++++
> hw/acpi/meson.build | 1 +
> hw/arm/cxl.c | 10 ++
> hw/arm/meson.build | 2 +
> hw/cxl/cxl-host.c | 63 ++++++++++
> hw/i386/acpi-build.c | 7 ++
> hw/i386/cxl.c | 11 ++
> hw/i386/meson.build | 1 +
> hw/i386/pc.c | 2 +
> hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
> hw/pci-bridge/pci_expander_bridge.c | 13 +-
> include/hw/cxl/cxl.h | 2 +
> include/hw/cxl/cxl_device.h | 6 +
> include/hw/cxl/cxl_host.h | 3 +
> include/hw/mem/cxl_type3.h | 8 ++
> 15 files changed, 334 insertions(+), 2 deletions(-)
> create mode 100644 hw/acpi/cxl_type3.c
> create mode 100644 hw/arm/cxl.c
> create mode 100644 hw/i386/cxl.c
> create mode 100644 include/hw/mem/cxl_type3.h
>
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH RFC 0/7] QEMU/hw/cxl: Allow CXL Type 3 memory regions to be initialized at guest boot
2026-11-05 21:54 ` Jonathan Cameron
@ 2026-11-07 18:39 ` Yukiya Takaki
0 siblings, 0 replies; 24+ messages in thread
From: Yukiya Takaki @ 2026-11-07 18:39 UTC (permalink / raw)
To: Jonathan Cameron, Yukiya Takaki
Cc: qemu-devel, Michael S. Tsirkin, Ani Sinha, Anisa Su,
Igor Mammedov, Junjie Cao, Paolo Bonzini, Peter Maydell,
Richard Henderson, linux-cxl, qemu-arm, Wolski, Tomasz, lizhijian,
y-goto
Hi Jonathan,
Thank you for your suggestion.
> On Mon, 26 Oct 2026 19:03:21 +0900
> Yukiya Takaki <fj2110jc@fujitsu.com> wrote:
>
>> Hello everyone,
>>
>> I’m a student intern, and this is my first time sending patches to an open source mailing list.
>>
>> This patch series configures QEMU’s CXL Type 3 devices so that guest operating systems can automatically discover
>> and create CXL regions during boot, aligning QEMU’s behavior with real hardware.
> Hi Yukiya,
>
> Interesting project.
>
>> Background and motivation
>> Currently, during guest boot QEMU does not provide the information or register programming needed for the guest OS
>> to initialize CXL Type 3 memory, so using it still requires an explicit action in the guest (for example, running the cxl create-region command).
>> By contrast, on some physical systems the firmware automatically allocates CXL memory ranges and exposes them to the OS,
>> allowing the Linux CXL driver to discover and create regions during boot.
>> This patch series is a first, experimental attempt to reproduce that behavior in QEMU.
>>
>> Some parts of this implementation are based on Li, Zhijian’s work in:
>> https://github.com/zhijianli88/qemu/tree/program-decoder-v2
>>
>> I would like to ask for feedback on whether this approach makes sense and whether I should continue to enhance it.
>>
>> Approach
>> This patch first adds a program_hdm_decoder option to cxl-type3.
>> When program-hdm-decoder is enabled on cxl-type3, QEMU will:
>> 1) program endpoint/host-bridge HDM decoder state,
>> 2) expose Type 3 ranges as E820_SOFT_RESERVED on x86,
>> 3) create memory-only NUMA nodes for those ranges,
>> 4) emit SRAT memory-affinity entries mapping each Type 3 HPA range to its node ID
>>
>> On real hardware, HDM decoder programming and memory layout are typically handled by firmware, so they would ideally be managed by SeaBIOS.
>> However, in the current QEMU/SeaBIOS model, E820 and SRAT are built on the QEMU side, and pushing HDM decoder programming
>> into SeaBIOS would introduce additional complexity and risk inconsistencies.
> So, did you explore doing this in a similar fashion to PCI enumeration?
>
> That's done the bios and is very similar in that there is a mix of
> policy and rules in how it is configured that we really don't want
> in qemu. With that ACPI tables are then built in QEMU that contain
> values read back from what was setup. The main thing I'm thinking of
> here is the PXB handling that relies on the bios having done the
> PCI enumeration first rather than doing it in qemu.
> The arm code is here
> https://elixir.bootlin.com/qemu/v11.1.3/source/hw/pci-host/gpex-acpi.c#L103
> x86 probably this:
> https://elixir.bootlin.com/qemu/v11.1.3/source/hw/i386/acpi-build.c#L1038
>
> My immediate instinct is that would look more like a conventional bios
> flow. It would also have the significant benefit of putting the enumerating
> and configuration code into an open source bios, something I hoped would come out
> early ARM work on this (see linux plumbers - I think this is right session)
> https://lpc.events/event/16/contributions/1254/attachments/1113/2136/CXL%20Type-3%20device%20discovery,%20configuration%20in%20firmware%20and%20prepare%20ACPI%20tables%20for%20kernel%20usage.pdf
>
> Then we'd regenerate the SRAT, HMAT etc before handling to the OS.
> (I'm not sure this works for e820 having never touched that flow).
>
> There is a lot of policy involved in CXL configuration and I'd not
> want to replicate that part in QEMU. At best we'd want to pass
> some hints to the bios.
>
> Flow wise that would mean adding to acpi_build_update() some code
> to do the necessary srat and other table updates. That would
> definitely work for the actual ACPI tables, but I gather e820 is
> rather different... However, e820 real table is I think
> actually constructed by seabios so can that just make the
> necessary updates after programming the CXL decoders etc?
>
> Now if you did this on SBSA-ref and arm you'd have to do it
> all in the bios (as that generates everything) and the whole
> problem would be much more standard.
>
> Given we have that option on arm/sbsa-ref I don't currently
> see a reason to explore the bios flows on arm/virt.
>
> I also gather (from asking an LLM so might be wrong) that there
> are options that boot qemu on x86 with tables generated
> by the bios, not qemu. Legacy path in seabios, coreboot and
> and ovmf path used with xen. Maybe working with them avoids
> the extra complexity of bounce backwards and forwards between
> qemu and the firmware (though I think I'd probably avoid these).
>
> Anyhow, interested to see where you go with this.
> Definitely useful for those of us working with platforms that
> never do the bios programmed flow but have to test it when
> making changes!
>
> Jonathan
I was not aware of how the PCI enumeration and acpi_build_update() flow
works.
Using this approach, I think we can automatically configure CXL Type 3
memory
while keeping the allocation policy on the BIOS side, and then have QEMU
update the SRAT based on the programmed decoders.
Also, although I have only looked into the SeaBIOS code so far, some E820
entries are indeed added directly in SeaBIOS, so I think we can insert the
E820_SOFT_RESERVED entries on the SeaBIOS side as well.
I will work on v2 using this approach.
Best regards,
Yukiya
>> Testing
>> We ran tests in the following environment and confirmed that CXL regions were created at boot, NUMA nodes were set up,
>> and E820_SOFT_RESERVED entries appeared in the E820 table:
>>
>> - BIOS: SeaBIOS c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf
>> - CXL memory: 4 GB
>> - CFMWS: 4 GB
>> - Host bridge: 1
>>
>> The following is the startup script.
>>
>> sudo /path/to/qemu/build/qemu-system-x86_64 \
>> -name guest=cxl \
>> -nographic \
>> -machine q35,accel=tcg,nvdimm=on,cxl=on \
>> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
>> -m size=4G,slots=10,maxmem=20G \
>> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
>> -bios /path/to/seabios/out/bios.bin \
>> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
>> -object memory-backend-ram,size=2G,id=m0 \
>> -object memory-backend-ram,size=2G,id=m1 \
>> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
>> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
>> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
>> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
>> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
>> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
>> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
>> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
>>
>> Furthermore, we conducted the experiment in which we added two CXL Type3.
>>
>> - CXL memory: 4 GB × 2
>> - CFMWS: 4 GB × 2
>>
>> The following is the startup script.
>>
>> sudo /path/to/qemu/build/qemu-system-x86_64 \
>> -name guest=cxl \
>> -nographic \
>> -machine q35,accel=tcg,nvdimm=on,cxl=on,hmat=on \
>> -cpu max -smp 4,sockets=4,cores=1,threads=1 \
>> -m size=4G,slots=10,maxmem=20G \
>> -drive file=/path/to/image.qcow2,format=qcow2,index=0,media=disk,id=hd \
>> -bios /path/to/seabios/out/bios.bin \
>> -nic bridge,br=virbr0,model=e1000e,mac=52:54:00:b5:4e:7c \
>> -object memory-backend-ram,size=2G,id=m0 \
>> -object memory-backend-ram,size=2G,id=m1 \
>> -numa node,nodeid=0,cpus=0-1,memdev=m0 \
>> -numa node,nodeid=1,cpus=2-3,memdev=m1 \
>> -device pcie-root-port,id=pci-root,slot=4,bus=pcie.0,chassis=0 \
>> -device pcie-root-port,id=pci-root-net,chassis=5,slot=5,bus=pcie.0 \
>> -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1,hdm_for_passthrough=true \
>> -device cxl-rp,port=0,id=root_port13,bus=cxl.1,chassis=0,slot=2 \
>> -device cxl-rp,port=3,id=root_port14,bus=cxl.1,chassis=2,slot=4 \
>> -object memory-backend-ram,size=4G,share=on,id=vmem0 \
>> -object memory-backend-ram,size=4G,share=on,id=vmem1 \
>> -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,program-hdm-decoder=true \
>> -device cxl-type3,bus=root_port14,volatile-memdev=vmem1,id=cxl-vmem1,program-hdm-decoder=true \
>> -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k,cxl-fmw.1.targets.0=cxl.1,cxl-fmw.1.size=4G,cxl-fmw.1.interleave-granularity=4k
>>
>>
>> How to use
>> Specify the following parameters when starting QEMU:
>>
>> - bios <SeaBIOS binary that includes commit c2a33ad9ad1452e23b41c4ac44a3bc6be8ebc4cf> to ensure that E820_SOFT_RESERVED entries are exposed
>> - Add hmat=on to the -machine option [1]
>> - Add program-hdm-decoder=true to each -device cxl-type3 option
>> [1] On Linux, the kernel first checks E820_SOFT_RESERVED ranges. When multiple CXL Type 3 devices share a contiguous HPA range,
>> these ranges tend to be merged into a single region, so the CXL driver cannot reliably initialize per‑device regions based on E820 alone.
>> SRAT entries for CXL memory are therefore needed, and the guest seems to discard SRAT when `hmat=on` is not specified,
>> so HMAT must be enabled.
>>
>> QEMU base commit: bddb785e791e415db38884848d068457e226f5e3
>>
>> TODO
>> With the current implementation of this patch, the following configurations are either not supported or have not been sufficiently tested.
>> As a result, automatic creation of CXL regions in the guest OS may fail:
>>
>> - A CXL persistent memory device is included
>> - Interleaved configurations
>> - Cases where the memory size specified by CFMWS does not match the size of the CXL Type 3 backend memory
>> - Configurations where CXL memory is attached via a switch
>> - Multi-target CFMWS environments
>> - ARM environments
>> - HMAT does not yet include information about CXL memory devices
>>
>> The following behaviors are also provisional implementations and may need further refinement:
>> - A new NUMA node ID is assigned to each CXL memory device. This should be improved either to reflect more accurate NUMA distances for CXL memory,
>> or to allow the NUMA node IDs to be specified by the user.
>> - Assignment of CXL Type 3 devices to CFMWS address ranges. On physical systems, platform firmware typically generates CFMWS entries
>> based on the detected CXL topology and its address-space allocation. In QEMU, however, CFMWS entries are configured explicitly
>> through machine options. The current implementation therefore has to infer which CFMWS range should be used for each Type 3 device.
>> This selection logic may need further refinement. Possible approaches include improving the selection and validation logic,
>> adding an explicit option to associate each Type 3 device with a CFMWS entry,
>> or generating CFMWS entries automatically from the configured CXL topology and devices.
>> - HDM decoder reset sequencing and state reconstruction.
>> The current implementation prevents the host bridge HDM decoder configuration from being overwritten later in the same reset
>> by moving the common initialization of the host bridge component registers from the reset path to the realization path.
>> However, ideally, after the registers have been restored to their reset values, the host bridge HDM decoder configuration
>> automatically programmed by QEMU should be reconstructed as part of reset processing.
>>
>> Thanks,
>>
>> ------
>>
>> Yukiya Takaki (7):
>> QEMU/hw/cxl: add program-hdm-decoder option for CXL Type 3 devices
>> QEMU/hw/cxl:add HPA offset tracking for CXL Type 3 devices
>> QEMU: hw/cxl: add helper to get HPA range for CXL Type 3
>> QEMU/hw/cxl: auto‑program HDM decoders for CXL Type 3
>> QEMU/hw/cxl: expose CXL Type 3 ranges as E820_SOFT_RESERVED on x86
>> QEMU/hw/cxl: Create NUMA nodes for CXL Type 3 memory
>> add CXL Type 3 memory ranges to SRAT
>>
>> hw/acpi/cxl_type3.c | 30 +++++
>> hw/acpi/meson.build | 1 +
>> hw/arm/cxl.c | 10 ++
>> hw/arm/meson.build | 2 +
>> hw/cxl/cxl-host.c | 63 ++++++++++
>> hw/i386/acpi-build.c | 7 ++
>> hw/i386/cxl.c | 11 ++
>> hw/i386/meson.build | 1 +
>> hw/i386/pc.c | 2 +
>> hw/mem/cxl_type3.c | 177 ++++++++++++++++++++++++++++
>> hw/pci-bridge/pci_expander_bridge.c | 13 +-
>> include/hw/cxl/cxl.h | 2 +
>> include/hw/cxl/cxl_device.h | 6 +
>> include/hw/cxl/cxl_host.h | 3 +
>> include/hw/mem/cxl_type3.h | 8 ++
>> 15 files changed, 334 insertions(+), 2 deletions(-)
>> create mode 100644 hw/acpi/cxl_type3.c
>> create mode 100644 hw/arm/cxl.c
>> create mode 100644 hw/i386/cxl.c
>> create mode 100644 include/hw/mem/cxl_type3.h
>>
^ permalink raw reply [flat|nested] 24+ messages in thread