* [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-06-18 9:38 ` Chen Pei
0 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-06-18 9:38 UTC (permalink / raw)
To: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren
Cc: qemu-riscv, qemu-devel, linux-cxl
CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
and the CXL device register BAR (BAR2 on Type3 device) are declared
as 64-bit non-prefetchable memory. Per the PCI-to-PCI Bridge
Architecture Specification Rev 1.2 (PCI-SIG, 2003):
- §3.2.5.8 (Memory Base/Limit): the non-prefetchable window covers
only 32-bit addresses (AD[31:20]); the Type 1 header defines no
upper-32-bit extension for it.
- §3.2.5.9 (Prefetchable Memory Base/Limit): the bottom 4 bits
encode 64-bit support (01h), but this applies exclusively to the
*prefetchable* window.
- §3.2.5.10 (Prefetchable Base/Limit Upper 32 Bits): optional
registers for AD[63:32] of the prefetchable range only.
The architecture therefore allows a 64-bit window only when it is also
prefetchable; there is no 64-bit non-prefetchable form. PCIe inherits
this Type 1 header layout unchanged. Linux thus places 64-bit
non-prefetchable BARs in the 32-bit non-prefetchable bridge window,
which requires the bridge to own enough address space below 4 GiB.
On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
have no non-prefetchable window and Linux fails to assign these BARs.
Marking the BARs prefetchable would work around it, but the CXL
component registers have read/write side effects and are not
prefetchable per the PCIe specification.
Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
CXL host bridges:
- Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
- Store the reserved range in a new gpex_cfg.cxl_mmio32 field
- In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
in the CXL host bridge _CRS instead of re-using build_crs() (which
returns an empty set when UEFI has not assigned resources yet)
- Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
firmware driven by device-tree also respects the reservation
Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
hw/pci-host/gpex-acpi.c | 36 +++++++++++++++++++++--
hw/riscv/virt.c | 58 +++++++++++++++++++++++++++++++-------
include/hw/pci-host/gpex.h | 1 +
3 files changed, 83 insertions(+), 12 deletions(-)
diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
index d9820f9b41..d8b943b665 100644
--- a/hw/pci-host/gpex-acpi.c
+++ b/hw/pci-host/gpex-acpi.c
@@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
* Resources defined for PXBs are composed of the following parts:
* 1. The resources the pci-bridge/pcie-root-port need.
* 2. The resources the devices behind pxb need.
+ *
+ * For CXL host bridges on platforms where UEFI (driven by
+ * FDT 'ranges') does not assign PCI resources for the CXL
+ * root bridge before ACPI table construction, build_crs()
+ * would return an empty resource set. When the platform
+ * has reserved a dedicated MMIO window for CXL host bridges
+ * (cfg->cxl_mmio32), emit that window as a static _CRS
+ * instead. The platform is responsible for shrinking PCI0's
+ * mmio32 window so the two do not overlap.
*/
- crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
- cfg->pio.base, 0, 0, 0);
+ if (is_cxl && cfg->cxl_mmio32.size) {
+ uint64_t cxl_base = cfg->cxl_mmio32.base;
+ uint64_t cxl_size = cfg->cxl_mmio32.size;
+
+ crs = aml_resource_template();
+
+ /* 32-bit MMIO range for CXL devices */
+ aml_append(crs,
+ aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
+ AML_MAX_FIXED, AML_NON_CACHEABLE,
+ AML_READ_WRITE, 0,
+ cxl_base,
+ cxl_base + cxl_size - 1,
+ 0, cxl_size));
+
+ /* Bus number range */
+ aml_append(crs,
+ aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
+ AML_POS_DECODE, 0,
+ bus_num, bus_num + 15,
+ 0, 16));
+ } else {
+ crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
+ &crs_range_set, cfg->pio.base, 0, 0, 0);
+ }
aml_append(dev, aml_name_decl("_CRS", crs));
if (is_cxl) {
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 84b91b4322..9c1a001553 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
/* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
#define VIRT64_HIGH_PCIE_MMIO_SIZE (16 * GiB)
+/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
+#define VIRT_CXL_MMIO32_SIZE (256 * MiB)
+
static MemMapEntry virt_high_pcie_memmap;
#define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
@@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
}
qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
- qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
- 1, FDT_PCI_RANGE_IOPORT, 2, 0,
- 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
- 1, FDT_PCI_RANGE_MMIO,
- 2, s->memmap[VIRT_PCIE_MMIO].base,
- 2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
- 1, FDT_PCI_RANGE_MMIO_64BIT,
- 2, virt_high_pcie_memmap.base,
- 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
+ {
+ /*
+ * When CXL is enabled, reserve the last 256 MiB of the 32-bit
+ * MMIO window for CXL host bridges and exclude it from the main
+ * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
+ * does not allocate that range to PCI0. The CXL host bridge
+ * _CRS declares this range independently.
+ */
+ hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
+ if (s->cxl_devices_state.is_enabled) {
+ mmio32_size -= VIRT_CXL_MMIO32_SIZE;
+ }
+ qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
+ 1, FDT_PCI_RANGE_IOPORT, 2, 0,
+ 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
+ 1, FDT_PCI_RANGE_MMIO,
+ 2, s->memmap[VIRT_PCIE_MMIO].base,
+ 2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
+ 1, FDT_PCI_RANGE_MMIO_64BIT,
+ 2, virt_high_pcie_memmap.base,
+ 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
+ }
if (virt_is_iommu_sys_enabled(s)) {
qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
@@ -1728,7 +1744,29 @@ static void virt_machine_init(MachineState *machine)
qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
}
- gpex_pcie_init(system_memory, pcie_irqchip, s);
+ DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
+
+ /*
+ * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
+ * window for CXL host bridges so the bridge non-prefetchable window
+ * can hold CXL device BARs (component registers and similar 64-bit
+ * non-prefetchable BARs that need a < 4 GiB address).
+ *
+ * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
+ * 256 MiB so the two ranges do not overlap (the FDT 'ranges'
+ * shrink happens in create_fdt_pcie()).
+ * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
+ * a correct _CRS for the CXL host bridge (ACPI0016).
+ */
+ if (s->cxl_devices_state.is_enabled) {
+ GPEXHost *gpex = GPEX_HOST(pcie_dev);
+ gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
+ gpex->gpex_cfg.cxl_mmio32.base =
+ s->memmap[VIRT_PCIE_MMIO].base +
+ s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
+ /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
+ gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
+ }
create_platform_bus(s, mmio_irqchip);
diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
index 1da9c85bce..d38fbbacd6 100644
--- a/include/hw/pci-host/gpex.h
+++ b/include/hw/pci-host/gpex.h
@@ -43,6 +43,7 @@ struct GPEXConfig {
MemMapEntry mmio32;
MemMapEntry mmio64;
MemMapEntry pio;
+ MemMapEntry cxl_mmio32;
int irq;
PCIBus *bus;
bool pci_native_hotplug;
--
2.50.1
^ permalink raw reply related [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
2026-06-18 9:38 ` [PATCH v2 3/4] hw/riscv/virt, gpex: " Chen Pei
(?)
@ 2026-06-24 17:21 ` Daniel Henrique Barboza via
-1 siblings, 0 replies; 24+ messages in thread
From: Daniel Henrique Barboza @ 2026-06-24 17:21 UTC (permalink / raw)
To: Chen Pei, jic23, pbonzini, palmer, alistair.francis, liwei1518,
zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren
Cc: qemu-riscv, qemu-devel, linux-cxl
On 6/18/2026 6:38 AM, Chen Pei wrote:
> CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
> and the CXL device register BAR (BAR2 on Type3 device) are declared
> as 64-bit non-prefetchable memory. Per the PCI-to-PCI Bridge
> Architecture Specification Rev 1.2 (PCI-SIG, 2003):
>
> - §3.2.5.8 (Memory Base/Limit): the non-prefetchable window covers
> only 32-bit addresses (AD[31:20]); the Type 1 header defines no
> upper-32-bit extension for it.
> - §3.2.5.9 (Prefetchable Memory Base/Limit): the bottom 4 bits
> encode 64-bit support (01h), but this applies exclusively to the
> *prefetchable* window.
> - §3.2.5.10 (Prefetchable Base/Limit Upper 32 Bits): optional
> registers for AD[63:32] of the prefetchable range only.
>
> The architecture therefore allows a 64-bit window only when it is also
> prefetchable; there is no 64-bit non-prefetchable form. PCIe inherits
> this Type 1 header layout unchanged. Linux thus places 64-bit
> non-prefetchable BARs in the 32-bit non-prefetchable bridge window,
> which requires the bridge to own enough address space below 4 GiB.
>
> On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
> currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
> have no non-prefetchable window and Linux fails to assign these BARs.
> Marking the BARs prefetchable would work around it, but the CXL
> component registers have read/write side effects and are not
> prefetchable per the PCIe specification.
>
> Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
> CXL host bridges:
> - Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
> PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
> - Store the reserved range in a new gpex_cfg.cxl_mmio32 field
> - In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
> in the CXL host bridge _CRS instead of re-using build_crs() (which
> returns an empty set when UEFI has not assigned resources yet)
> - Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
> firmware driven by device-tree also respects the reservation
>
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> ---
> hw/pci-host/gpex-acpi.c | 36 +++++++++++++++++++++--
> hw/riscv/virt.c | 58 +++++++++++++++++++++++++++++++-------
> include/hw/pci-host/gpex.h | 1 +
> 3 files changed, 83 insertions(+), 12 deletions(-)
>
> diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
> index d9820f9b41..d8b943b665 100644
> --- a/hw/pci-host/gpex-acpi.c
> +++ b/hw/pci-host/gpex-acpi.c
> @@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
> * Resources defined for PXBs are composed of the following parts:
> * 1. The resources the pci-bridge/pcie-root-port need.
> * 2. The resources the devices behind pxb need.
> + *
> + * For CXL host bridges on platforms where UEFI (driven by
> + * FDT 'ranges') does not assign PCI resources for the CXL
> + * root bridge before ACPI table construction, build_crs()
> + * would return an empty resource set. When the platform
> + * has reserved a dedicated MMIO window for CXL host bridges
> + * (cfg->cxl_mmio32), emit that window as a static _CRS
> + * instead. The platform is responsible for shrinking PCI0's
> + * mmio32 window so the two do not overlap.
> */
> - crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
> - cfg->pio.base, 0, 0, 0);
> + if (is_cxl && cfg->cxl_mmio32.size) {
> + uint64_t cxl_base = cfg->cxl_mmio32.base;
> + uint64_t cxl_size = cfg->cxl_mmio32.size;
> +
> + crs = aml_resource_template();
> +
> + /* 32-bit MMIO range for CXL devices */
> + aml_append(crs,
> + aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> + AML_MAX_FIXED, AML_NON_CACHEABLE,
> + AML_READ_WRITE, 0,
> + cxl_base,
> + cxl_base + cxl_size - 1,
> + 0, cxl_size));
> +
> + /* Bus number range */
> + aml_append(crs,
> + aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
> + AML_POS_DECODE, 0,
> + bus_num, bus_num + 15,
> + 0, 16));
> + } else {
> + crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
> + &crs_range_set, cfg->pio.base, 0, 0, 0);
> + }
> aml_append(dev, aml_name_decl("_CRS", crs));
>
> if (is_cxl) {
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index 84b91b4322..9c1a001553 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
> /* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
> #define VIRT64_HIGH_PCIE_MMIO_SIZE (16 * GiB)
>
> +/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
> +#define VIRT_CXL_MMIO32_SIZE (256 * MiB)
> +
> static MemMapEntry virt_high_pcie_memmap;
>
> #define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
> @@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
> }
> qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
> s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
> - qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> - 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> - 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> - 1, FDT_PCI_RANGE_MMIO,
> - 2, s->memmap[VIRT_PCIE_MMIO].base,
> - 2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
> - 1, FDT_PCI_RANGE_MMIO_64BIT,
> - 2, virt_high_pcie_memmap.base,
> - 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + {
> + /*
> + * When CXL is enabled, reserve the last 256 MiB of the 32-bit
> + * MMIO window for CXL host bridges and exclude it from the main
> + * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
> + * does not allocate that range to PCI0. The CXL host bridge
> + * _CRS declares this range independently.
> + */
> + hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
> + if (s->cxl_devices_state.is_enabled) {
> + mmio32_size -= VIRT_CXL_MMIO32_SIZE;
> + }
> + qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> + 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> + 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> + 1, FDT_PCI_RANGE_MMIO,
> + 2, s->memmap[VIRT_PCIE_MMIO].base,
> + 2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
> + 1, FDT_PCI_RANGE_MMIO_64BIT,
> + 2, virt_high_pcie_memmap.base,
> + 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + }
We can get rid of the braces - it's just adding an extra identation level while
not doing anything special. 'mmio32_size' can be declared at the start
of create_fdt_pcie().
Thanks,
Daniel
>
> if (virt_is_iommu_sys_enabled(s)) {
> qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
> @@ -1728,7 +1744,29 @@ static void virt_machine_init(MachineState *machine)
> qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
> }
>
> - gpex_pcie_init(system_memory, pcie_irqchip, s);
> + DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
> +
> + /*
> + * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
> + * window for CXL host bridges so the bridge non-prefetchable window
> + * can hold CXL device BARs (component registers and similar 64-bit
> + * non-prefetchable BARs that need a < 4 GiB address).
> + *
> + * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
> + * 256 MiB so the two ranges do not overlap (the FDT 'ranges'
> + * shrink happens in create_fdt_pcie()).
> + * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
> + * a correct _CRS for the CXL host bridge (ACPI0016).
> + */
> + if (s->cxl_devices_state.is_enabled) {
> + GPEXHost *gpex = GPEX_HOST(pcie_dev);
> + gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
> + gpex->gpex_cfg.cxl_mmio32.base =
> + s->memmap[VIRT_PCIE_MMIO].base +
> + s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
> + /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
> + gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
> + }
>
> create_platform_bus(s, mmio_irqchip);
>
> diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
> index 1da9c85bce..d38fbbacd6 100644
> --- a/include/hw/pci-host/gpex.h
> +++ b/include/hw/pci-host/gpex.h
> @@ -43,6 +43,7 @@ struct GPEXConfig {
> MemMapEntry mmio32;
> MemMapEntry mmio64;
> MemMapEntry pio;
> + MemMapEntry cxl_mmio32;
> int irq;
> PCIBus *bus;
> bool pci_native_hotplug;
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-06-24 17:21 ` Daniel Henrique Barboza via
0 siblings, 0 replies; 24+ messages in thread
From: Daniel Henrique Barboza via qemu development @ 2026-06-24 17:21 UTC (permalink / raw)
To: Chen Pei, jic23, pbonzini, palmer, alistair.francis, liwei1518,
zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren
Cc: qemu-riscv, qemu-devel, linux-cxl
On 6/18/2026 6:38 AM, Chen Pei wrote:
> CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
> and the CXL device register BAR (BAR2 on Type3 device) are declared
> as 64-bit non-prefetchable memory. Per the PCI-to-PCI Bridge
> Architecture Specification Rev 1.2 (PCI-SIG, 2003):
>
> - §3.2.5.8 (Memory Base/Limit): the non-prefetchable window covers
> only 32-bit addresses (AD[31:20]); the Type 1 header defines no
> upper-32-bit extension for it.
> - §3.2.5.9 (Prefetchable Memory Base/Limit): the bottom 4 bits
> encode 64-bit support (01h), but this applies exclusively to the
> *prefetchable* window.
> - §3.2.5.10 (Prefetchable Base/Limit Upper 32 Bits): optional
> registers for AD[63:32] of the prefetchable range only.
>
> The architecture therefore allows a 64-bit window only when it is also
> prefetchable; there is no 64-bit non-prefetchable form. PCIe inherits
> this Type 1 header layout unchanged. Linux thus places 64-bit
> non-prefetchable BARs in the 32-bit non-prefetchable bridge window,
> which requires the bridge to own enough address space below 4 GiB.
>
> On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
> currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
> have no non-prefetchable window and Linux fails to assign these BARs.
> Marking the BARs prefetchable would work around it, but the CXL
> component registers have read/write side effects and are not
> prefetchable per the PCIe specification.
>
> Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
> CXL host bridges:
> - Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
> PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
> - Store the reserved range in a new gpex_cfg.cxl_mmio32 field
> - In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
> in the CXL host bridge _CRS instead of re-using build_crs() (which
> returns an empty set when UEFI has not assigned resources yet)
> - Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
> firmware driven by device-tree also respects the reservation
>
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> ---
> hw/pci-host/gpex-acpi.c | 36 +++++++++++++++++++++--
> hw/riscv/virt.c | 58 +++++++++++++++++++++++++++++++-------
> include/hw/pci-host/gpex.h | 1 +
> 3 files changed, 83 insertions(+), 12 deletions(-)
>
> diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
> index d9820f9b41..d8b943b665 100644
> --- a/hw/pci-host/gpex-acpi.c
> +++ b/hw/pci-host/gpex-acpi.c
> @@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
> * Resources defined for PXBs are composed of the following parts:
> * 1. The resources the pci-bridge/pcie-root-port need.
> * 2. The resources the devices behind pxb need.
> + *
> + * For CXL host bridges on platforms where UEFI (driven by
> + * FDT 'ranges') does not assign PCI resources for the CXL
> + * root bridge before ACPI table construction, build_crs()
> + * would return an empty resource set. When the platform
> + * has reserved a dedicated MMIO window for CXL host bridges
> + * (cfg->cxl_mmio32), emit that window as a static _CRS
> + * instead. The platform is responsible for shrinking PCI0's
> + * mmio32 window so the two do not overlap.
> */
> - crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
> - cfg->pio.base, 0, 0, 0);
> + if (is_cxl && cfg->cxl_mmio32.size) {
> + uint64_t cxl_base = cfg->cxl_mmio32.base;
> + uint64_t cxl_size = cfg->cxl_mmio32.size;
> +
> + crs = aml_resource_template();
> +
> + /* 32-bit MMIO range for CXL devices */
> + aml_append(crs,
> + aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> + AML_MAX_FIXED, AML_NON_CACHEABLE,
> + AML_READ_WRITE, 0,
> + cxl_base,
> + cxl_base + cxl_size - 1,
> + 0, cxl_size));
> +
> + /* Bus number range */
> + aml_append(crs,
> + aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
> + AML_POS_DECODE, 0,
> + bus_num, bus_num + 15,
> + 0, 16));
> + } else {
> + crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
> + &crs_range_set, cfg->pio.base, 0, 0, 0);
> + }
> aml_append(dev, aml_name_decl("_CRS", crs));
>
> if (is_cxl) {
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index 84b91b4322..9c1a001553 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
> /* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
> #define VIRT64_HIGH_PCIE_MMIO_SIZE (16 * GiB)
>
> +/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
> +#define VIRT_CXL_MMIO32_SIZE (256 * MiB)
> +
> static MemMapEntry virt_high_pcie_memmap;
>
> #define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
> @@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
> }
> qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
> s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
> - qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> - 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> - 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> - 1, FDT_PCI_RANGE_MMIO,
> - 2, s->memmap[VIRT_PCIE_MMIO].base,
> - 2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
> - 1, FDT_PCI_RANGE_MMIO_64BIT,
> - 2, virt_high_pcie_memmap.base,
> - 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + {
> + /*
> + * When CXL is enabled, reserve the last 256 MiB of the 32-bit
> + * MMIO window for CXL host bridges and exclude it from the main
> + * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
> + * does not allocate that range to PCI0. The CXL host bridge
> + * _CRS declares this range independently.
> + */
> + hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
> + if (s->cxl_devices_state.is_enabled) {
> + mmio32_size -= VIRT_CXL_MMIO32_SIZE;
> + }
> + qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> + 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> + 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> + 1, FDT_PCI_RANGE_MMIO,
> + 2, s->memmap[VIRT_PCIE_MMIO].base,
> + 2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
> + 1, FDT_PCI_RANGE_MMIO_64BIT,
> + 2, virt_high_pcie_memmap.base,
> + 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + }
We can get rid of the braces - it's just adding an extra identation level while
not doing anything special. 'mmio32_size' can be declared at the start
of create_fdt_pcie().
Thanks,
Daniel
>
> if (virt_is_iommu_sys_enabled(s)) {
> qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
> @@ -1728,7 +1744,29 @@ static void virt_machine_init(MachineState *machine)
> qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
> }
>
> - gpex_pcie_init(system_memory, pcie_irqchip, s);
> + DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
> +
> + /*
> + * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
> + * window for CXL host bridges so the bridge non-prefetchable window
> + * can hold CXL device BARs (component registers and similar 64-bit
> + * non-prefetchable BARs that need a < 4 GiB address).
> + *
> + * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
> + * 256 MiB so the two ranges do not overlap (the FDT 'ranges'
> + * shrink happens in create_fdt_pcie()).
> + * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
> + * a correct _CRS for the CXL host bridge (ACPI0016).
> + */
> + if (s->cxl_devices_state.is_enabled) {
> + GPEXHost *gpex = GPEX_HOST(pcie_dev);
> + gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
> + gpex->gpex_cfg.cxl_mmio32.base =
> + s->memmap[VIRT_PCIE_MMIO].base +
> + s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
> + /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
> + gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
> + }
>
> create_platform_bus(s, mmio_irqchip);
>
> diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
> index 1da9c85bce..d38fbbacd6 100644
> --- a/include/hw/pci-host/gpex.h
> +++ b/include/hw/pci-host/gpex.h
> @@ -43,6 +43,7 @@ struct GPEXConfig {
> MemMapEntry mmio32;
> MemMapEntry mmio64;
> MemMapEntry pio;
> + MemMapEntry cxl_mmio32;
> int irq;
> PCIBus *bus;
> bool pci_native_hotplug;
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-06-24 17:21 ` Daniel Henrique Barboza via
0 siblings, 0 replies; 24+ messages in thread
From: Daniel Henrique Barboza via @ 2026-06-24 17:21 UTC (permalink / raw)
To: Chen Pei, jic23, pbonzini, palmer, alistair.francis, liwei1518,
zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren
Cc: qemu-riscv, qemu-devel, linux-cxl
On 6/18/2026 6:38 AM, Chen Pei wrote:
> CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
> and the CXL device register BAR (BAR2 on Type3 device) are declared
> as 64-bit non-prefetchable memory. Per the PCI-to-PCI Bridge
> Architecture Specification Rev 1.2 (PCI-SIG, 2003):
>
> - §3.2.5.8 (Memory Base/Limit): the non-prefetchable window covers
> only 32-bit addresses (AD[31:20]); the Type 1 header defines no
> upper-32-bit extension for it.
> - §3.2.5.9 (Prefetchable Memory Base/Limit): the bottom 4 bits
> encode 64-bit support (01h), but this applies exclusively to the
> *prefetchable* window.
> - §3.2.5.10 (Prefetchable Base/Limit Upper 32 Bits): optional
> registers for AD[63:32] of the prefetchable range only.
>
> The architecture therefore allows a 64-bit window only when it is also
> prefetchable; there is no 64-bit non-prefetchable form. PCIe inherits
> this Type 1 header layout unchanged. Linux thus places 64-bit
> non-prefetchable BARs in the 32-bit non-prefetchable bridge window,
> which requires the bridge to own enough address space below 4 GiB.
>
> On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
> currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
> have no non-prefetchable window and Linux fails to assign these BARs.
> Marking the BARs prefetchable would work around it, but the CXL
> component registers have read/write side effects and are not
> prefetchable per the PCIe specification.
>
> Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
> CXL host bridges:
> - Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
> PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
> - Store the reserved range in a new gpex_cfg.cxl_mmio32 field
> - In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
> in the CXL host bridge _CRS instead of re-using build_crs() (which
> returns an empty set when UEFI has not assigned resources yet)
> - Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
> firmware driven by device-tree also respects the reservation
>
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> ---
> hw/pci-host/gpex-acpi.c | 36 +++++++++++++++++++++--
> hw/riscv/virt.c | 58 +++++++++++++++++++++++++++++++-------
> include/hw/pci-host/gpex.h | 1 +
> 3 files changed, 83 insertions(+), 12 deletions(-)
>
> diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
> index d9820f9b41..d8b943b665 100644
> --- a/hw/pci-host/gpex-acpi.c
> +++ b/hw/pci-host/gpex-acpi.c
> @@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
> * Resources defined for PXBs are composed of the following parts:
> * 1. The resources the pci-bridge/pcie-root-port need.
> * 2. The resources the devices behind pxb need.
> + *
> + * For CXL host bridges on platforms where UEFI (driven by
> + * FDT 'ranges') does not assign PCI resources for the CXL
> + * root bridge before ACPI table construction, build_crs()
> + * would return an empty resource set. When the platform
> + * has reserved a dedicated MMIO window for CXL host bridges
> + * (cfg->cxl_mmio32), emit that window as a static _CRS
> + * instead. The platform is responsible for shrinking PCI0's
> + * mmio32 window so the two do not overlap.
> */
> - crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
> - cfg->pio.base, 0, 0, 0);
> + if (is_cxl && cfg->cxl_mmio32.size) {
> + uint64_t cxl_base = cfg->cxl_mmio32.base;
> + uint64_t cxl_size = cfg->cxl_mmio32.size;
> +
> + crs = aml_resource_template();
> +
> + /* 32-bit MMIO range for CXL devices */
> + aml_append(crs,
> + aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> + AML_MAX_FIXED, AML_NON_CACHEABLE,
> + AML_READ_WRITE, 0,
> + cxl_base,
> + cxl_base + cxl_size - 1,
> + 0, cxl_size));
> +
> + /* Bus number range */
> + aml_append(crs,
> + aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
> + AML_POS_DECODE, 0,
> + bus_num, bus_num + 15,
> + 0, 16));
> + } else {
> + crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
> + &crs_range_set, cfg->pio.base, 0, 0, 0);
> + }
> aml_append(dev, aml_name_decl("_CRS", crs));
>
> if (is_cxl) {
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index 84b91b4322..9c1a001553 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
> /* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
> #define VIRT64_HIGH_PCIE_MMIO_SIZE (16 * GiB)
>
> +/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
> +#define VIRT_CXL_MMIO32_SIZE (256 * MiB)
> +
> static MemMapEntry virt_high_pcie_memmap;
>
> #define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
> @@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
> }
> qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
> s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
> - qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> - 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> - 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> - 1, FDT_PCI_RANGE_MMIO,
> - 2, s->memmap[VIRT_PCIE_MMIO].base,
> - 2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
> - 1, FDT_PCI_RANGE_MMIO_64BIT,
> - 2, virt_high_pcie_memmap.base,
> - 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + {
> + /*
> + * When CXL is enabled, reserve the last 256 MiB of the 32-bit
> + * MMIO window for CXL host bridges and exclude it from the main
> + * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
> + * does not allocate that range to PCI0. The CXL host bridge
> + * _CRS declares this range independently.
> + */
> + hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
> + if (s->cxl_devices_state.is_enabled) {
> + mmio32_size -= VIRT_CXL_MMIO32_SIZE;
> + }
> + qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> + 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> + 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> + 1, FDT_PCI_RANGE_MMIO,
> + 2, s->memmap[VIRT_PCIE_MMIO].base,
> + 2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
> + 1, FDT_PCI_RANGE_MMIO_64BIT,
> + 2, virt_high_pcie_memmap.base,
> + 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + }
We can get rid of the braces - it's just adding an extra identation level while
not doing anything special. 'mmio32_size' can be declared at the start
of create_fdt_pcie().
Thanks,
Daniel
>
> if (virt_is_iommu_sys_enabled(s)) {
> qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
> @@ -1728,7 +1744,29 @@ static void virt_machine_init(MachineState *machine)
> qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
> }
>
> - gpex_pcie_init(system_memory, pcie_irqchip, s);
> + DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
> +
> + /*
> + * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
> + * window for CXL host bridges so the bridge non-prefetchable window
> + * can hold CXL device BARs (component registers and similar 64-bit
> + * non-prefetchable BARs that need a < 4 GiB address).
> + *
> + * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
> + * 256 MiB so the two ranges do not overlap (the FDT 'ranges'
> + * shrink happens in create_fdt_pcie()).
> + * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
> + * a correct _CRS for the CXL host bridge (ACPI0016).
> + */
> + if (s->cxl_devices_state.is_enabled) {
> + GPEXHost *gpex = GPEX_HOST(pcie_dev);
> + gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
> + gpex->gpex_cfg.cxl_mmio32.base =
> + s->memmap[VIRT_PCIE_MMIO].base +
> + s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
> + /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
> + gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
> + }
>
> create_platform_bus(s, mmio_irqchip);
>
> diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
> index 1da9c85bce..d38fbbacd6 100644
> --- a/include/hw/pci-host/gpex.h
> +++ b/include/hw/pci-host/gpex.h
> @@ -43,6 +43,7 @@ struct GPEXConfig {
> MemMapEntry mmio32;
> MemMapEntry mmio64;
> MemMapEntry pio;
> + MemMapEntry cxl_mmio32;
> int irq;
> PCIBus *bus;
> bool pci_native_hotplug;
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
2026-06-24 17:21 ` Daniel Henrique Barboza via
@ 2026-06-29 9:10 ` Chen Pei
-1 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-06-29 9:10 UTC (permalink / raw)
To: Daniel Henrique Barboza
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518, zhiwei_liu,
chao.liu.zevorn, sunilvl, dave.jiang, alison.schofield, imammedo,
mst, guoren, qemu-riscv, qemu-devel, linux-cxl
Hi Daniel,
On 6/24/2026 2:21 PM, Daniel Henrique Barboza wrote:
> We can get rid of the braces - it's just adding an extra identation level while
> not doing anything special. 'mmio32_size' can be declared at the start
> of create_fdt_pcie().
Makes sense. I'll move mmio32_size to the top of create_fdt_pcie() and
remove the extra braces in v3.
Thanks,
Pei
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-06-29 9:10 ` Chen Pei
0 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-06-29 9:10 UTC (permalink / raw)
To: Daniel Henrique Barboza
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518, zhiwei_liu,
chao.liu.zevorn, sunilvl, dave.jiang, alison.schofield, imammedo,
mst, guoren, qemu-riscv, qemu-devel, linux-cxl
Hi Daniel,
On 6/24/2026 2:21 PM, Daniel Henrique Barboza wrote:
> We can get rid of the braces - it's just adding an extra identation level while
> not doing anything special. 'mmio32_size' can be declared at the start
> of create_fdt_pcie().
Makes sense. I'll move mmio32_size to the top of create_fdt_pcie() and
remove the extra braces in v3.
Thanks,
Pei
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
2026-06-18 9:38 ` [PATCH v2 3/4] hw/riscv/virt, gpex: " Chen Pei
@ 2026-07-22 13:52 ` Junjie Cao
-1 siblings, 0 replies; 24+ messages in thread
From: Junjie Cao @ 2026-07-22 13:52 UTC (permalink / raw)
To: Chen Pei
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren, qemu-riscv, qemu-devel,
linux-cxl
Hi Chen Pei,
On Thu, 18 Jun 2026 17:38:25 +0800, Chen Pei wrote:
> + if (is_cxl && cfg->cxl_mmio32.size) {
> + uint64_t cxl_base = cfg->cxl_mmio32.base;
> + uint64_t cxl_size = cfg->cxl_mmio32.size;
> + crs = aml_resource_template();
> + /* 32-bit MMIO range for CXL devices */
> + aml_append(crs,
> + aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> + AML_MAX_FIXED, AML_NON_CACHEABLE,
> + AML_READ_WRITE, 0,
> + cxl_base, cxl_base + cxl_size - 1,
> + 0, cxl_size));
cfg->cxl_mmio32 is a single range on the main GPEX host, but this block
runs once per CXL host bridge in the QLIST_FOREACH(bus, &bus->child)
loop, so with more than one pxb-cxl every ACPI0016 gets a _CRS that
declares the *same* [cxl_base, cxl_base + 256 MiB) window.
I realise build_crs() returns nothing usable for these bridges at
ACPI-build time (the BARs aren't assigned yet, per your commit message),
which is why the static path exists -- but it hands that one shared
cfg->cxl_mmio32 to every CXL bridge.
I checked the generated DSDT for a two-pxb-cxl riscv64 config (cxl.1 at
bus_nr=12, cxl.2 at bus_nr=200): both ACPI0016 host bridges (PC0C and
PCC8) come up with the identical producer window
DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, ...)
0x70000000, // Range Minimum
0x7FFFFFFF, // Range Maximum
0x10000000, // Length
while PCI0 is correctly shrunk to 0x40000000-0x6FFFFFFF. That 256 MiB
window is the only memory range either ACPI0016 advertises, so every BAR
behind the two bridges has to come from it; and since two host bridges
can't both decode the same range, at most one of them can actually
claim it. Multiple pxb-cxl is a supported topology in QEMU generally.
Partitioning the window per bridge may be one way out, but whether
that's the right model is your call -- is supporting more than one
pxb-cxl in scope for this series, or is a single CXL host bridge the
intended target for now? If the latter, a note to that effect in the
commit message would help.
One smaller thing about the hand-built template: the bus range is
hard-coded to [bus_num, bus_num + 15], so with cxl.1 at bus_nr=12 and
cxl.2 at bus_nr=16 the two _CRS come out as [0x0C,0x1B] and [0x10,0x1F],
which overlap on buses 0x10-0x1B. The real width depends on how many
buses the pxb-cxl spans, which isn't known at ACPI-build time (pxb-cxl
only carries a bus_nr). build_crs() is no drop-in here either: with the
static path disabled I saw the range collapse to a single bus
[bus_num, bus_num], since no subordinate buses are enumerated yet -- so
it under-sizes rather than over-sizes.
Both really trace back to the same spot: the static branch synthesises
the bridge's window and bus range from machine-global constants
(cfg->cxl_mmio32 and the fixed +15) rather than anything per-bridge, so
neither scales past one pxb-cxl. Going static is fair enough given
build_crs() has nothing to offer at build time -- as above, it supplies
neither the window nor a usable bus range -- so the real question is
just where the per-bridge window size and bus span should come from.
Many thanks,
Junjie
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-07-22 13:52 ` Junjie Cao
0 siblings, 0 replies; 24+ messages in thread
From: Junjie Cao @ 2026-07-22 13:52 UTC (permalink / raw)
To: Chen Pei
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren, qemu-riscv, qemu-devel,
linux-cxl
Hi Chen Pei,
On Thu, 18 Jun 2026 17:38:25 +0800, Chen Pei wrote:
> + if (is_cxl && cfg->cxl_mmio32.size) {
> + uint64_t cxl_base = cfg->cxl_mmio32.base;
> + uint64_t cxl_size = cfg->cxl_mmio32.size;
> + crs = aml_resource_template();
> + /* 32-bit MMIO range for CXL devices */
> + aml_append(crs,
> + aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> + AML_MAX_FIXED, AML_NON_CACHEABLE,
> + AML_READ_WRITE, 0,
> + cxl_base, cxl_base + cxl_size - 1,
> + 0, cxl_size));
cfg->cxl_mmio32 is a single range on the main GPEX host, but this block
runs once per CXL host bridge in the QLIST_FOREACH(bus, &bus->child)
loop, so with more than one pxb-cxl every ACPI0016 gets a _CRS that
declares the *same* [cxl_base, cxl_base + 256 MiB) window.
I realise build_crs() returns nothing usable for these bridges at
ACPI-build time (the BARs aren't assigned yet, per your commit message),
which is why the static path exists -- but it hands that one shared
cfg->cxl_mmio32 to every CXL bridge.
I checked the generated DSDT for a two-pxb-cxl riscv64 config (cxl.1 at
bus_nr=12, cxl.2 at bus_nr=200): both ACPI0016 host bridges (PC0C and
PCC8) come up with the identical producer window
DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, ...)
0x70000000, // Range Minimum
0x7FFFFFFF, // Range Maximum
0x10000000, // Length
while PCI0 is correctly shrunk to 0x40000000-0x6FFFFFFF. That 256 MiB
window is the only memory range either ACPI0016 advertises, so every BAR
behind the two bridges has to come from it; and since two host bridges
can't both decode the same range, at most one of them can actually
claim it. Multiple pxb-cxl is a supported topology in QEMU generally.
Partitioning the window per bridge may be one way out, but whether
that's the right model is your call -- is supporting more than one
pxb-cxl in scope for this series, or is a single CXL host bridge the
intended target for now? If the latter, a note to that effect in the
commit message would help.
One smaller thing about the hand-built template: the bus range is
hard-coded to [bus_num, bus_num + 15], so with cxl.1 at bus_nr=12 and
cxl.2 at bus_nr=16 the two _CRS come out as [0x0C,0x1B] and [0x10,0x1F],
which overlap on buses 0x10-0x1B. The real width depends on how many
buses the pxb-cxl spans, which isn't known at ACPI-build time (pxb-cxl
only carries a bus_nr). build_crs() is no drop-in here either: with the
static path disabled I saw the range collapse to a single bus
[bus_num, bus_num], since no subordinate buses are enumerated yet -- so
it under-sizes rather than over-sizes.
Both really trace back to the same spot: the static branch synthesises
the bridge's window and bus range from machine-global constants
(cfg->cxl_mmio32 and the fixed +15) rather than anything per-bridge, so
neither scales past one pxb-cxl. Going static is fair enough given
build_crs() has nothing to offer at build time -- as above, it supplies
neither the window nor a usable bus range -- so the real question is
just where the per-bridge window size and bus span should come from.
Many thanks,
Junjie
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
2026-07-22 13:52 ` [PATCH v2 3/4] hw/riscv/virt,gpex: " Junjie Cao
@ 2026-07-24 12:10 ` Chen Pei
-1 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-07-24 12:10 UTC (permalink / raw)
To: Junjie Cao
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren, qemu-riscv, qemu-devel,
linux-cxl
Hi Junjie,
Thanks for the thorough review.
> cfg->cxl_mmio32 is a single range on the main GPEX host, but this block
> runs once per CXL host bridge in the QLIST_FOREACH(bus, &bus->child)
> loop, so with more than one pxb-cxl every ACPI0016 gets a _CRS that
> declares the *same* [cxl_base, cxl_base + 256 MiB) window.
You're right. The static path synthesises the bridge window from the
machine-global cfg->cxl_mmio32 rather than per-bridge state, so it
does not scale past one pxb-cxl.
> is supporting more than one pxb-cxl in scope for this series, or is a
> single CXL host bridge the intended target for now? If the latter, a
> note to that effect in the commit message would help.
For this series the intended target is a single CXL host bridge,
which matches the bios-tables test case (one pxb-cxl at bus_nr=12).
I'll add a note to the commit message in v3.
> One smaller thing about the hand-built template: the bus range is
> hard-coded to [bus_num, bus_num + 15], so with cxl.1 at bus_nr=12 and
> cxl.2 at bus_nr=16 the two _CRS come out as [0x0C,0x1B] and [0x10,0x1F],
> which overlap on buses 0x10-0x1B.
This is a related symptom of the same root cause: pxb-cxl only carries
a bus_nr and the actual subordinate bus count is not known at
ACPI-build time, so neither build_crs() nor a static constant can
produce a correct span for the multi-bridge case. The +15 heuristic
is borrowed from the ARM virt machine's gpex handling where a single
pxb is the norm.
> Both really trace back to the same spot: the static branch synthesises
> the bridge's window and bus range from machine-global constants
> (cfg->cxl_mmio32 and the fixed +15) rather than anything per-bridge, so
> neither scales past one pxb-cxl.
Agreed. I'll add a TODO comment at the static branch in gpex-acpi.c
to flag this for future work, and document the single-bridge limitation
in the commit message as noted above.
Thanks,
Pei
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-07-24 12:10 ` Chen Pei
0 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-07-24 12:10 UTC (permalink / raw)
To: Junjie Cao
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, imammedo, mst, guoren, qemu-riscv, qemu-devel,
linux-cxl
Hi Junjie,
Thanks for the thorough review.
> cfg->cxl_mmio32 is a single range on the main GPEX host, but this block
> runs once per CXL host bridge in the QLIST_FOREACH(bus, &bus->child)
> loop, so with more than one pxb-cxl every ACPI0016 gets a _CRS that
> declares the *same* [cxl_base, cxl_base + 256 MiB) window.
You're right. The static path synthesises the bridge window from the
machine-global cfg->cxl_mmio32 rather than per-bridge state, so it
does not scale past one pxb-cxl.
> is supporting more than one pxb-cxl in scope for this series, or is a
> single CXL host bridge the intended target for now? If the latter, a
> note to that effect in the commit message would help.
For this series the intended target is a single CXL host bridge,
which matches the bios-tables test case (one pxb-cxl at bus_nr=12).
I'll add a note to the commit message in v3.
> One smaller thing about the hand-built template: the bus range is
> hard-coded to [bus_num, bus_num + 15], so with cxl.1 at bus_nr=12 and
> cxl.2 at bus_nr=16 the two _CRS come out as [0x0C,0x1B] and [0x10,0x1F],
> which overlap on buses 0x10-0x1B.
This is a related symptom of the same root cause: pxb-cxl only carries
a bus_nr and the actual subordinate bus count is not known at
ACPI-build time, so neither build_crs() nor a static constant can
produce a correct span for the multi-bridge case. The +15 heuristic
is borrowed from the ARM virt machine's gpex handling where a single
pxb is the norm.
> Both really trace back to the same spot: the static branch synthesises
> the bridge's window and bus range from machine-global constants
> (cfg->cxl_mmio32 and the fixed +15) rather than anything per-bridge, so
> neither scales past one pxb-cxl.
Agreed. I'll add a TODO comment at the static branch in gpex-acpi.c
to flag this for future work, and document the single-bridge limitation
in the commit message as noted above.
Thanks,
Pei
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2 3/4] hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
2026-06-18 9:38 ` [PATCH v2 3/4] hw/riscv/virt, gpex: " Chen Pei
` (2 preceding siblings ...)
(?)
@ 2026-07-27 9:06 ` Igor Mammedov
2026-07-29 12:36 ` [PATCH v2 3/4] hw/riscv/virt, gpex: " Chen Pei
-1 siblings, 1 reply; 24+ messages in thread
From: Igor Mammedov @ 2026-07-27 9:06 UTC (permalink / raw)
To: Chen Pei
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, mst, guoren, qemu-riscv, qemu-devel, linux-cxl
On Thu, 18 Jun 2026 17:38:25 +0800
Chen Pei <cp0613@linux.alibaba.com> wrote:
> CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
> and the CXL device register BAR (BAR2 on Type3 device) are declared
> as 64-bit non-prefetchable memory. Per the PCI-to-PCI Bridge
> Architecture Specification Rev 1.2 (PCI-SIG, 2003):
>
> - §3.2.5.8 (Memory Base/Limit): the non-prefetchable window covers
> only 32-bit addresses (AD[31:20]); the Type 1 header defines no
> upper-32-bit extension for it.
> - §3.2.5.9 (Prefetchable Memory Base/Limit): the bottom 4 bits
> encode 64-bit support (01h), but this applies exclusively to the
> *prefetchable* window.
> - §3.2.5.10 (Prefetchable Base/Limit Upper 32 Bits): optional
> registers for AD[63:32] of the prefetchable range only.
>
> The architecture therefore allows a 64-bit window only when it is also
> prefetchable; there is no 64-bit non-prefetchable form. PCIe inherits
> this Type 1 header layout unchanged. Linux thus places 64-bit
> non-prefetchable BARs in the 32-bit non-prefetchable bridge window,
> which requires the bridge to own enough address space below 4 GiB.
>
> On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
> currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
> have no non-prefetchable window and Linux fails to assign these BARs.
> Marking the BARs prefetchable would work around it, but the CXL
> component registers have read/write side effects and are not
> prefetchable per the PCIe specification.
>
> Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
> CXL host bridges:
> - Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
> PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
> - Store the reserved range in a new gpex_cfg.cxl_mmio32 field
> - In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
> in the CXL host bridge _CRS instead of re-using build_crs() (which
> returns an empty set when UEFI has not assigned resources yet)
> - Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
> firmware driven by device-tree also respects the reservation
>
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> ---
> hw/pci-host/gpex-acpi.c | 36 +++++++++++++++++++++--
> hw/riscv/virt.c | 58 +++++++++++++++++++++++++++++++-------
> include/hw/pci-host/gpex.h | 1 +
> 3 files changed, 83 insertions(+), 12 deletions(-)
>
> diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
> index d9820f9b41..d8b943b665 100644
> --- a/hw/pci-host/gpex-acpi.c
> +++ b/hw/pci-host/gpex-acpi.c
> @@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
> * Resources defined for PXBs are composed of the following parts:
> * 1. The resources the pci-bridge/pcie-root-port need.
> * 2. The resources the devices behind pxb need.
> + *
> + * For CXL host bridges on platforms where UEFI (driven by
> + * FDT 'ranges') does not assign PCI resources for the CXL
> + * root bridge before ACPI table construction, build_crs()
I kind of dislike (basically riscv specific hack) in generic code.
well, correct way to tackle this is
1: fix UEFI to perform required initialization.
> + * would return an empty resource set. When the platform
> + * has reserved a dedicated MMIO window for CXL host bridges
> + * (cfg->cxl_mmio32), emit that window as a static _CRS
> + * instead. The platform is responsible for shrinking PCI0's
> + * mmio32 window so the two do not overlap.
if you have to put a hack into QEMU, make it work with current build_crs()
and localize 'proper simulated initialization' in riscv codebase
(i.e. do not spread it to common codebase). But before doing it see (#1 note).
> */
> - crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
> - cfg->pio.base, 0, 0, 0);
> + if (is_cxl && cfg->cxl_mmio32.size) {
> + uint64_t cxl_base = cfg->cxl_mmio32.base;
> + uint64_t cxl_size = cfg->cxl_mmio32.size;
> +
> + crs = aml_resource_template();
> +
> + /* 32-bit MMIO range for CXL devices */
> + aml_append(crs,
> + aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> + AML_MAX_FIXED, AML_NON_CACHEABLE,
> + AML_READ_WRITE, 0,
> + cxl_base,
> + cxl_base + cxl_size - 1,
> + 0, cxl_size));
> +
> + /* Bus number range */
> + aml_append(crs,
> + aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
> + AML_POS_DECODE, 0,
> + bus_num, bus_num + 15,
> + 0, 16));
> + } else {
> + crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
> + &crs_range_set, cfg->pio.base, 0, 0, 0);
> + }
> aml_append(dev, aml_name_decl("_CRS", crs));
>
> if (is_cxl) {
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index 84b91b4322..9c1a001553 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
> /* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
> #define VIRT64_HIGH_PCIE_MMIO_SIZE (16 * GiB)
>
> +/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
> +#define VIRT_CXL_MMIO32_SIZE (256 * MiB)
> +
> static MemMapEntry virt_high_pcie_memmap;
>
> #define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
> @@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
> }
> qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
> s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
> - qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> - 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> - 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> - 1, FDT_PCI_RANGE_MMIO,
> - 2, s->memmap[VIRT_PCIE_MMIO].base,
> - 2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
> - 1, FDT_PCI_RANGE_MMIO_64BIT,
> - 2, virt_high_pcie_memmap.base,
> - 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + {
> + /*
> + * When CXL is enabled, reserve the last 256 MiB of the 32-bit
> + * MMIO window for CXL host bridges and exclude it from the main
> + * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
> + * does not allocate that range to PCI0. The CXL host bridge
> + * _CRS declares this range independently.
> + */
> + hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
> + if (s->cxl_devices_state.is_enabled) {
> + mmio32_size -= VIRT_CXL_MMIO32_SIZE;
> + }
> + qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> + 1, FDT_PCI_RANGE_IOPORT, 2, 0,
> + 2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> + 1, FDT_PCI_RANGE_MMIO,
> + 2, s->memmap[VIRT_PCIE_MMIO].base,
> + 2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
> + 1, FDT_PCI_RANGE_MMIO_64BIT,
> + 2, virt_high_pcie_memmap.base,
> + 2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> + }
>
> if (virt_is_iommu_sys_enabled(s)) {
> qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
> @@ -1728,7 +1744,29 @@ static void virt_machine_init(MachineState *machine)
> qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
> }
>
> - gpex_pcie_init(system_memory, pcie_irqchip, s);
> + DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
> +
> + /*
> + * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
> + * window for CXL host bridges so the bridge non-prefetchable window
> + * can hold CXL device BARs (component registers and similar 64-bit
> + * non-prefetchable BARs that need a < 4 GiB address).
> + *
> + * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
> + * 256 MiB so the two ranges do not overlap (the FDT 'ranges'
> + * shrink happens in create_fdt_pcie()).
> + * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
> + * a correct _CRS for the CXL host bridge (ACPI0016).
> + */
> + if (s->cxl_devices_state.is_enabled) {
> + GPEXHost *gpex = GPEX_HOST(pcie_dev);
> + gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
> + gpex->gpex_cfg.cxl_mmio32.base =
> + s->memmap[VIRT_PCIE_MMIO].base +
> + s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
> + /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
> + gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
> + }
>
> create_platform_bus(s, mmio_irqchip);
>
> diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
> index 1da9c85bce..d38fbbacd6 100644
> --- a/include/hw/pci-host/gpex.h
> +++ b/include/hw/pci-host/gpex.h
> @@ -43,6 +43,7 @@ struct GPEXConfig {
> MemMapEntry mmio32;
> MemMapEntry mmio64;
> MemMapEntry pio;
> + MemMapEntry cxl_mmio32;
> int irq;
> PCIBus *bus;
> bool pci_native_hotplug;
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
2026-07-27 9:06 ` [PATCH v2 3/4] hw/riscv/virt,gpex: " Igor Mammedov
@ 2026-07-29 12:36 ` Chen Pei
0 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-07-29 12:36 UTC (permalink / raw)
To: Igor Mammedov
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, mst, guoren, qemu-riscv, qemu-devel, linux-cxl
Hi Igor,
Thanks for the review.
On Mon, 27 Jul 2026 11:06:15 +0200
Igor Mammedov <imammedo@redhat.com> wrote:
> I kind of dislike (basically riscv specific hack) in generic code.
> well, correct way to tackle this is
> 1: fix UEFI to perform required initialization.
I agree this is not a clean solution, and I acknowledge the concern
about putting riscv-specific behavior into the shared gpex code.
I did try the "fix UEFI" direction first. With the current EDK2
(edk2-stable202602, the same FdtPciHostBridgeLib/PciBusDxe used by
both aarch64 and riscv virt), I booted a riscv virt machine with
pxb-cxl and dumped the ECAM config space after the firmware finished
enumeration:
- pxb-cxl (1b36:000b) on bus 0: class 0x0600 (host bridge),
header type 0, memory/prefetchable window registers all zero;
- cxl-rp on bus 12: memory window base > limit (empty, reset value);
- cxl-type3 on bus 13: entire config space reads 0xffffffff,
i.e. the firmware never enumerated behind the CXL host bridge.
The root cause is that PciBusDxe only recurses into devices for which
IS_PCI_BRIDGE() is true (header layout 1), while pxb-cxl presents
header type 0 and class 0x0600, so the firmware neither assigns it a
secondary bus number nor any MMIO window. FdtPciHostBridgeLib likewise
only consumes the single "pci-host-ecam-generic" node describing PCI0
and has no knowledge of the expander bridge. As a result build_crs()
genuinely returns an empty set on riscv today - this is not a timing
artifact.
> if you have to put a hack into QEMU, make it work with current build_crs()
> and localize 'proper simulated initialization' in riscv codebase
> (i.e. do not spread it to common codebase). But before doing it see (#1 not=
e).
Understood. Since #1 (a UEFI that performs the initialization) is not
available on riscv at the moment, the static window is currently the
only path that makes CXL host bridge _CRS usable here.
For what it is worth, this is consistent with how CXL is being handled
on the ARM side. The sbsa-ref enablement (Yuquan Wang, RFC up to v7,
"hw/arm/sbsa-ref: Support CXL Host Bridge & CFMW" [1], plus the
matching edk2-platforms RFC [2]) deliberately carves out exclusive
static MMIO32/MMIO64 and bus-number ranges for the CXL host bridge and
builds static ACPI tables, explicitly noting that it "doesn't need to
communicate CXL contents via DT to edk2". Jonathan Cameron's arm/virt
approach instead relies on EDK2 performing the enumeration and QEMU
reading the result back via build_crs(), but that depends on a recent
EDK2 with expander-bridge support that we do not have on riscv yet;
his earlier DT-binding RFC for expander bridges [3] also notes that on
the DT path QEMU has to allocate the windows itself.
[1] https://lore.kernel.org/all/20250807111037.241118-1-wangyuquan1236@phytium.com.cn/
[2] https://op-lists.linaro.org/archives/list/asa-dev@op-lists.linaro.org/thread/EPPN4R6KLGKYLWPOZUBWJG5HCXAAO3SQ/
[3] https://lore.kernel.org/all/20230421165037.2506-1-Jonathan.Cameron@huawei.com/
Given that, my tentative plan for v3 is to keep the static-window
approach for now but address your objection by localizing it: drop the
cxl_mmio32 field and the is_cxl branch from the common gpex code, and
instead simulate the initialization in riscv virt code so that the
existing build_crs() path produces the correct _CRS (i.e. have riscv
pre-populate the bridge window registers / range set the way a
firmware would), keeping the common codebase untouched.
Before I rework it that way, I would appreciate any suggestions you
may have on this approach.
Thanks,
Pei
^ permalink raw reply [flat|nested] 24+ messages in thread* Re: [PATCH v2 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges
@ 2026-07-29 12:36 ` Chen Pei
0 siblings, 0 replies; 24+ messages in thread
From: Chen Pei @ 2026-07-29 12:36 UTC (permalink / raw)
To: Igor Mammedov
Cc: jic23, pbonzini, palmer, alistair.francis, liwei1518,
daniel.barboza, zhiwei_liu, chao.liu.zevorn, sunilvl, dave.jiang,
alison.schofield, mst, guoren, qemu-riscv, qemu-devel, linux-cxl
Hi Igor,
Thanks for the review.
On Mon, 27 Jul 2026 11:06:15 +0200
Igor Mammedov <imammedo@redhat.com> wrote:
> I kind of dislike (basically riscv specific hack) in generic code.
> well, correct way to tackle this is
> 1: fix UEFI to perform required initialization.
I agree this is not a clean solution, and I acknowledge the concern
about putting riscv-specific behavior into the shared gpex code.
I did try the "fix UEFI" direction first. With the current EDK2
(edk2-stable202602, the same FdtPciHostBridgeLib/PciBusDxe used by
both aarch64 and riscv virt), I booted a riscv virt machine with
pxb-cxl and dumped the ECAM config space after the firmware finished
enumeration:
- pxb-cxl (1b36:000b) on bus 0: class 0x0600 (host bridge),
header type 0, memory/prefetchable window registers all zero;
- cxl-rp on bus 12: memory window base > limit (empty, reset value);
- cxl-type3 on bus 13: entire config space reads 0xffffffff,
i.e. the firmware never enumerated behind the CXL host bridge.
The root cause is that PciBusDxe only recurses into devices for which
IS_PCI_BRIDGE() is true (header layout 1), while pxb-cxl presents
header type 0 and class 0x0600, so the firmware neither assigns it a
secondary bus number nor any MMIO window. FdtPciHostBridgeLib likewise
only consumes the single "pci-host-ecam-generic" node describing PCI0
and has no knowledge of the expander bridge. As a result build_crs()
genuinely returns an empty set on riscv today - this is not a timing
artifact.
> if you have to put a hack into QEMU, make it work with current build_crs()
> and localize 'proper simulated initialization' in riscv codebase
> (i.e. do not spread it to common codebase). But before doing it see (#1 not=
e).
Understood. Since #1 (a UEFI that performs the initialization) is not
available on riscv at the moment, the static window is currently the
only path that makes CXL host bridge _CRS usable here.
For what it is worth, this is consistent with how CXL is being handled
on the ARM side. The sbsa-ref enablement (Yuquan Wang, RFC up to v7,
"hw/arm/sbsa-ref: Support CXL Host Bridge & CFMW" [1], plus the
matching edk2-platforms RFC [2]) deliberately carves out exclusive
static MMIO32/MMIO64 and bus-number ranges for the CXL host bridge and
builds static ACPI tables, explicitly noting that it "doesn't need to
communicate CXL contents via DT to edk2". Jonathan Cameron's arm/virt
approach instead relies on EDK2 performing the enumeration and QEMU
reading the result back via build_crs(), but that depends on a recent
EDK2 with expander-bridge support that we do not have on riscv yet;
his earlier DT-binding RFC for expander bridges [3] also notes that on
the DT path QEMU has to allocate the windows itself.
[1] https://lore.kernel.org/all/20250807111037.241118-1-wangyuquan1236@phytium.com.cn/
[2] https://op-lists.linaro.org/archives/list/asa-dev@op-lists.linaro.org/thread/EPPN4R6KLGKYLWPOZUBWJG5HCXAAO3SQ/
[3] https://lore.kernel.org/all/20230421165037.2506-1-Jonathan.Cameron@huawei.com/
Given that, my tentative plan for v3 is to keep the static-window
approach for now but address your objection by localizing it: drop the
cxl_mmio32 field and the is_cxl branch from the common gpex code, and
instead simulate the initialization in riscv virt code so that the
existing build_crs() path produces the correct _CRS (i.e. have riscv
pre-populate the bridge window registers / range set the way a
firmware would), keeping the common codebase untouched.
Before I rework it that way, I would appreciate any suggestions you
may have on this approach.
Thanks,
Pei
^ permalink raw reply [flat|nested] 24+ messages in thread