[PATCH 0/4] hw/riscv/virt: Add CXL support and fix virtio DMA into CXL memory

[PATCH 0/4] hw/riscv/virt: Add CXL support and fix virtio DMA into CXL memory

[Chen Pei]

This series enables CXL (Compute Express Link) on the RISC-V 'virt'
machine (patch 1, mirroring hw/arm/virt), fixes two RISC-V-specific
gaps that block Type-3 enumeration today (patches 2 and 3), and
fixes one arch-agnostic CXL bug that surfaces as soon as a Type-3
device is brought online as system RAM (patch 4).

  1/4 Wires up CXLState, the CXL host register region, the Fixed
      Memory Windows and the ACPI0017 / CEDT plumbing on the
      RISC-V virt machine, mirroring hw/arm/virt.

  2/4 Adds an ACPI _DEP from ACPI0017 to the ACPI0016 CXL host
      bridges so cxl_acpi probes after acpi_pci_root has attached
      the bridges.  Without this the CXL port topology comes up
      incomplete on RISC-V because probe ordering is not guaranteed.

      Kernel dependency: this patch relies on a small kernel-side
      change that calls acpi_dev_clear_dependencies() in
      acpi_pci_root_add().  That kernel patch has completed review
      on linux-pci:
      https://lore.kernel.org/linux-pci/20260526025118.38935-1-cp0613@linux.alibaba.com/

  3/4 Reserves the top 256 MiB of the RISC-V virt 32-bit PCIe MMIO
      window exclusively for CXL host bridges and emits it via the
      ACPI0016 _CRS, so Linux can place the 64-bit non-prefetchable
      CXL component/device register BARs below 4 GiB.  PCI0's
      mmio32 window and FDT 'ranges' are shrunk by the same amount.

  4/4 Overlays a RAM alias of the Type-3 device's memory backend at
      the committed HDM decoder's HPA range, so that
      address_space_map() on CFMW addresses returns a direct host
      pointer instead of taking the 4 KiB bounce-buffer path.
      Without it, once 'daxctl online-memory' exposes CXL memory to
      the kernel allocator, any virtio scatter list larger than
      4 KiB into that range dies with:

          virtio: bogus descriptor or out of resources

      This bug is not RISC-V specific -- CFMW registration and the
      bounce-buffer limit are both arch-agnostic -- but the RISC-V
      virt machine is where the rest of this series first lights up
      the daxctl + virtio path end-to-end.

      Scope: the fix covers the common single-device, single-decoder
      configuration.  interleave_ways > 1 and HDM decoders inside a
      CXL switch keep falling through to the existing cfmws_ops +
      bounce-buffer path, exactly as today (no regression; left as
      follow-up work).  An alternative implementation that was
      considered is teaching the CFMW MemoryRegion itself to expose
      a ram_block; the overlay-on-commit approach was preferred as
      it mirrors the existing PCI BAR / IOMMU MR overlay pattern
      and is local to cxl_type3.

Tested on the RISC-V virt machine with EDK2 firmware and a buildroot
guest carrying the kernel patches above.

QEMU invocation (CXL-relevant options shown; EDK2 pflash, -bios,
-kernel, -append and the virtio-blk rootfs are as usual):

    qemu-system-riscv64 \
        -M virt,aia=aplic-imsic,acpi=on,cxl=on \
        -cpu rv64 -smp 2 -m 4G,maxmem=8G,slots=8 \
        -object memory-backend-ram,id=vmem0,share=on,size=4G \
        -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
        -device cxl-rp,port=0,bus=cxl.1,id=rp0,chassis=0,slot=2 \
        -device cxl-type3,bus=rp0,volatile-memdev=vmem0,id=cxl-vmem0 \
        -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G \
        ...   # EDK2 pflash + -bios fw_dynamic.bin + -kernel Image
              # + virtio-blk rootfs + -append "root=/dev/vda ..."

Verification (compare the two `free -h` outputs: total system memory
grows by 4 GiB after onlining):

    # free -h
    # cxl list
    # cxl enable-memdev mem0
    # cxl create-region -m -t ram -d decoder0.0 -w 1 mem0 -s 4G
    # daxctl online-memory dax0.0
    # free -h

Chen Pei (4):
  hw/riscv/virt: Add CXL support to the RISC-V virt machine
  hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge
    dependency
  hw/riscv/virt,gpex: Provide 32-bit MMIO window for CXL host bridges
  hw/cxl: Map committed HDM decoder ranges as RAM for direct DMA

 hw/mem/cxl_type3.c          |  81 +++++++++++++++++++++++++++++++++++
 hw/pci-host/gpex-acpi.c     |  36 +++++++++++++++-
 hw/riscv/Kconfig            |   2 +
 hw/riscv/virt-acpi-build.c  |  52 +++++++++++++++++++++++
 hw/riscv/virt.c             | 100 +++++++++++++++++++++++++++++++++++++++-----
 include/hw/cxl/cxl_device.h |   4 ++
 include/hw/pci-host/gpex.h  |   1 +
 include/hw/riscv/virt.h     |   3 ++
 8 files changed, 267 insertions(+), 12 deletions(-)

--
2.50.1

[PATCH 1/4] hw/riscv/virt: Add CXL support to the RISC-V virt machine

[Chen Pei]

Enable CXL support on the RISC-V virt machine following the same
approach used by the ARM virt machine:
- Add PXB and ACPI_CXL Kconfig selections
- Add CXLState and PCIBus pointer to RISCVVirtState
- Register CXL machine properties via cxl_machine_init()
- Create CXL host register region above the PCIe high MMIO region
- Call cxl_hook_up_pxb_registers() and cxl_fmws_link_targets() at
  machine_done time
- Map Fixed Memory Windows above the CXL host register region
- Add ACPI0017 device in DSDT and build CEDT table in virt-acpi-build.c

Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
 hw/riscv/Kconfig           |  2 ++
 hw/riscv/virt-acpi-build.c | 20 ++++++++++++++++++
 hw/riscv/virt.c            | 42 ++++++++++++++++++++++++++++++++++++++
 include/hw/riscv/virt.h    |  3 +++
 4 files changed, 67 insertions(+)

diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig
index 2518b04175..ebd0355f09 100644
--- a/hw/riscv/Kconfig
+++ b/hw/riscv/Kconfig
@@ -66,8 +66,10 @@ config RISCV_VIRT
     select VIRTIO_MMIO
     select FW_CFG_DMA
     select PLATFORM_BUS
+    select PXB
     select ACPI
     select ACPI_PCI
+    select ACPI_CXL
 
 config SHAKTI_C
     bool
diff --git a/hw/riscv/virt-acpi-build.c b/hw/riscv/virt-acpi-build.c
index 413d47d70e..309d64b322 100644
--- a/hw/riscv/virt-acpi-build.c
+++ b/hw/riscv/virt-acpi-build.c
@@ -29,12 +29,16 @@
 #include "hw/acpi/aml-build.h"
 #include "hw/acpi/pci.h"
 #include "hw/acpi/utils.h"
+#include "hw/acpi/cxl.h"
 #include "hw/intc/riscv_aclint.h"
 #include "hw/nvram/fw_cfg_acpi.h"
 #include "hw/pci-host/gpex.h"
+#include "hw/pci/pci_bus.h"
 #include "hw/riscv/virt.h"
 #include "hw/riscv/numa.h"
 #include "hw/virtio/virtio-acpi.h"
+#include "hw/cxl/cxl.h"
+#include "hw/cxl/cxl_host.h"
 #include "kvm/kvm_riscv.h"
 #include "migration/vmstate.h"
 #include "qapi/error.h"
@@ -503,6 +507,17 @@ static void build_dsdt(GArray *table_data,
         acpi_dsdt_add_gpex_host(scope, PCIE_IRQ + VIRT_IRQCHIP_NUM_SOURCES * 2);
     }
 
+    if (s->cxl_devices_state.is_enabled) {
+        Aml *cxl_dev = aml_device("CXLM");
+        aml_append(cxl_dev, aml_name_decl("_HID", aml_string("ACPI0017")));
+        Aml *method = aml_method("_STA", 0, AML_NOTSERIALIZED);
+        aml_append(method, aml_return(aml_int(0x0B)));
+        aml_append(cxl_dev, method);
+        build_cxl_dsm_method(cxl_dev);
+
+        aml_append(scope, cxl_dev);
+    }
+
     aml_append(dsdt, scope);
 
     /* copy AML table into ACPI tables blob and patch header there */
@@ -910,6 +925,11 @@ static void virt_acpi_build(RISCVVirtState *s, AcpiBuildTables *tables)
                    s->oem_table_id);
     }
 
+    if (s->cxl_devices_state.is_enabled) {
+        cxl_build_cedt(table_offsets, tables_blob, tables->linker,
+                       s->oem_id, s->oem_table_id, &s->cxl_devices_state);
+    }
+
     if (ms->numa_state->num_nodes > 0) {
         acpi_add_table(table_offsets, tables_blob);
         build_srat(tables_blob, tables->linker, s);
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index ce64eaaef7..899f632de7 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -55,6 +55,8 @@
 #include "hw/pci/pci.h"
 #include "hw/pci-host/gpex.h"
 #include "hw/display/ramfb.h"
+#include "hw/cxl/cxl.h"
+#include "hw/cxl/cxl_host.h"
 #include "hw/acpi/aml-build.h"
 #include "qapi/qapi-visit-common.h"
 #include "hw/virtio/virtio-iommu.h"
@@ -1259,9 +1261,27 @@ static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem,
     }
 
     GPEX_HOST(dev)->gpex_cfg.bus = PCI_HOST_BRIDGE(dev)->bus;
+    s->bus = PCI_HOST_BRIDGE(dev)->bus;
     return dev;
 }
 
+static void create_cxl_host_reg_region(RISCVVirtState *s)
+{
+    MemoryRegion *sysmem = get_system_memory();
+    MemoryRegion *mr = &s->cxl_devices_state.host_mr;
+    hwaddr base;
+
+    if (!s->cxl_devices_state.is_enabled) {
+        return;
+    }
+
+    base = virt_high_pcie_memmap.base + virt_high_pcie_memmap.size;
+    base = ROUND_UP(base, 64 * KiB);
+
+    memory_region_init(mr, OBJECT(s), "cxl_host_reg", 64 * KiB * 16);
+    memory_region_add_subregion(sysmem, base, mr);
+}
+
 static FWCfgState *create_fw_cfg(const MachineState *ms, hwaddr base)
 {
     FWCfgState *fw_cfg;
@@ -1426,6 +1446,15 @@ static void virt_machine_done(Notifier *notifier, void *data)
                                      machine_done);
     MachineState *machine = MACHINE(s);
     hwaddr start_addr = s->memmap[VIRT_DRAM].base;
+
+    if (s->bus) {
+        cxl_hook_up_pxb_registers(s->bus, &s->cxl_devices_state,
+                                  &error_fatal);
+    }
+
+    if (s->cxl_devices_state.is_enabled) {
+        cxl_fmws_link_targets(&error_fatal);
+    }
     hwaddr firmware_end_addr;
     vaddr kernel_start_addr;
     const char *firmware_name = riscv_default_firmware_name(&s->soc[0]);
@@ -1663,6 +1692,17 @@ static void virt_machine_init(MachineState *machine)
             ROUND_UP(virt_high_pcie_memmap.base, virt_high_pcie_memmap.size);
     }
 
+    create_cxl_host_reg_region(s);
+
+    if (s->cxl_devices_state.is_enabled) {
+        hwaddr cxl_base = virt_high_pcie_memmap.base +
+                          virt_high_pcie_memmap.size;
+        cxl_base += memory_region_size(&s->cxl_devices_state.host_mr);
+        cxl_base = ROUND_UP(cxl_base, 256 * MiB);
+        cxl_fmws_set_memmap(cxl_base, UINT64_MAX);
+        cxl_fmws_update_mmio();
+    }
+
     /* register system main memory (actual RAM) */
     memory_region_add_subregion(system_memory, s->memmap[VIRT_DRAM].base,
                                 machine->ram);
@@ -1769,6 +1809,8 @@ static void virt_machine_instance_init(Object *obj)
     s->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
     s->acpi = ON_OFF_AUTO_AUTO;
     s->iommu_sys = ON_OFF_AUTO_AUTO;
+
+    cxl_machine_init(obj, &s->cxl_devices_state);
 }
 
 static char *virt_get_aia_guests(Object *obj, Error **errp)
diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h
index 18a2a323a3..c3201588bb 100644
--- a/include/hw/riscv/virt.h
+++ b/include/hw/riscv/virt.h
@@ -24,6 +24,7 @@
 #include "hw/core/sysbus.h"
 #include "hw/block/flash.h"
 #include "hw/intc/riscv_imsic.h"
+#include "hw/cxl/cxl.h"
 
 #define VIRT_CPUS_MAX_BITS             9
 #define VIRT_CPUS_MAX                  (1 << VIRT_CPUS_MAX_BITS)
@@ -64,6 +65,8 @@ struct RISCVVirtState {
     struct GPEXHost *gpex_host;
     OnOffAuto iommu_sys;
     uint16_t pci_iommu_bdf;
+    CXLState cxl_devices_state;
+    PCIBus *bus;
 };
 
 enum {
-- 
2.50.1

Re: [PATCH 1/4] hw/riscv/virt: Add CXL support to the RISC-V virt machine

[Jonathan Cameron]

On Tue,  2 Jun 2026 15:41:24 +0800
Chen Pei <cp0613@linux.alibaba.com> wrote:

> Enable CXL support on the RISC-V virt machine following the same
> approach used by the ARM virt machine:
> - Add PXB and ACPI_CXL Kconfig selections
> - Add CXLState and PCIBus pointer to RISCVVirtState
> - Register CXL machine properties via cxl_machine_init()
> - Create CXL host register region above the PCIe high MMIO region
> - Call cxl_hook_up_pxb_registers() and cxl_fmws_link_targets() at
>   machine_done time
> - Map Fixed Memory Windows above the CXL host register region
> - Add ACPI0017 device in DSDT and build CEDT table in virt-acpi-build.c
> 
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
Just really minor stuff inline. Been a while since we added an architecture
but all looks fine to me.

Reviewed-by: Jonathan Cameron <jic23@kernel.org>

> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index ce64eaaef7..899f632de7 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -55,6 +55,8 @@
>  #include "hw/pci/pci.h"
>  #include "hw/pci-host/gpex.h"
>  #include "hw/display/ramfb.h"
> +#include "hw/cxl/cxl.h"
> +#include "hw/cxl/cxl_host.h"
>  #include "hw/acpi/aml-build.h"
>  #include "qapi/qapi-visit-common.h"
>  #include "hw/virtio/virtio-iommu.h"
> @@ -1259,9 +1261,27 @@ static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem,
>      }
>  
>      GPEX_HOST(dev)->gpex_cfg.bus = PCI_HOST_BRIDGE(dev)->bus;
> +    s->bus = PCI_HOST_BRIDGE(dev)->bus;
>      return dev;
>  }

>  static FWCfgState *create_fw_cfg(const MachineState *ms, hwaddr base)
>  {
>      FWCfgState *fw_cfg;
> @@ -1426,6 +1446,15 @@ static void virt_machine_done(Notifier *notifier, void *data)
>                                       machine_done);
>      MachineState *machine = MACHINE(s);
>      hwaddr start_addr = s->memmap[VIRT_DRAM].base;
> +
> +    if (s->bus) {

There is a guard against !s->bus inside the function so you should be
able to do this unconditionally.

> +        cxl_hook_up_pxb_registers(s->bus, &s->cxl_devices_state,
> +                                  &error_fatal);
> +    }
> +
> +    if (s->cxl_devices_state.is_enabled) {
> +        cxl_fmws_link_targets(&error_fatal);
> +    }
>      hwaddr firmware_end_addr;
>      vaddr kernel_start_addr;
>      const char *firmware_name = riscv_default_firmware_name(&s->soc[0]);
> @@ -1663,6 +1692,17 @@ static void virt_machine_init(MachineState *machine)
>              ROUND_UP(virt_high_pcie_memmap.base, virt_high_pcie_memmap.size);
>      }
>  
> +    create_cxl_host_reg_region(s);
> +
> +    if (s->cxl_devices_state.is_enabled) {
> +        hwaddr cxl_base = virt_high_pcie_memmap.base +
> +                          virt_high_pcie_memmap.size;
> +        cxl_base += memory_region_size(&s->cxl_devices_state.host_mr);
> +        cxl_base = ROUND_UP(cxl_base, 256 * MiB);
> +        cxl_fmws_set_memmap(cxl_base, UINT64_MAX);
> +        cxl_fmws_update_mmio();
> +    }
> +
>      /* register system main memory (actual RAM) */
>      memory_region_add_subregion(system_memory, s->memmap[VIRT_DRAM].base,
>                                  machine->ram);
> @@ -1769,6 +1809,8 @@ static void virt_machine_instance_init(Object *obj)
>      s->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
>      s->acpi = ON_OFF_AUTO_AUTO;
>      s->iommu_sys = ON_OFF_AUTO_AUTO;
> +
> +    cxl_machine_init(obj, &s->cxl_devices_state);
>  }
>  
>  static char *virt_get_aia_guests(Object *obj, Error **errp)
> diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h
> index 18a2a323a3..c3201588bb 100644
> --- a/include/hw/riscv/virt.h
> +++ b/include/hw/riscv/virt.h
> @@ -24,6 +24,7 @@
>  #include "hw/core/sysbus.h"
>  #include "hw/block/flash.h"
>  #include "hw/intc/riscv_imsic.h"
> +#include "hw/cxl/cxl.h"
>  
>  #define VIRT_CPUS_MAX_BITS             9
>  #define VIRT_CPUS_MAX                  (1 << VIRT_CPUS_MAX_BITS)
> @@ -64,6 +65,8 @@ struct RISCVVirtState {
>      struct GPEXHost *gpex_host;
>      OnOffAuto iommu_sys;
>      uint16_t pci_iommu_bdf;
> +    CXLState cxl_devices_state;
> +    PCIBus *bus;

That's a very vague bit of naming.  I'd make it explicit what PCIBus this is.

>  };
>  
>  enum {

Re: [PATCH 1/4] hw/riscv/virt: Add CXL support to the RISC-V virt machine

[Chen Pei]

On Tue, 9 Jun 2026 13:41:01 +0100, Jonathan Cameron <jic23@kernel.org> wrote:

Hi Jonathan,

Thanks for the review.

> > @@ -1426,6 +1446,15 @@ static void virt_machine_done(Notifier *notifier, void *data)
> >                                       machine_done);
> >      MachineState *machine = MACHINE(s);
> >      hwaddr start_addr = s->memmap[VIRT_DRAM].base;
> > +
> > +    if (s->bus) {
> 
> There is a guard against !s->bus inside the function so you should be
> able to do this unconditionally.

You're right. Will drop the outer if (s->bus) guard in v2.

> > @@ -64,6 +65,8 @@ struct RISCVVirtState {
> >      struct GPEXHost *gpex_host;
> >      OnOffAuto iommu_sys;
> >      uint16_t pci_iommu_bdf;
> > +    CXLState cxl_devices_state;
> > +    PCIBus *bus;
> 
> That's a very vague bit of naming.  I'd make it explicit what PCIBus this is.

Agreed. Will rename it to `pci_bus` in v2 to make it clear this is
the root bus of the PCIe host bridge (PCI0).

Best,
Pei

[PATCH 2/4] hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge dependency

[Chen Pei]

On RISC-V QEMU virt platform with CXL enabled, the probe ordering
of acpi_pci_root (ACPI0016) and cxl_acpi (ACPI0017) is not
guaranteed. If cxl_acpi probes before acpi_pci_root has attached
the CXL host bridges, the CXL port topology will be incomplete
because to_cxl_host_bridge() silently skips devices whose PCI root
is not yet ready.

Add a _DEP object to the ACPI0017 device in the DSDT, declaring
its dependency on the ACPI0016 CXL host bridge devices. This tells
the OS to defer ACPI0017 enumeration until all ACPI0016 devices
have been attached by acpi_pci_root.

This requires a corresponding kernel change to call
acpi_dev_clear_dependencies() in acpi_pci_root_add().

Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
 hw/riscv/virt-acpi-build.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/hw/riscv/virt-acpi-build.c b/hw/riscv/virt-acpi-build.c
index 309d64b322..a5bafd1dcf 100644
--- a/hw/riscv/virt-acpi-build.c
+++ b/hw/riscv/virt-acpi-build.c
@@ -510,6 +510,38 @@ static void build_dsdt(GArray *table_data,
     if (s->cxl_devices_state.is_enabled) {
         Aml *cxl_dev = aml_device("CXLM");
         aml_append(cxl_dev, aml_name_decl("_HID", aml_string("ACPI0017")));
+
+        /*
+         * Declare a _DEP on every ACPI0016 CXL host bridge so the OS
+         * defers ACPI0017 enumeration until acpi_pci_root has attached
+         * the CXL host bridges. Without this, cxl_acpi may probe before
+         * to_cxl_host_bridge() can resolve the PCI root and the CXL
+         * port topology comes up empty.
+         */
+        if (s->bus) {
+            PCIBus *bus;
+            uint32_t num_cxl_hbs = 0;
+
+            QLIST_FOREACH(bus, &s->bus->child, sibling) {
+                if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
+                    num_cxl_hbs++;
+                }
+            }
+
+            if (num_cxl_hbs > 0) {
+                Aml *dep_pkg = aml_package(num_cxl_hbs);
+
+                QLIST_FOREACH(bus, &s->bus->child, sibling) {
+                    if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
+                        aml_append(dep_pkg,
+                                   aml_name("\\_SB.PC%.02X",
+                                            pci_bus_num(bus)));
+                    }
+                }
+                aml_append(cxl_dev, aml_name_decl("_DEP", dep_pkg));
+            }
+        }
+
         Aml *method = aml_method("_STA", 0, AML_NOTSERIALIZED);
         aml_append(method, aml_return(aml_int(0x0B)));
         aml_append(cxl_dev, method);
-- 
2.50.1

Re: [PATCH 2/4] hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge dependency

[Jonathan Cameron]

On Tue,  2 Jun 2026 15:41:25 +0800
Chen Pei <cp0613@linux.alibaba.com> wrote:

> On RISC-V QEMU virt platform with CXL enabled, the probe ordering
> of acpi_pci_root (ACPI0016) and cxl_acpi (ACPI0017) is not
> guaranteed. If cxl_acpi probes before acpi_pci_root has attached
> the CXL host bridges, the CXL port topology will be incomplete
> because to_cxl_host_bridge() silently skips devices whose PCI root
> is not yet ready.
> 
> Add a _DEP object to the ACPI0017 device in the DSDT, declaring
> its dependency on the ACPI0016 CXL host bridge devices. This tells
> the OS to defer ACPI0017 enumeration until all ACPI0016 devices
> have been attached by acpi_pci_root.
> 
> This requires a corresponding kernel change to call
> acpi_dev_clear_dependencies() in acpi_pci_root_add().
> 
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>

It may be a good idea to add a bios tables test as well.
 
We only have the x86 q35 one today because I argued at the time
the ARM64 one would be just duplication.  Now we have this new _DEP
stuff we should probably add something to cover it.

Please also include an iasl -d dump of the relevant additions to DSDT
in the patch description.  Much easier to review than the code that
generates it!

In general looks fine to me and great that you are clearing this up.
Seems it's luck that x86 and ARM64 worked without this (or strictly
speaking other things enforcing the ordering).  Ultimately we probably
want to add this to those two architectures a well.

One small thing inline.

Thanks,

Jonathan


> ---
>  hw/riscv/virt-acpi-build.c | 32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)
> 
> diff --git a/hw/riscv/virt-acpi-build.c b/hw/riscv/virt-acpi-build.c
> index 309d64b322..a5bafd1dcf 100644
> --- a/hw/riscv/virt-acpi-build.c
> +++ b/hw/riscv/virt-acpi-build.c
> @@ -510,6 +510,38 @@ static void build_dsdt(GArray *table_data,
>      if (s->cxl_devices_state.is_enabled) {
>          Aml *cxl_dev = aml_device("CXLM");
>          aml_append(cxl_dev, aml_name_decl("_HID", aml_string("ACPI0017")));
> +
> +        /*
> +         * Declare a _DEP on every ACPI0016 CXL host bridge so the OS
> +         * defers ACPI0017 enumeration until acpi_pci_root has attached
> +         * the CXL host bridges. Without this, cxl_acpi may probe before
> +         * to_cxl_host_bridge() can resolve the PCI root and the CXL
> +         * port topology comes up empty.
> +         */
> +        if (s->bus) {
> +            PCIBus *bus;
> +            uint32_t num_cxl_hbs = 0;
> +
> +            QLIST_FOREACH(bus, &s->bus->child, sibling) {
> +                if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
> +                    num_cxl_hbs++;

I think you only care if there is at least one. Instead of counting, just set
a bool and break out early if you find one.

> +                }
> +            }
> +
> +            if (num_cxl_hbs > 0) {
> +                Aml *dep_pkg = aml_package(num_cxl_hbs);
> +
> +                QLIST_FOREACH(bus, &s->bus->child, sibling) {
> +                    if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
> +                        aml_append(dep_pkg,
> +                                   aml_name("\\_SB.PC%.02X",
> +                                            pci_bus_num(bus)));
> +                    }
> +                }
> +                aml_append(cxl_dev, aml_name_decl("_DEP", dep_pkg));
> +            }
> +        }
> +
>          Aml *method = aml_method("_STA", 0, AML_NOTSERIALIZED);
>          aml_append(method, aml_return(aml_int(0x0B)));
>          aml_append(cxl_dev, method);

Re: [PATCH 2/4] hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge dependency

[Peter Maydell]

On Tue, 9 Jun 2026 at 13:48, Jonathan Cameron <jic23@kernel.org> wrote:
>
> On Tue,  2 Jun 2026 15:41:25 +0800
> Chen Pei <cp0613@linux.alibaba.com> wrote:
>
> > On RISC-V QEMU virt platform with CXL enabled, the probe ordering
> > of acpi_pci_root (ACPI0016) and cxl_acpi (ACPI0017) is not
> > guaranteed. If cxl_acpi probes before acpi_pci_root has attached
> > the CXL host bridges, the CXL port topology will be incomplete
> > because to_cxl_host_bridge() silently skips devices whose PCI root
> > is not yet ready.
> >
> > Add a _DEP object to the ACPI0017 device in the DSDT, declaring
> > its dependency on the ACPI0016 CXL host bridge devices. This tells
> > the OS to defer ACPI0017 enumeration until all ACPI0016 devices
> > have been attached by acpi_pci_root.
> >
> > This requires a corresponding kernel change to call
> > acpi_dev_clear_dependencies() in acpi_pci_root_add().
> >
> > Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
>
> It may be a good idea to add a bios tables test as well.
>
> We only have the x86 q35 one today because I argued at the time
> the ARM64 one would be just duplication.  Now we have this new _DEP
> stuff we should probably add something to cover it.
>
> Please also include an iasl -d dump of the relevant additions to DSDT
> in the patch description.  Much easier to review than the code that
> generates it!
>
> In general looks fine to me and great that you are clearing this up.
> Seems it's luck that x86 and ARM64 worked without this (or strictly
> speaking other things enforcing the ordering).  Ultimately we probably
> want to add this to those two architectures a well.
>
> One small thing inline.

> > +
> > +            QLIST_FOREACH(bus, &s->bus->child, sibling) {
> > +                if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
> > +                    num_cxl_hbs++;
>
> I think you only care if there is at least one. Instead of counting, just set
> a bool and break out early if you find one.

> > +                Aml *dep_pkg = aml_package(num_cxl_hbs);

Looks like we need it so we can set the num_elements here.

(If we had a version of aml_package() that let you set the number
of elements after the fact we could do this all with a single loop,
but it doesn't look like we have any other places where that would
be useful. Almost all the callers pass a fixed number.)

-- PMM

Re: [PATCH 2/4] hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge dependency

[Chen Pei]

On Tue, 9 Jun 2026 13:47:09 +0100, Jonathan Cameron <jic23@kernel.org> wrote:

Hi Jonathan,

Thanks for the detailed review!

> It may be a good idea to add a bios tables test as well.

Okay. I'll add a RISC-V ACPI bios tables test in v2 to cover
the _DEP additions and prevent regressions.

> Please also include an iasl -d dump of the relevant additions to DSDT
> in the patch description. Much easier to review than the code that
> generates it!

Will do. I'll include the iasl -d decompiled output of the ACPI0017
device (with _DEP) in the v2 commit message.

> In general looks fine to me and great that you are clearing this up.
> Seems it's luck that x86 and ARM64 worked without this (or strictly
> speaking other things enforcing the ordering). Ultimately we probably
> want to add this to those two architectures as well.

Agreed. I can send a follow-up series to add _DEP for x86/ARM64
once this lands on RISC-V.

> > +        if (s->bus) {
> > +            PCIBus *bus;
> > +            uint32_t num_cxl_hbs = 0;
> > +
> > +            QLIST_FOREACH(bus, &s->bus->child, sibling) {
> > +                if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
> > +                    num_cxl_hbs++;
> 
> I think you only care if there is at least one. Instead of counting, just set
> a bool and break out early if you find one.

As Peter pointed out, the count is actually needed for
aml_package(num_cxl_hbs) to set the correct number of elements.
We'd need two loops regardless (one to count, one to populate), so
I'll keep the current structure.

Best,
Pei

Re: [PATCH 2/4] hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge dependency

[Sunil V L]

On Tue, Jun 2, 2026 at 1:18 PM Chen Pei <cp0613@linux.alibaba.com> wrote:
>
> On RISC-V QEMU virt platform with CXL enabled, the probe ordering
> of acpi_pci_root (ACPI0016) and cxl_acpi (ACPI0017) is not
> guaranteed. If cxl_acpi probes before acpi_pci_root has attached
> the CXL host bridges, the CXL port topology will be incomplete
> because to_cxl_host_bridge() silently skips devices whose PCI root
> is not yet ready.
>
> Add a _DEP object to the ACPI0017 device in the DSDT, declaring
> its dependency on the ACPI0016 CXL host bridge devices. This tells
> the OS to defer ACPI0017 enumeration until all ACPI0016 devices
> have been attached by acpi_pci_root.
>
> This requires a corresponding kernel change to call
> acpi_dev_clear_dependencies() in acpi_pci_root_add().
>
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> ---
>  hw/riscv/virt-acpi-build.c | 32 ++++++++++++++++++++++++++++++++
>  1 file changed, 32 insertions(+)
>
> diff --git a/hw/riscv/virt-acpi-build.c b/hw/riscv/virt-acpi-build.c
> index 309d64b322..a5bafd1dcf 100644
> --- a/hw/riscv/virt-acpi-build.c
> +++ b/hw/riscv/virt-acpi-build.c
> @@ -510,6 +510,38 @@ static void build_dsdt(GArray *table_data,
>      if (s->cxl_devices_state.is_enabled) {
>          Aml *cxl_dev = aml_device("CXLM");
>          aml_append(cxl_dev, aml_name_decl("_HID", aml_string("ACPI0017")));
> +
> +        /*
> +         * Declare a _DEP on every ACPI0016 CXL host bridge so the OS
> +         * defers ACPI0017 enumeration until acpi_pci_root has attached
> +         * the CXL host bridges. Without this, cxl_acpi may probe before
> +         * to_cxl_host_bridge() can resolve the PCI root and the CXL
> +         * port topology comes up empty.
> +         */
> +        if (s->bus) {
> +            PCIBus *bus;
> +            uint32_t num_cxl_hbs = 0;
> +
> +            QLIST_FOREACH(bus, &s->bus->child, sibling) {
> +                if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
> +                    num_cxl_hbs++;
> +                }
> +            }
> +
> +            if (num_cxl_hbs > 0) {
> +                Aml *dep_pkg = aml_package(num_cxl_hbs);
> +
> +                QLIST_FOREACH(bus, &s->bus->child, sibling) {
> +                    if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
> +                        aml_append(dep_pkg,
> +                                   aml_name("\\_SB.PC%.02X",
> +                                            pci_bus_num(bus)));
> +                    }
> +                }
> +                aml_append(cxl_dev, aml_name_decl("_DEP", dep_pkg));
> +            }
> +        }
> +
>
LGTM.
Reviewed-by: Sunil V L <sunilvl@oss.qualcomm.com>

[PATCH 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges

[Chen Pei]

CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
and the CXL device register BAR (BAR2 on Type3 device) are declared
as 64-bit non-prefetchable memory.  A standard PCIe-to-PCI bridge
exposes a 32-bit non-prefetchable memory window plus an (optional)
64-bit prefetchable memory window, but no 64-bit non-prefetchable
window.  Linux therefore places 64-bit non-prefetchable BARs in the
32-bit non-prefetchable bridge window, which requires the bridge to
own enough address space below 4 GiB.

On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
have no non-prefetchable window and Linux fails to assign these BARs.
Marking the BARs prefetchable would work around it, but the CXL
component registers have read/write side effects and are not
prefetchable per the PCIe specification.

Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
CXL host bridges:
- Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
  PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
- Store the reserved range in a new gpex_cfg.cxl_mmio32 field
- In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
  in the CXL host bridge _CRS instead of re-using build_crs() (which
  returns an empty set when UEFI has not assigned resources yet)
- Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
  firmware driven by device-tree also respects the reservation

Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
 hw/pci-host/gpex-acpi.c    | 36 +++++++++++++++++++++--
 hw/riscv/virt.c            | 58 +++++++++++++++++++++++++++++++-------
 include/hw/pci-host/gpex.h |  1 +
 3 files changed, 83 insertions(+), 12 deletions(-)

diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
index d9820f9b41..d8b943b665 100644
--- a/hw/pci-host/gpex-acpi.c
+++ b/hw/pci-host/gpex-acpi.c
@@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
              * Resources defined for PXBs are composed of the following parts:
              * 1. The resources the pci-bridge/pcie-root-port need.
              * 2. The resources the devices behind pxb need.
+             *
+             * For CXL host bridges on platforms where UEFI (driven by
+             * FDT 'ranges') does not assign PCI resources for the CXL
+             * root bridge before ACPI table construction, build_crs()
+             * would return an empty resource set.  When the platform
+             * has reserved a dedicated MMIO window for CXL host bridges
+             * (cfg->cxl_mmio32), emit that window as a static _CRS
+             * instead.  The platform is responsible for shrinking PCI0's
+             * mmio32 window so the two do not overlap.
              */
-            crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
-                            cfg->pio.base, 0, 0, 0);
+            if (is_cxl && cfg->cxl_mmio32.size) {
+                uint64_t cxl_base = cfg->cxl_mmio32.base;
+                uint64_t cxl_size = cfg->cxl_mmio32.size;
+
+                crs = aml_resource_template();
+
+                /* 32-bit MMIO range for CXL devices */
+                aml_append(crs,
+                    aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
+                                     AML_MAX_FIXED, AML_NON_CACHEABLE,
+                                     AML_READ_WRITE, 0,
+                                     cxl_base,
+                                     cxl_base + cxl_size - 1,
+                                     0, cxl_size));
+
+                /* Bus number range */
+                aml_append(crs,
+                    aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
+                                       AML_POS_DECODE, 0,
+                                       bus_num, bus_num + 15,
+                                       0, 16));
+            } else {
+                crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
+                                &crs_range_set, cfg->pio.base, 0, 0, 0);
+            }
             aml_append(dev, aml_name_decl("_CRS", crs));
 
             if (is_cxl) {
diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
index 899f632de7..929c01fb26 100644
--- a/hw/riscv/virt.c
+++ b/hw/riscv/virt.c
@@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
 /* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
 #define VIRT64_HIGH_PCIE_MMIO_SIZE  (16 * GiB)
 
+/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
+#define VIRT_CXL_MMIO32_SIZE        (256 * MiB)
+
 static MemMapEntry virt_high_pcie_memmap;
 
 #define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
@@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
     }
     qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
         s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
-    qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
-        1, FDT_PCI_RANGE_IOPORT, 2, 0,
-        2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
-        1, FDT_PCI_RANGE_MMIO,
-        2, s->memmap[VIRT_PCIE_MMIO].base,
-        2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
-        1, FDT_PCI_RANGE_MMIO_64BIT,
-        2, virt_high_pcie_memmap.base,
-        2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
+    {
+        /*
+         * When CXL is enabled, reserve the last 256 MiB of the 32-bit
+         * MMIO window for CXL host bridges and exclude it from the main
+         * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
+         * does not allocate that range to PCI0.  The CXL host bridge
+         * _CRS declares this range independently.
+         */
+        hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
+        if (s->cxl_devices_state.is_enabled) {
+            mmio32_size -= VIRT_CXL_MMIO32_SIZE;
+        }
+        qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
+            1, FDT_PCI_RANGE_IOPORT, 2, 0,
+            2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
+            1, FDT_PCI_RANGE_MMIO,
+            2, s->memmap[VIRT_PCIE_MMIO].base,
+            2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
+            1, FDT_PCI_RANGE_MMIO_64BIT,
+            2, virt_high_pcie_memmap.base,
+            2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
+    }
 
     if (virt_is_iommu_sys_enabled(s)) {
         qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
@@ -1730,7 +1746,29 @@ static void virt_machine_init(MachineState *machine)
             qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
     }
 
-    gpex_pcie_init(system_memory, pcie_irqchip, s);
+    DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
+
+    /*
+     * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
+     * window for CXL host bridges so the bridge non-prefetchable window
+     * can hold CXL device BARs (component registers and similar 64-bit
+     * non-prefetchable BARs that need a < 4 GiB address).
+     *
+     * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
+     *   256 MiB so the two ranges do not overlap (the FDT 'ranges'
+     *   shrink happens in create_fdt_pcie()).
+     * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
+     *   a correct _CRS for the CXL host bridge (ACPI0016).
+     */
+    if (s->cxl_devices_state.is_enabled) {
+        GPEXHost *gpex = GPEX_HOST(pcie_dev);
+        gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
+        gpex->gpex_cfg.cxl_mmio32.base =
+            s->memmap[VIRT_PCIE_MMIO].base +
+            s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
+        /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
+        gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
+    }
 
     create_platform_bus(s, mmio_irqchip);
 
diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
index 1da9c85bce..d38fbbacd6 100644
--- a/include/hw/pci-host/gpex.h
+++ b/include/hw/pci-host/gpex.h
@@ -43,6 +43,7 @@ struct GPEXConfig {
     MemMapEntry mmio32;
     MemMapEntry mmio64;
     MemMapEntry pio;
+    MemMapEntry cxl_mmio32;
     int         irq;
     PCIBus      *bus;
     bool        pci_native_hotplug;
-- 
2.50.1

Re: [PATCH 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges

[Jonathan Cameron]

On Tue,  2 Jun 2026 15:41:26 +0800
Chen Pei <cp0613@linux.alibaba.com> wrote:

> CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
> and the CXL device register BAR (BAR2 on Type3 device) are declared
> as 64-bit non-prefetchable memory.  A standard PCIe-to-PCI bridge
> exposes a 32-bit non-prefetchable memory window plus an (optional)
> 64-bit prefetchable memory window, but no 64-bit non-prefetchable
> window.

When you say 'standard' do you mean that is all the PCI spec allows
for? (Right now I'm not working for a SIG member so don't have a copy
to check)

>  Linux therefore places 64-bit non-prefetchable BARs in the
> 32-bit non-prefetchable bridge window, which requires the bridge to
> own enough address space below 4 GiB.
> 

This sounds a bit like the issue that Dave Jiang reported with recent
EDK2 on x86 (fedora upgraded).  We don't see it with the older EDK2
that ships with QEMU. I haven't yet figured out exactly why.
Arguably whatever they changed is a regression but we don't have good
enough testing in place to have detected it early enough.

I'd like some input from PCI / ACPI experts on this. +CC Michael and
Igor.

Like the previous patch we'd definitely want some testing around this
to make sure it doesn't accidentally get broken in future.


> On RISC-V virt the 32-bit PCIe MMIO range (1 GiB at 0x40000000) is
> currently consumed entirely by PCI0, so CXL host bridges (ACPI0016)
> have no non-prefetchable window and Linux fails to assign these BARs.
> Marking the BARs prefetchable would work around it, but the CXL
> component registers have read/write side effects and are not
> prefetchable per the PCIe specification.
> 
> Reserve the top 256 MiB of the 32-bit MMIO window exclusively for
> CXL host bridges:
> - Shrink PCI0's mmio32 window by 256 MiB in virt.c so that UEFI's
>   PciHostBridgeDxe and the ACPI _CRS for PCI0 never claim that range
> - Store the reserved range in a new gpex_cfg.cxl_mmio32 field
> - In gpex-acpi.c, emit the cxl_mmio32 range as the Memory resource
>   in the CXL host bridge _CRS instead of re-using build_crs() (which
>   returns an empty set when UEFI has not assigned resources yet)
> - Reduce the FDT 'ranges' for PCI0 by the same 256 MiB so that UEFI
>   firmware driven by device-tree also respects the reservation
> 
> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
> ---
>  hw/pci-host/gpex-acpi.c    | 36 +++++++++++++++++++++--
>  hw/riscv/virt.c            | 58 +++++++++++++++++++++++++++++++-------
>  include/hw/pci-host/gpex.h |  1 +
>  3 files changed, 83 insertions(+), 12 deletions(-)
> 
> diff --git a/hw/pci-host/gpex-acpi.c b/hw/pci-host/gpex-acpi.c
> index d9820f9b41..d8b943b665 100644
> --- a/hw/pci-host/gpex-acpi.c
> +++ b/hw/pci-host/gpex-acpi.c
> @@ -158,9 +158,41 @@ void acpi_dsdt_add_gpex(Aml *scope, struct GPEXConfig *cfg)
>               * Resources defined for PXBs are composed of the following parts:
>               * 1. The resources the pci-bridge/pcie-root-port need.
>               * 2. The resources the devices behind pxb need.
> +             *
> +             * For CXL host bridges on platforms where UEFI (driven by
> +             * FDT 'ranges') does not assign PCI resources for the CXL
> +             * root bridge before ACPI table construction, build_crs()
> +             * would return an empty resource set.  When the platform
> +             * has reserved a dedicated MMIO window for CXL host bridges
> +             * (cfg->cxl_mmio32), emit that window as a static _CRS
> +             * instead.  The platform is responsible for shrinking PCI0's
> +             * mmio32 window so the two do not overlap.
>               */
> -            crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent), &crs_range_set,
> -                            cfg->pio.base, 0, 0, 0);
> +            if (is_cxl && cfg->cxl_mmio32.size) {
> +                uint64_t cxl_base = cfg->cxl_mmio32.base;
> +                uint64_t cxl_size = cfg->cxl_mmio32.size;
> +
> +                crs = aml_resource_template();
> +
> +                /* 32-bit MMIO range for CXL devices */
> +                aml_append(crs,
> +                    aml_dword_memory(AML_POS_DECODE, AML_MIN_FIXED,
> +                                     AML_MAX_FIXED, AML_NON_CACHEABLE,
> +                                     AML_READ_WRITE, 0,
> +                                     cxl_base,
> +                                     cxl_base + cxl_size - 1,
> +                                     0, cxl_size));
> +
> +                /* Bus number range */
> +                aml_append(crs,
> +                    aml_word_bus_number(AML_MIN_FIXED, AML_MAX_FIXED,
> +                                       AML_POS_DECODE, 0,
> +                                       bus_num, bus_num + 15,
> +                                       0, 16));
> +            } else {
> +                crs = build_crs(PCI_HOST_BRIDGE(BUS(bus)->parent),
> +                                &crs_range_set, cfg->pio.base, 0, 0, 0);
> +            }
>              aml_append(dev, aml_name_decl("_CRS", crs));
>  
>              if (is_cxl) {
> diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c
> index 899f632de7..929c01fb26 100644
> --- a/hw/riscv/virt.c
> +++ b/hw/riscv/virt.c
> @@ -113,6 +113,9 @@ static const MemMapEntry virt_memmap[] = {
>  /* PCIe high mmio for RV64, size is fixed but base depends on top of RAM */
>  #define VIRT64_HIGH_PCIE_MMIO_SIZE  (16 * GiB)
>  
> +/* 32-bit MMIO range carved out of VIRT_PCIE_MMIO for CXL host bridges */
> +#define VIRT_CXL_MMIO32_SIZE        (256 * MiB)
> +
>  static MemMapEntry virt_high_pcie_memmap;
>  
>  #define VIRT_FLASH_SECTOR_SIZE (256 * KiB)
> @@ -890,15 +893,28 @@ static void create_fdt_pcie(RISCVVirtState *s,
>      }
>      qemu_fdt_setprop_sized_cells(ms->fdt, name, "reg", 2,
>          s->memmap[VIRT_PCIE_ECAM].base, 2, s->memmap[VIRT_PCIE_ECAM].size);
> -    qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> -        1, FDT_PCI_RANGE_IOPORT, 2, 0,
> -        2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> -        1, FDT_PCI_RANGE_MMIO,
> -        2, s->memmap[VIRT_PCIE_MMIO].base,
> -        2, s->memmap[VIRT_PCIE_MMIO].base, 2, s->memmap[VIRT_PCIE_MMIO].size,
> -        1, FDT_PCI_RANGE_MMIO_64BIT,
> -        2, virt_high_pcie_memmap.base,
> -        2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> +    {
> +        /*
> +         * When CXL is enabled, reserve the last 256 MiB of the 32-bit
> +         * MMIO window for CXL host bridges and exclude it from the main
> +         * PCIe host bridge's FDT 'ranges' so UEFI's PciHostBridgeDxe
> +         * does not allocate that range to PCI0.  The CXL host bridge
> +         * _CRS declares this range independently.
> +         */
> +        hwaddr mmio32_size = s->memmap[VIRT_PCIE_MMIO].size;
> +        if (s->cxl_devices_state.is_enabled) {
> +            mmio32_size -= VIRT_CXL_MMIO32_SIZE;
> +        }
> +        qemu_fdt_setprop_sized_cells(ms->fdt, name, "ranges",
> +            1, FDT_PCI_RANGE_IOPORT, 2, 0,
> +            2, s->memmap[VIRT_PCIE_PIO].base, 2, s->memmap[VIRT_PCIE_PIO].size,
> +            1, FDT_PCI_RANGE_MMIO,
> +            2, s->memmap[VIRT_PCIE_MMIO].base,
> +            2, s->memmap[VIRT_PCIE_MMIO].base, 2, mmio32_size,
> +            1, FDT_PCI_RANGE_MMIO_64BIT,
> +            2, virt_high_pcie_memmap.base,
> +            2, virt_high_pcie_memmap.base, 2, virt_high_pcie_memmap.size);
> +    }
>  
>      if (virt_is_iommu_sys_enabled(s)) {
>          qemu_fdt_setprop_cells(ms->fdt, name, "iommu-map",
> @@ -1730,7 +1746,29 @@ static void virt_machine_init(MachineState *machine)
>              qdev_get_gpio_in(virtio_irqchip, VIRTIO_IRQ + i));
>      }
>  
> -    gpex_pcie_init(system_memory, pcie_irqchip, s);
> +    DeviceState *pcie_dev = gpex_pcie_init(system_memory, pcie_irqchip, s);
> +
> +    /*
> +     * If CXL is enabled, reserve the last 256 MiB of the 32-bit MMIO
> +     * window for CXL host bridges so the bridge non-prefetchable window
> +     * can hold CXL device BARs (component registers and similar 64-bit
> +     * non-prefetchable BARs that need a < 4 GiB address).
> +     *
> +     * - Shrink PCI0's mmio32 advertised in the ACPI _CRS by the same
> +     *   256 MiB so the two ranges do not overlap (the FDT 'ranges'
> +     *   shrink happens in create_fdt_pcie()).
> +     * - Store the reserved range in cxl_mmio32 so gpex-acpi.c can emit
> +     *   a correct _CRS for the CXL host bridge (ACPI0016).
> +     */
> +    if (s->cxl_devices_state.is_enabled) {
> +        GPEXHost *gpex = GPEX_HOST(pcie_dev);
> +        gpex->gpex_cfg.cxl_mmio32.size = VIRT_CXL_MMIO32_SIZE;
> +        gpex->gpex_cfg.cxl_mmio32.base =
> +            s->memmap[VIRT_PCIE_MMIO].base +
> +            s->memmap[VIRT_PCIE_MMIO].size - VIRT_CXL_MMIO32_SIZE;
> +        /* Shrink PCI0's advertised 32-bit MMIO window to exclude CXL range */
> +        gpex->gpex_cfg.mmio32.size -= VIRT_CXL_MMIO32_SIZE;
> +    }
>  
>      create_platform_bus(s, mmio_irqchip);
>  
> diff --git a/include/hw/pci-host/gpex.h b/include/hw/pci-host/gpex.h
> index 1da9c85bce..d38fbbacd6 100644
> --- a/include/hw/pci-host/gpex.h
> +++ b/include/hw/pci-host/gpex.h
> @@ -43,6 +43,7 @@ struct GPEXConfig {
>      MemMapEntry mmio32;
>      MemMapEntry mmio64;
>      MemMapEntry pio;
> +    MemMapEntry cxl_mmio32;
>      int         irq;
>      PCIBus      *bus;
>      bool        pci_native_hotplug;

Re: [PATCH 3/4] hw/riscv/virt, gpex: Provide 32-bit MMIO window for CXL host bridges

[Chen Pei]

On Tue, 9 Jun 2026 13:56:11 +0100, Jonathan Cameron <jic23@kernel.org> wrote:

Hi Jonathan,

> > CXL component register BAR (BAR0 on CXL Root Port and Type3 device)
> > and the CXL device register BAR (BAR2 on Type3 device) are declared
> > as 64-bit non-prefetchable memory.  A standard PCIe-to-PCI bridge
> > exposes a 32-bit non-prefetchable memory window plus an (optional)
> > 64-bit prefetchable memory window, but no 64-bit non-prefetchable
> > window.
> 
> When you say 'standard' do you mean that is all the PCI spec allows
> for?

Yes, I confirmed against the PCI-to-PCI Bridge Architecture
Specification, Revision 1.2 (PCI-SIG, 2003). Relevant sections:

  - Section 3.2.5.8 (Memory Base Register and Memory Limit Register):
      "The upper 12 bits of both the Memory Base and Memory Limit
       registers are read/write and correspond to the upper 12 address
       bits, AD[31::20], of 32-bit addresses."
    -> The non-prefetchable memory window is 32-bit only, and the
       Type 1 header defines no upper-32-bit extension for it.

  - Section 3.2.5.9 (Prefetchable Memory Base/Limit Register):
      The bottom 4 bits encode 64-bit support: 0h = 32-bit only,
      01h = 64-bit (extended via 3.2.5.10).

  - Section 3.2.5.10 (Prefetchable Base/Limit Upper 32 Bits):
      Optional registers that hold AD[63::32] for the 64-bit
      *prefetchable* range.

So the architecture only allows a 64-bit window when it is also
prefetchable; there is no 64-bit non-prefetchable form. PCIe
inherits this Type 1 header layout unchanged.

Public mirror of the spec text (Rev 1.1, identical wording for these
sections):
  https://0x04.net/~mwk/doc/pci/PCI-to-PCI%20Bridge%20Architecture%20Specification.pdf

Official Rev 1.2 page (PCI-SIG, member access):
  https://pcisig.com/specifications/conventional/pci_bridge_2_1

> >  Linux therefore places 64-bit non-prefetchable BARs in the
> > 32-bit non-prefetchable bridge window, which requires the bridge to
> > own enough address space below 4 GiB.
> > 
> 
> This sounds a bit like the issue that Dave Jiang reported with recent
> EDK2 on x86 (fedora upgraded).  We don't see it with the older EDK2
> that ships with QEMU. I haven't yet figured out exactly why.
> Arguably whatever they changed is a regression but we don't have good
> enough testing in place to have detected it early enough.
> 
> I'd like some input from PCI / ACPI experts on this. +CC Michael and
> Igor.
> 
> Like the previous patch we'd definitely want some testing around this
> to make sure it doesn't accidentally get broken in future.

The symptoms do look related -- in both cases UEFI's
PciHostBridgeDxe ends up not giving a 32-bit non-prefetchable
window to the CXL host bridge (ACPI0016), so build_crs() in
gpex-acpi.c returns an empty resource set. I haven't dug deep
enough into Dave's case to say whether the underlying trigger is
the same as on RISC-V virt though.

Would appreciate input from Michael / Igor / Dave on whether the
approach in this patch (carve out a dedicated cxl_mmio32 range and
emit a static _CRS for ACPI0016) is a reasonable shape for a
shared fix, or if there's a preferred direction on the PCI/ACPI
side.

Best,
Pei

[PATCH 4/4] hw/cxl: Map committed HDM decoder ranges as RAM for direct DMA

[Chen Pei]

The CXL Fixed Memory Window (CFMW) is registered with
memory_region_init_io() and has no backing ram_block.  As a result
address_space_map() on a guest physical address inside the CFMW
takes the bounce-buffer path and is bounded by
DEFAULT_MAX_BOUNCE_BUFFER_SIZE (4 KiB by default for the system
AddressSpace).  Once a Type-3 device is brought online as system RAM
(daxctl online-memory), the kernel happily allocates DMA buffers from
the CFMW range and any virtio operation whose scatter list exceeds
4 KiB or that overlaps with another in-flight transfer fails with:

    qemu-system-riscv64: virtio: bogus descriptor or out of resources

The bug is not RISC-V specific: CFMW registration and the bounce
buffer limit are both arch-agnostic, so any guest that onlines CXL
memory and issues DMA larger than 4 KiB into it is affected.  It
shows up first on RISC-V virt because that is where the rest of
this series enables the daxctl + virtio path end-to-end.

Reproduce on RISC-V virt with cxl=on and a single Type-3 device:

    cxl create-region -m -t ram -d decoder0.0 -w 1 mem0 -s 4G
    daxctl online-memory dax0.0
    free -h    # triggers the error and stalls the guest

Fix it by overlaying a RAM alias of the device's memory backend
(hostvmem / hostpmem) at the committed HDM decoder's HPA range, with
higher priority than the CFMW I/O region.  flatview_translate() then
hits the alias, address_space_map() returns a direct host pointer,
and DMA proceeds without bouncing.  This mirrors the existing QEMU
pattern of PCI BAR and IOMMU MR overlays.  The alias is torn down on
hdm_decoder_uncommit() so subsequent region tear-down + re-creation
works.

Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
 hw/mem/cxl_type3.c          | 81 +++++++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl_device.h |  4 ++
 2 files changed, 85 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 4739239da3..f962bce66a 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -24,6 +24,7 @@
 #include "qemu/module.h"
 #include "qemu/pmem.h"
 #include "qemu/range.h"
+#include "system/address-spaces.h"
 #include "qemu/rcu.h"
 #include "qemu/guest-random.h"
 #include "system/hostmem.h"
@@ -420,6 +421,11 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
     ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
     uint32_t *cache_mem = cregs->cache_mem_registers;
     uint32_t ctrl;
+    uint32_t low, high;
+    uint64_t decoder_base, decoder_size;
+    MemoryRegion *mr = NULL;
+    uint64_t dpa_offset = 0;
+    char *alias_name;
 
     ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc);
     /* TODO: Sanity checks that the decoder is possible */
@@ -427,6 +433,73 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
     ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
 
     stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl);
+
+    /*
+     * Create a RAM alias in system memory for the committed decoder range.
+     * This enables direct DMA mapping (address_space_map) for devices like
+     * virtio that need to DMA to/from CXL memory.  Without this, the CFMW
+     * I/O region would require bounce buffering which is limited to 4KB.
+     */
+    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + which * hdm_inc);
+    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + which * hdm_inc);
+    decoder_base = ((uint64_t)high << 32) | (low & 0xf0000000);
+
+    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + which * hdm_inc);
+    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + which * hdm_inc);
+    decoder_size = ((uint64_t)high << 32) | (low & 0xf0000000);
+
+    if (!decoder_base || !decoder_size) {
+        return;
+    }
+
+    /* Calculate DPA offset by summing sizes of preceding decoders */
+    for (int i = 0; i < which; i++) {
+        uint32_t prev_low, prev_high;
+        uint64_t prev_size;
+        uint32_t prev_ctrl;
+
+        prev_ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL +
+                             i * hdm_inc);
+        if (!FIELD_EX32(prev_ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) {
+            continue;
+        }
+        prev_low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO +
+                            i * hdm_inc);
+        prev_high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI +
+                             i * hdm_inc);
+        prev_size = ((uint64_t)prev_high << 32) | (prev_low & 0xf0000000);
+        dpa_offset += prev_size;
+    }
+
+    /* Determine which memory backend to alias */
+    if (ct3d->hostvmem) {
+        MemoryRegion *vmr = host_memory_backend_get_memory(ct3d->hostvmem);
+        uint64_t vmr_size = memory_region_size(vmr);
+
+        if (dpa_offset < vmr_size) {
+            mr = vmr;
+        }
+    }
+    if (!mr && ct3d->hostpmem) {
+        MemoryRegion *pmr = host_memory_backend_get_memory(ct3d->hostpmem);
+        uint64_t vmr_size = ct3d->hostvmem ?
+            memory_region_size(
+                host_memory_backend_get_memory(ct3d->hostvmem)) : 0;
+        mr = pmr;
+        dpa_offset -= vmr_size;
+    }
+
+    if (!mr) {
+        return;
+    }
+
+    alias_name = g_strdup_printf("cxl-hdm%d-ram-alias", which);
+    memory_region_init_alias(&ct3d->hdm_ram_alias[which], OBJECT(ct3d),
+                             alias_name, mr, dpa_offset, decoder_size);
+    memory_region_add_subregion_overlap(get_system_memory(), decoder_base,
+                                        &ct3d->hdm_ram_alias[which], 1);
+    ct3d->hdm_ram_alias_valid[which] = true;
+    g_free(alias_name);
 }
 
 static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which)
@@ -442,6 +515,14 @@ static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which)
     ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 0);
 
     stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl);
+
+    /* Remove the RAM alias if it was added during commit */
+    if (ct3d->hdm_ram_alias_valid[which]) {
+        memory_region_del_subregion(get_system_memory(),
+                                    &ct3d->hdm_ram_alias[which]);
+        object_unparent(OBJECT(&ct3d->hdm_ram_alias[which]));
+        ct3d->hdm_ram_alias_valid[which] = false;
+    }
 }
 
 static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err)
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 393f312217..07deef2e2c 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -714,6 +714,10 @@ struct CXLType3Dev {
     /* State */
     AddressSpace hostvmem_as;
     AddressSpace hostpmem_as;
+
+    /* RAM aliases for HDM decoders - enables direct DMA mapping */
+    MemoryRegion hdm_ram_alias[CXL_HDM_DECODER_COUNT];
+    bool hdm_ram_alias_valid[CXL_HDM_DECODER_COUNT];
     CXLComponentState cxl_cstate;
     CXLDeviceState cxl_dstate;
     CXLCCI cci; /* Primary PCI mailbox CCI */
-- 
2.50.1

Re: [PATCH 4/4] hw/cxl: Map committed HDM decoder ranges as RAM for direct DMA

[Chen Pei]

On 2 Jun 2026 15:41:27 +0800, cp0613@linux.alibaba.com wrote:

CC Jonathan.

Re: [PATCH 4/4] hw/cxl: Map committed HDM decoder ranges as RAM for direct DMA

[Jonathan Cameron]

On Tue,  2 Jun 2026 15:41:27 +0800
Chen Pei <cp0613@linux.alibaba.com> wrote:

> The CXL Fixed Memory Window (CFMW) is registered with
> memory_region_init_io() and has no backing ram_block.  As a result
> address_space_map() on a guest physical address inside the CFMW
> takes the bounce-buffer path and is bounded by
> DEFAULT_MAX_BOUNCE_BUFFER_SIZE (4 KiB by default for the system
> AddressSpace).  Once a Type-3 device is brought online as system RAM
> (daxctl online-memory), the kernel happily allocates DMA buffers from
> the CFMW range and any virtio operation whose scatter list exceeds
> 4 KiB or that overlaps with another in-flight transfer fails with:
> 
>     qemu-system-riscv64: virtio: bogus descriptor or out of resources

This is half the bug if you are running without the stuff I talk about
below - you can't run instructions out of that memory either.

As to the bonce buffer that should be larger anyway - there has
been some discussion in the past on how to do that for the global one
(there is support for PCI specific cases) but we never got to a
standard solution.

> 
> The bug is not RISC-V specific: CFMW registration and the bounce
> buffer limit are both arch-agnostic, so any guest that onlines CXL
> memory and issues DMA larger than 4 KiB into it is affected.  It
> shows up first on RISC-V virt because that is where the rest of
> this series enables the daxctl + virtio path end-to-end.
> 
> Reproduce on RISC-V virt with cxl=on and a single Type-3 device:
> 
>     cxl create-region -m -t ram -d decoder0.0 -w 1 mem0 -s 4G
>     daxctl online-memory dax0.0
>     free -h    # triggers the error and stalls the guest
> 
> Fix it by overlaying a RAM alias of the device's memory backend
> (hostvmem / hostpmem) at the committed HDM decoder's HPA range, with
> higher priority than the CFMW I/O region.  flatview_translate() then
> hits the alias, address_space_map() returns a direct host pointer,
> and DMA proceeds without bouncing.  This mirrors the existing QEMU
> pattern of PCI BAR and IOMMU MR overlays.  The alias is torn down on
> hdm_decoder_uncommit() so subsequent region tear-down + re-creation
> works.
> 

+CC linux-cxl as that's where you are more likely to get feedback on 
CXL aspects.

For this one there is a more comprehensive solution that restricts this
to the cases that are valid (no interleaving going on).
https://lore.kernel.org/qemu-devel/20260318171918.146-1-alireza.sanaee@huawei.com/#t
It is rather more complex as a result.

For other cases we have always intentionally left it as not supporting
what you have here because we want the interleave to be correct and testable.

As far as I'm concerned that series of Alireza's has been ready to merge for
a while. I've been a bit busy with other things so haven't been poking
for that though.  Might be a week or two before I get back on top of things.

Anyhow, please give Ali's series a go and see if that works for you.
There may well be other corners we haven't thought of!

Jonathan



> Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>



> ---
>  hw/mem/cxl_type3.c          | 81 +++++++++++++++++++++++++++++++++++++
>  include/hw/cxl/cxl_device.h |  4 ++
>  2 files changed, 85 insertions(+)
> 
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 4739239da3..f962bce66a 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -24,6 +24,7 @@
>  #include "qemu/module.h"
>  #include "qemu/pmem.h"
>  #include "qemu/range.h"
> +#include "system/address-spaces.h"
>  #include "qemu/rcu.h"
>  #include "qemu/guest-random.h"
>  #include "system/hostmem.h"
> @@ -420,6 +421,11 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
>      ComponentRegisters *cregs = &ct3d->cxl_cstate.crb;
>      uint32_t *cache_mem = cregs->cache_mem_registers;
>      uint32_t ctrl;
> +    uint32_t low, high;
> +    uint64_t decoder_base, decoder_size;
> +    MemoryRegion *mr = NULL;
> +    uint64_t dpa_offset = 0;
> +    char *alias_name;
>  
>      ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc);
>      /* TODO: Sanity checks that the decoder is possible */
> @@ -427,6 +433,73 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int which)
>      ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
>  
>      stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl);
> +
> +    /*
> +     * Create a RAM alias in system memory for the committed decoder range.
> +     * This enables direct DMA mapping (address_space_map) for devices like
> +     * virtio that need to DMA to/from CXL memory.  Without this, the CFMW
> +     * I/O region would require bounce buffering which is limited to 4KB.
> +     */
> +    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_LO + which * hdm_inc);
> +    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_BASE_HI + which * hdm_inc);
> +    decoder_base = ((uint64_t)high << 32) | (low & 0xf0000000);
> +
> +    low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO + which * hdm_inc);
> +    high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI + which * hdm_inc);
> +    decoder_size = ((uint64_t)high << 32) | (low & 0xf0000000);
> +
> +    if (!decoder_base || !decoder_size) {
> +        return;
> +    }
> +
> +    /* Calculate DPA offset by summing sizes of preceding decoders */
> +    for (int i = 0; i < which; i++) {
> +        uint32_t prev_low, prev_high;
> +        uint64_t prev_size;
> +        uint32_t prev_ctrl;
> +
> +        prev_ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL +
> +                             i * hdm_inc);
> +        if (!FIELD_EX32(prev_ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) {
> +            continue;
> +        }
> +        prev_low = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_LO +
> +                            i * hdm_inc);
> +        prev_high = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_SIZE_HI +
> +                             i * hdm_inc);
> +        prev_size = ((uint64_t)prev_high << 32) | (prev_low & 0xf0000000);
> +        dpa_offset += prev_size;
> +    }
> +
> +    /* Determine which memory backend to alias */
> +    if (ct3d->hostvmem) {
> +        MemoryRegion *vmr = host_memory_backend_get_memory(ct3d->hostvmem);
> +        uint64_t vmr_size = memory_region_size(vmr);
> +
> +        if (dpa_offset < vmr_size) {
> +            mr = vmr;
> +        }
> +    }
> +    if (!mr && ct3d->hostpmem) {
> +        MemoryRegion *pmr = host_memory_backend_get_memory(ct3d->hostpmem);
> +        uint64_t vmr_size = ct3d->hostvmem ?
> +            memory_region_size(
> +                host_memory_backend_get_memory(ct3d->hostvmem)) : 0;
> +        mr = pmr;
> +        dpa_offset -= vmr_size;
> +    }
> +
> +    if (!mr) {
> +        return;
> +    }
> +
> +    alias_name = g_strdup_printf("cxl-hdm%d-ram-alias", which);
> +    memory_region_init_alias(&ct3d->hdm_ram_alias[which], OBJECT(ct3d),
> +                             alias_name, mr, dpa_offset, decoder_size);
> +    memory_region_add_subregion_overlap(get_system_memory(), decoder_base,
> +                                        &ct3d->hdm_ram_alias[which], 1);
> +    ct3d->hdm_ram_alias_valid[which] = true;
> +    g_free(alias_name);
>  }
>  
>  static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which)
> @@ -442,6 +515,14 @@ static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which)
>      ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 0);
>  
>      stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL + which * hdm_inc, ctrl);
> +
> +    /* Remove the RAM alias if it was added during commit */
> +    if (ct3d->hdm_ram_alias_valid[which]) {
> +        memory_region_del_subregion(get_system_memory(),
> +                                    &ct3d->hdm_ram_alias[which]);
> +        object_unparent(OBJECT(&ct3d->hdm_ram_alias[which]));
> +        ct3d->hdm_ram_alias_valid[which] = false;
> +    }
>  }
>  
>  static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err)
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 393f312217..07deef2e2c 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -714,6 +714,10 @@ struct CXLType3Dev {
>      /* State */
>      AddressSpace hostvmem_as;
>      AddressSpace hostpmem_as;
> +
> +    /* RAM aliases for HDM decoders - enables direct DMA mapping */
> +    MemoryRegion hdm_ram_alias[CXL_HDM_DECODER_COUNT];
> +    bool hdm_ram_alias_valid[CXL_HDM_DECODER_COUNT];
>      CXLComponentState cxl_cstate;
>      CXLDeviceState cxl_dstate;
>      CXLCCI cci; /* Primary PCI mailbox CCI */

Re: [PATCH 4/4] hw/cxl: Map committed HDM decoder ranges as RAM for direct DMA

[Chen Pei]

On Tue, 9 Jun 2026 13:56:11 +0100, Jonathan Cameron <jic23@kernel.org> wrote:

Hi Jonathan,

> +CC linux-cxl as that's where you are more likely to get feedback on 
> CXL aspects.
> 
> For this one there is a more comprehensive solution that restricts this
> to the cases that are valid (no interleaving going on).
> https://lore.kernel.org/qemu-devel/20260318171918.146-1-alireza.sanaee@huawei.com/#t
> It is rather more complex as a result.
> 
> For other cases we have always intentionally left it as not supporting
> what you have here because we want the interleave to be correct and testable.
> 
> As far as I'm concerned that series of Alireza's has been ready to merge for
> a while. I've been a bit busy with other things so haven't been poking
> for that though.  Might be a week or two before I get back on top of things.
> 
> Anyhow, please give Ali's series a go and see if that works for you.
> There may well be other corners we haven't thought of!

Thanks for pointing out Alireza's series.

I'll test Alireza's series against the RISC-V virt machine with the
same CXL + daxctl + virtio DMA workflow. If it resolves the issue,
I'll drop this patch from v2 and just rebase patches 1-3.

I'll report back once I've tested.

Best,
Pei

[PATCH 0/4] hw/riscv/virt: Add CXL support to the RISC-V virt machine

[Chen Pei]

This series adds CXL support to the RISC-V virt machine, following the
approach used by the ARM virt machine: CXL host bridges (pxb-cxl) are
described as ACPI0016 devices, an ACPI0017 (CXLM) device is added to the
DSDT, and a CEDT table is built.  A bios-tables test is added to pin down
the generated ACPI tables.

Prerequisite
------------
The v2 posting depended on Alireza Sanaee's v8 series [1] (the
performant non-interleaved CFMW lookup path).  That series has since
been merged upstream, so this version has no outstanding prerequisite
and applies directly on current upstream master.

Changes since v2
----------------
  - hw/riscv/virt: The MMIO-window patch no longer touches the common
    gpex code (Igor).  The gpex_cfg.cxl_mmio32 field and the is_cxl
    static _CRS branch are dropped; the ACPI0016 _CRS now comes from the
    generic build_crs() path.  Since EDK2 does not enumerate the pxb-cxl
    expander bridge, riscv simulates the firmware PCI initialization
    (reserved window + depth-first bridge bus numbers) and re-applies it
    via a reset handler.
  - hw/riscv/virt: Drops the machine-global window/bus-range synthesis
    that did not scale past one pxb-cxl (Junjie).  The series targets a
    single CXL host bridge for now, documented in the commit message and
    a TODO.
  - hw/riscv/virt: CXL host register region and FMW setup folded into a
    single cxl_host_state_init() helper; redundant braces removed in
    create_fdt_pcie() (Daniel).
  - hw/riscv/virt-acpi-build: the _DEP commit message documents the
    kernel-compatibility behaviour (Alistair).
  - Carried review tags: Sunil V L's Reviewed-by on the _DEP patch and
    Alistair Francis's Acked-by on the test.

Changes since v1
---------------
  - hw/riscv/virt: PCIBus *bus renamed to PCIBus *pci_bus (Jonathan).
  - hw/riscv/virt: Dropped outer if (s->pci_bus) guard around
    cxl_hook_up_pxb_registers(); the function already handles a NULL
    bus internally (Jonathan).
  - hw/riscv/virt-acpi-build: All s->bus references updated to
    s->pci_bus; iasl -d decompiled DSDT fragment added to commit
    message.
  - hw/riscv/virt,gpex: Commit message expanded with PCI-to-PCI Bridge
    Spec §3.2.5.8/9/10 citations (Jonathan).
  - Original patch 4 ("Map committed HDM decoder ranges as RAM for
    direct DMA") dropped; superseded by Alireza Sanaee's v8 series [1],
    which is now merged upstream and thus no longer a prerequisite.
  - New patch 4: RISC-V ACPI bios-tables test for CXL, with golden AML
    files generated and included.

[1] https://lore.kernel.org/qemu-devel/20260318171918.146-1-alireza.sanaee@huawei.com/

Test
----
Built riscv64-softmmu and ran the new bios-tables test together with the
existing riscv64 ACPI tests; all pass.  Also booted an RVA22 kernel with
pxb-cxl + cxl-rp + cxl-type3 + a CFMW: the root port and Type3 device
enumerate (0000:0c:00.0 / 0000:0d:00.0) and 'cxl list' reports the
memdev (4 GiB) and the CFMWS root decoder.

QEMU invocation (CXL-relevant options shown):

    qemu-system-riscv64 \
        -M virt,aia=aplic-imsic,acpi=on,cxl=on \
        -cpu rv64 -smp 2 -m 4G,maxmem=8G,slots=8 \
        -object memory-backend-ram,id=vmem0,share=on,size=4G \
        -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
        -device cxl-rp,port=0,bus=cxl.1,id=rp0,chassis=0,slot=2 \
        -device cxl-type3,bus=rp0,volatile-memdev=vmem0,id=cxl-vmem0 \
        -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G \
        ...

Verification (total system memory grows by 4 GiB after onlining):

    # cxl list
    # cxl enable-memdev mem0
    # cxl create-region -m -t ram -d decoder0.0 -w 1 mem0 -s 4G
    # daxctl online-memory dax0.0
    # free -h

Chen Pei (4):
  hw/riscv/virt: Add CXL support to the RISC-V virt machine
  hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge
    dependency
  hw/riscv/virt: Provide a 32-bit MMIO window for CXL host bridges
  tests/qtest: Add RISC-V ACPI bios tables test for CXL

 hw/riscv/Kconfig                      |   2 +
 hw/riscv/virt-acpi-build.c            |  52 +++++++
 hw/riscv/virt.c                       | 191 +++++++++++++++++++++++++-
 include/hw/riscv/virt.h               |   3 +
 tests/data/acpi/riscv64/virt/CEDT.cxl | Bin 0 -> 108 bytes
 tests/data/acpi/riscv64/virt/DSDT.cxl | Bin 0 -> 6212 bytes
 tests/qtest/bios-tables-test.c        |  54 ++++++++
 7 files changed, 301 insertions(+), 1 deletion(-)
 create mode 100644 tests/data/acpi/riscv64/virt/CEDT.cxl
 create mode 100644 tests/data/acpi/riscv64/virt/DSDT.cxl

-- 
2.50.1

[PATCH 2/4] hw/riscv/virt-acpi-build: Add _DEP to ACPI0017 for CXL host bridge dependency

[Chen Pei]

On RISC-V QEMU virt platform with CXL enabled, the probe ordering
of acpi_pci_root (ACPI0016) and cxl_acpi (ACPI0017) is not
guaranteed. If cxl_acpi probes before acpi_pci_root has attached
the CXL host bridges, the CXL port topology will be incomplete
because to_cxl_host_bridge() silently skips devices whose PCI root
is not yet ready.

Add a _DEP object to the ACPI0017 device in the DSDT, declaring
its dependency on the ACPI0016 CXL host bridge devices. This tells
the OS to defer ACPI0017 enumeration until all ACPI0016 devices
have been attached by acpi_pci_root.

This requires a corresponding kernel change to call
acpi_dev_clear_dependencies() in acpi_pci_root_add().

Note on kernel compatibility: this is a "new QEMU feature requires a
sufficiently new kernel" situation.  Kernels that process _DEP but
lack the acpi_dev_clear_dependencies() call will leave the CXLM device
deferred, so CXL memory is unusable there; kernels old enough to ignore
_DEP entirely behave exactly as before this change.  In neither case is
existing (non-CXL) PCIe affected.

The resulting DSDT fragment (iasl -d output) for a single CXL host
bridge at bus 0x01 looks like:

  Device (CXLM)
  {
      Name (_HID, "ACPI0017")
      Name (_DEP, Package (0x01)
      {
          \_SB.PC01
      })
      Method (_STA, 0, NotSerialized)
      {
          Return (0x0B)
      }
      ...
  }

Reviewed-by: Sunil V L <sunilvl@oss.qualcomm.com>
Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
---
 hw/riscv/virt-acpi-build.c | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/hw/riscv/virt-acpi-build.c b/hw/riscv/virt-acpi-build.c
index 3188cdc5bf..56292f04a7 100644
--- a/hw/riscv/virt-acpi-build.c
+++ b/hw/riscv/virt-acpi-build.c
@@ -521,6 +521,38 @@ static void build_dsdt(GArray *table_data,
     if (s->cxl_devices_state.is_enabled) {
         Aml *cxl_dev = aml_device("CXLM");
         aml_append(cxl_dev, aml_name_decl("_HID", aml_string("ACPI0017")));
+
+        /*
+         * Declare a _DEP on every ACPI0016 CXL host bridge so the OS
+         * defers ACPI0017 enumeration until acpi_pci_root has attached
+         * the CXL host bridges. Without this, cxl_acpi may probe before
+         * to_cxl_host_bridge() can resolve the PCI root and the CXL
+         * port topology comes up empty.
+         */
+        if (s->pci_bus) {
+            PCIBus *bus;
+            uint32_t num_cxl_hbs = 0;
+
+            QLIST_FOREACH(bus, &s->pci_bus->child, sibling) {
+                if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
+                    num_cxl_hbs++;
+                }
+            }
+
+            if (num_cxl_hbs > 0) {
+                Aml *dep_pkg = aml_package(num_cxl_hbs);
+
+                QLIST_FOREACH(bus, &s->pci_bus->child, sibling) {
+                    if (pci_bus_is_root(bus) && pci_bus_is_cxl(bus)) {
+                        aml_append(dep_pkg,
+                                   aml_name("\\_SB.PC%.02X",
+                                            pci_bus_num(bus)));
+                    }
+                }
+                aml_append(cxl_dev, aml_name_decl("_DEP", dep_pkg));
+            }
+        }
+
         Aml *method = aml_method("_STA", 0, AML_NOTSERIALIZED);
         aml_append(method, aml_return(aml_int(0x0B)));
         aml_append(cxl_dev, method);
-- 
2.50.1