[PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[peng guo]

When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
9p device fails to initialize properly. The kernel reports the following:

    virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
    9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22

Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
device was overlapped by the memory window allocated for the CXL devices. As a
result, the kernel could not correctly access the BAR region, causing the
virtio device to malfunction.

An excerpt from /proc/iomem shows:

    480010000-cffffffff : CXL Window 0
      480010000-4bfffffff : PCI Bus 0000:00
      4c0000000-4c01fffff : PCI Bus 0000:0c
        4c0000000-4c01fffff : PCI Bus 0000:0d
      4c0200000-cffffffff : PCI Bus 0000:00
        4c0200000-4c0203fff : 0000:00:03.0
          4c0200000-4c0203fff : virtio-pci-modern

To address this issue, this patch uses the value of `cxl_resv_end` to reserve
sufficient address space and ensure that CXL memory windows are allocated
beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
as those used by virtio or other pcie devices, resolving the conflict.

QEMU Build Configuration:

    ./configure --prefix=/home/work/qemu_master/build/ \
                --target-list=x86_64-softmmu \
                --enable-kvm \
                --enable-virtfs

QEMU Boot Command:

    sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
        -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
        -hda /home/work/gp_qemu/rootfs.img \
        -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
        -kernel /home/work/linux_output/arch/x86/boot/bzImage \
        --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
        -object memory-backend-ram,id=vmem0,share=on,size=4096M \
        -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
        -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
        -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
        -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G

Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.

Signed-off-by: peng guo <engguopeng@buaa.edu.cn>
---
 hw/i386/pc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2f58e73d3347..180bc615f3f0 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
 
     rom_set_fw(fw_cfg);
 
-    if (machine->device_memory) {
+    if (machine->device_memory || cxl_resv_end) {
         uint64_t *val = g_malloc(sizeof(*val));
         uint64_t res_mem_end;
 
-- 
2.43.0

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[Michael S. Tsirkin]

On Fri, Jul 18, 2025 at 09:35:45PM +0800, peng guo wrote:
> When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> 9p device fails to initialize properly. The kernel reports the following:
> 
>     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
>     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> 
> Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> device was overlapped by the memory window allocated for the CXL devices. As a
> result, the kernel could not correctly access the BAR region, causing the
> virtio device to malfunction.
> 
> An excerpt from /proc/iomem shows:
> 
>     480010000-cffffffff : CXL Window 0
>       480010000-4bfffffff : PCI Bus 0000:00
>       4c0000000-4c01fffff : PCI Bus 0000:0c
>         4c0000000-4c01fffff : PCI Bus 0000:0d
>       4c0200000-cffffffff : PCI Bus 0000:00
>         4c0200000-4c0203fff : 0000:00:03.0
>           4c0200000-4c0203fff : virtio-pci-modern
> 
> To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> sufficient address space and ensure that CXL memory windows are allocated
> beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> as those used by virtio or other pcie devices, resolving the conflict.
> 
> QEMU Build Configuration:
> 
>     ./configure --prefix=/home/work/qemu_master/build/ \
>                 --target-list=x86_64-softmmu \
>                 --enable-kvm \
>                 --enable-virtfs
> 
> QEMU Boot Command:
> 
>     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
>         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
>         -hda /home/work/gp_qemu/rootfs.img \
>         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
>         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
>         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
>         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
>         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
>         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
>         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
>         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> 
> Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> 
> Signed-off-by: peng guo <engguopeng@buaa.edu.cn>

Cc Jonathan.
Any input on this?


> ---
>  hw/i386/pc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 2f58e73d3347..180bc615f3f0 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
>  
>      rom_set_fw(fw_cfg);
>  
> -    if (machine->device_memory) {
> +    if (machine->device_memory || cxl_resv_end) {
>          uint64_t *val = g_malloc(sizeof(*val));
>          uint64_t res_mem_end;
>  
> -- 
> 2.43.0

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[Jonathan Cameron]

On Thu, 24 Jul 2025 03:43:56 -0400
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Fri, Jul 18, 2025 at 09:35:45PM +0800, peng guo wrote:
> > When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> > 9p device fails to initialize properly. The kernel reports the following:
> > 
> >     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
> >     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> > 
> > Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> > device was overlapped by the memory window allocated for the CXL devices. As a
> > result, the kernel could not correctly access the BAR region, causing the
> > virtio device to malfunction.
> > 
> > An excerpt from /proc/iomem shows:
> > 
> >     480010000-cffffffff : CXL Window 0
> >       480010000-4bfffffff : PCI Bus 0000:00
> >       4c0000000-4c01fffff : PCI Bus 0000:0c
> >         4c0000000-4c01fffff : PCI Bus 0000:0d
> >       4c0200000-cffffffff : PCI Bus 0000:00
> >         4c0200000-4c0203fff : 0000:00:03.0
> >           4c0200000-4c0203fff : virtio-pci-modern
> > 
> > To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> > sufficient address space and ensure that CXL memory windows are allocated
> > beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> > as those used by virtio or other pcie devices, resolving the conflict.
> > 
> > QEMU Build Configuration:
> > 
> >     ./configure --prefix=/home/work/qemu_master/build/ \
> >                 --target-list=x86_64-softmmu \
> >                 --enable-kvm \
> >                 --enable-virtfs
> > 
> > QEMU Boot Command:
> > 
> >     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
> >         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
> >         -hda /home/work/gp_qemu/rootfs.img \
> >         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
> >         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
> >         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
> >         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
> >         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> >         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
> >         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
> >         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> > 
> > Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> > 
> > Signed-off-by: peng guo <engguopeng@buaa.edu.cn>  
> 
> Cc Jonathan.
> Any input on this?

I've been more or less offline for a few days.  Will be back in office tomorrow.
At first glance this looks correct to me, but I want to take a closer look.

Jonathan

> 
> 
> > ---
> >  hw/i386/pc.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 2f58e73d3347..180bc615f3f0 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
> >  
> >      rom_set_fw(fw_cfg);
> >  
> > -    if (machine->device_memory) {
> > +    if (machine->device_memory || cxl_resv_end) {
> >          uint64_t *val = g_malloc(sizeof(*val));
> >          uint64_t res_mem_end;
> >  
> > -- 
> > 2.43.0  
> 
> 

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[Jonathan Cameron]

On Fri, 18 Jul 2025 21:35:45 +0800
peng guo <engguopeng@buaa.edu.cn> wrote:

> When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> 9p device fails to initialize properly. The kernel reports the following:
> 
>     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
>     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> 
> Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> device was overlapped by the memory window allocated for the CXL devices. As a
> result, the kernel could not correctly access the BAR region, causing the
> virtio device to malfunction.
> 
> An excerpt from /proc/iomem shows:
> 
>     480010000-cffffffff : CXL Window 0
>       480010000-4bfffffff : PCI Bus 0000:00
>       4c0000000-4c01fffff : PCI Bus 0000:0c
>         4c0000000-4c01fffff : PCI Bus 0000:0d
>       4c0200000-cffffffff : PCI Bus 0000:00
>         4c0200000-4c0203fff : 0000:00:03.0
>           4c0200000-4c0203fff : virtio-pci-modern
> 
> To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> sufficient address space and ensure that CXL memory windows are allocated
> beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> as those used by virtio or other pcie devices, resolving the conflict.
> 
> QEMU Build Configuration:
> 
>     ./configure --prefix=/home/work/qemu_master/build/ \
>                 --target-list=x86_64-softmmu \
>                 --enable-kvm \
>                 --enable-virtfs
> 
> QEMU Boot Command:
> 
>     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
>         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
>         -hda /home/work/gp_qemu/rootfs.img \
>         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
>         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
>         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
>         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
>         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
>         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
>         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
>         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> 
> Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> 
> Signed-off-by: peng guo <engguopeng@buaa.edu.cn>
Analysis looks good.

For the patch I wonder if we should match the check that follows
for pcms->cxl_devices_state.is_enabled rather than checking cxl_resv_end
(which is only set to non 0 if that is_enabled is set).

Probably better to use a consistent condition for checking if CXL is
there or not.

We also ideally need a suitable fixes tag.  I couldn't immediately find one
so maybe it goes a long way back.

> ---
>  hw/i386/pc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 2f58e73d3347..180bc615f3f0 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
>  
>      rom_set_fw(fw_cfg);
>  
> -    if (machine->device_memory) {
> +    if (machine->device_memory || cxl_resv_end) {
>          uint64_t *val = g_malloc(sizeof(*val));
>          uint64_t res_mem_end;
>  

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[peng guo]

On Fri, Jul 25, 2025 at 02:53:37PM +0100, Jonathan Cameron wrote:
> On Fri, 18 Jul 2025 21:35:45 +0800
> peng guo <engguopeng@buaa.edu.cn> wrote:
> 
> > When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> > 9p device fails to initialize properly. The kernel reports the following:
> > 
> >     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
> >     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> > 
> > Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> > device was overlapped by the memory window allocated for the CXL devices. As a
> > result, the kernel could not correctly access the BAR region, causing the
> > virtio device to malfunction.
> > 
> > An excerpt from /proc/iomem shows:
> > 
> >     480010000-cffffffff : CXL Window 0
> >       480010000-4bfffffff : PCI Bus 0000:00
> >       4c0000000-4c01fffff : PCI Bus 0000:0c
> >         4c0000000-4c01fffff : PCI Bus 0000:0d
> >       4c0200000-cffffffff : PCI Bus 0000:00
> >         4c0200000-4c0203fff : 0000:00:03.0
> >           4c0200000-4c0203fff : virtio-pci-modern
> > 
> > To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> > sufficient address space and ensure that CXL memory windows are allocated
> > beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> > as those used by virtio or other pcie devices, resolving the conflict.
> > 
> > QEMU Build Configuration:
> > 
> >     ./configure --prefix=/home/work/qemu_master/build/ \
> >                 --target-list=x86_64-softmmu \
> >                 --enable-kvm \
> >                 --enable-virtfs
> > 
> > QEMU Boot Command:
> > 
> >     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
> >         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
> >         -hda /home/work/gp_qemu/rootfs.img \
> >         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
> >         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
> >         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
> >         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
> >         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> >         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
> >         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
> >         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> > 
> > Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> > 
> > Signed-off-by: peng guo <engguopeng@buaa.edu.cn>
> Analysis looks good.
> 
> For the patch I wonder if we should match the check that follows
> for pcms->cxl_devices_state.is_enabled rather than checking cxl_resv_end
> (which is only set to non 0 if that is_enabled is set).
> 
> Probably better to use a consistent condition for checking if CXL is
> there or not.
> 
> We also ideally need a suitable fixes tag.  I couldn't immediately find one
> so maybe it goes a long way back.
> 

Checking `is_enabled` instead of `cxl_resv_end` makes sense.

Building on that, I wonder if it would be worthwhile to move the assignment of 
`res_mem_end` outside the conditional block. Then simply use `res_mem_end` 
itself as the condition. That would eliminate the need to check `is_enabled` 
directly in this spot and simplify the logic slighly. The benefit may 
be minor, but it might help unify the logic around how `res_mem_end` is used.

I will make an effort to identify the appropriate fixes tags related to this. My 
guess is that it relates to the patch where the CXL windows were originally 
introduced and activated in the system.

> > ---
> >  hw/i386/pc.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 2f58e73d3347..180bc615f3f0 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
> >  
> >      rom_set_fw(fw_cfg);
> >  
> > -    if (machine->device_memory) {
> > +    if (machine->device_memory || cxl_resv_end) {
> >          uint64_t *val = g_malloc(sizeof(*val));
> >          uint64_t res_mem_end;
> >  

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[Fan Ni]

On Fri, Jul 25, 2025 at 02:53:37PM +0100, Jonathan Cameron wrote:
> On Fri, 18 Jul 2025 21:35:45 +0800
> peng guo <engguopeng@buaa.edu.cn> wrote:
> 
> > When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> > 9p device fails to initialize properly. The kernel reports the following:
> > 
> >     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
> >     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> > 
> > Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> > device was overlapped by the memory window allocated for the CXL devices. As a
> > result, the kernel could not correctly access the BAR region, causing the
> > virtio device to malfunction.
> > 
> > An excerpt from /proc/iomem shows:
> > 
> >     480010000-cffffffff : CXL Window 0
> >       480010000-4bfffffff : PCI Bus 0000:00
> >       4c0000000-4c01fffff : PCI Bus 0000:0c
> >         4c0000000-4c01fffff : PCI Bus 0000:0d
> >       4c0200000-cffffffff : PCI Bus 0000:00
> >         4c0200000-4c0203fff : 0000:00:03.0
> >           4c0200000-4c0203fff : virtio-pci-modern
> > 
> > To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> > sufficient address space and ensure that CXL memory windows are allocated
> > beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> > as those used by virtio or other pcie devices, resolving the conflict.
> > 
> > QEMU Build Configuration:
> > 
> >     ./configure --prefix=/home/work/qemu_master/build/ \
> >                 --target-list=x86_64-softmmu \
> >                 --enable-kvm \
> >                 --enable-virtfs
> > 
> > QEMU Boot Command:
> > 
> >     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
> >         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
> >         -hda /home/work/gp_qemu/rootfs.img \
> >         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
> >         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
> >         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
> >         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
> >         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> >         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
> >         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
> >         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> > 
> > Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> > 
> > Signed-off-by: peng guo <engguopeng@buaa.edu.cn>
> Analysis looks good.
> 
> For the patch I wonder if we should match the check that follows
> for pcms->cxl_devices_state.is_enabled rather than checking cxl_resv_end
> (which is only set to non 0 if that is_enabled is set).
> 
> Probably better to use a consistent condition for checking if CXL is
> there or not.
> 
> We also ideally need a suitable fixes tag.  I couldn't immediately find one
> so maybe it goes a long way back.

FYI. Commit histroy related to the line changed,


commit 78732a765986d5270d6b3d88afeb9e4d33092360
Author: David Hildenbrand <david@redhat.com>
Date:   Fri Jun 23 14:45:49 2023 +0200

    hw/i386/pc: Use machine_memory_devices_init()

    Let's use our new helper and stop always allocating ms->device_memory.
    Once allcoated, we're sure that the size > 0 and that the base was
    initialized.

    Adjust the code in pc_memory_init() to check for machine->device_memory
    instead of pcmc->has_reserved_memory and machine->device_memory->base.

    Cc: Paolo Bonzini <pbonzini@redhat.com>
    Cc: Richard Henderson <richard.henderson@linaro.org>
    Cc: Eduardo Habkost <eduardo@habkost.net>
    Cc: "Michael S. Tsirkin" <mst@redhat.com>
    Cc: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
    Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
    Acked-by: Michael S. Tsirkin <mst@redhat.com>
    Signed-off-by: David Hildenbrand <david@redhat.com>
    Message-Id: <20230623124553.400585-7-david@redhat.com>
    Signed-off-by: David Hildenbrand <david@redhat.com>

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1123,1 +1116,1 @@
-    if (pcmc->has_reserved_memory && machine->device_memory->base) {
+    if (machine->device_memory) {

commit b0c14ec4efe912ae6f14a4802574f7b6b6db0648
Author: David Hildenbrand <david@redhat.com>
Date:   Mon Apr 23 18:51:17 2018 +0200

    machine: make MemoryHotplugState accessible via the machine

    Let's allow to query the MemoryHotplugState directly from the machine.
    If the pointer is NULL, the machine does not support memory devices. If
    the pointer is !NULL, the machine supports memory devices and the
    data structure contains information about the applicable physical
    guest address space region.

    This allows us to generically detect if a certain machine has support
    for memory devices, and to generically manage it (find free address
    range, plug/unplug a memory region).

    We will rename "MemoryHotplugState" to something more meaningful
    ("DeviceMemory") after we completed factoring out the pc-dimm code into
    MemoryDevice code.

    Signed-off-by: David Hildenbrand <david@redhat.com>
    Message-Id: <20180423165126.15441-3-david@redhat.com>
    Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
    [ehabkost: rebased series, solved conflicts at spapr.c]
    [ehabkost: squashed fix to use g_malloc0()]
    Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1432,1 +1435,1 @@
-    if (pcmc->has_reserved_memory && pcms->hotplug_memory.base) {
+    if (pcmc->has_reserved_memory && machine->device_memory->base) {

commit bb292f5a9b944e47fae88a20767967e7e20122b4
Author: Eduardo Habkost <ehabkost@redhat.com>
Date:   Fri Dec 11 16:42:28 2015 -0200

    pc: Remove compat fields from PcGuestInfo

    Remove the fields: legacy_acpi_table_size, has_acpi_build,
    has_reserved_memory, and rsdp_in_ram from PcGuestInfo, and let
    the existing code use the PCMachineClass fields directly.

    Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
    Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
    Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
    Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1385,1 +1385,1 @@
-    if (guest_info->has_reserved_memory && pcms->hotplug_memory.base) {
+    if (pcmc->has_reserved_memory && pcms->hotplug_memory.base) {

commit a7d69ff10b085ba6f8236600829532984cdea714
Author: Bharata B Rao <bharata@linux.vnet.ibm.com>
Date:   Mon Jun 29 13:50:22 2015 +0530

    pc,pc-dimm: Extract hotplug related fields in PCMachineState to a structure

    Move hotplug_memory_base and hotplug_memory fields of PCMachineState
    into a separate structure so that the same can be made use of from
    other architectures supporing memory hotplug.

    Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
    Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
    Reviewed-by: Igor Mammedov <imammedo@redhat.com>
    Tested-by: Igor Mammedov <imammedo@redhat.com>
    Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1336,1 +1336,1 @@
-    if (guest_info->has_reserved_memory && pcms->hotplug_memory_base) {
+    if (guest_info->has_reserved_memory && pcms->hotplug_memory.base) {

commit de268e134c03612970d6f2c214df6287c9621cc8
Author: Igor Mammedov <imammedo@redhat.com>
Date:   Mon Jun 2 15:25:10 2014 +0200

    pc: add 'etc/reserved-memory-end' fw_cfg interface for SeaBIOS

    'etc/reserved-memory-end' will allow QEMU to tell BIOS where PCI
    BARs mapping could safely start in high memory.

    Allowing BIOS to start mapping 64-bit PCI BARs at address where it
    wouldn't conflict with other mappings QEMU might place before it.

    That permits QEMU to reserve extra address space before
    64-bit PCI hole for memory hotplug.

    Signed-off-by: Igor Mammedov <imammedo@redhat.com>
    Acked-by: Peter Crosthwaite <peter.crosthwaite@xilinx.com>
    Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
    Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1269,0 +1270,1 @@
+    if (guest_info->has_reserved_memory && pcms->hotplug_memory_base) {


Fan

> 
> > ---
> >  hw/i386/pc.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 2f58e73d3347..180bc615f3f0 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
> >  
> >      rom_set_fw(fw_cfg);
> >  
> > -    if (machine->device_memory) {
> > +    if (machine->device_memory || cxl_resv_end) {
> >          uint64_t *val = g_malloc(sizeof(*val));
> >          uint64_t res_mem_end;
> >  
> 

-- 
Fan Ni

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[Jonathan Cameron]

On Sat, 26 Jul 2025 20:50:35 +0800
peng guo <engguopeng@buaa.edu.cn> wrote:

> On Fri, Jul 25, 2025 at 02:53:37PM +0100, Jonathan Cameron wrote:
> > On Fri, 18 Jul 2025 21:35:45 +0800
> > peng guo <engguopeng@buaa.edu.cn> wrote:
> >   
> > > When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> > > 9p device fails to initialize properly. The kernel reports the following:
> > > 
> > >     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
> > >     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> > > 
> > > Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> > > device was overlapped by the memory window allocated for the CXL devices. As a
> > > result, the kernel could not correctly access the BAR region, causing the
> > > virtio device to malfunction.
> > > 
> > > An excerpt from /proc/iomem shows:
> > > 
> > >     480010000-cffffffff : CXL Window 0
> > >       480010000-4bfffffff : PCI Bus 0000:00
> > >       4c0000000-4c01fffff : PCI Bus 0000:0c
> > >         4c0000000-4c01fffff : PCI Bus 0000:0d
> > >       4c0200000-cffffffff : PCI Bus 0000:00
> > >         4c0200000-4c0203fff : 0000:00:03.0
> > >           4c0200000-4c0203fff : virtio-pci-modern
> > > 
> > > To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> > > sufficient address space and ensure that CXL memory windows are allocated
> > > beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> > > as those used by virtio or other pcie devices, resolving the conflict.
> > > 
> > > QEMU Build Configuration:
> > > 
> > >     ./configure --prefix=/home/work/qemu_master/build/ \
> > >                 --target-list=x86_64-softmmu \
> > >                 --enable-kvm \
> > >                 --enable-virtfs
> > > 
> > > QEMU Boot Command:
> > > 
> > >     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
> > >         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
> > >         -hda /home/work/gp_qemu/rootfs.img \
> > >         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
> > >         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
> > >         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
> > >         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
> > >         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> > >         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
> > >         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
> > >         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> > > 
> > > Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> > > 
> > > Signed-off-by: peng guo <engguopeng@buaa.edu.cn>  
> > Analysis looks good.
> > 
> > For the patch I wonder if we should match the check that follows
> > for pcms->cxl_devices_state.is_enabled rather than checking cxl_resv_end
> > (which is only set to non 0 if that is_enabled is set).
> > 
> > Probably better to use a consistent condition for checking if CXL is
> > there or not.
> > 
> > We also ideally need a suitable fixes tag.  I couldn't immediately find one
> > so maybe it goes a long way back.
> >   
> 
> Checking `is_enabled` instead of `cxl_resv_end` makes sense.
> 
> Building on that, I wonder if it would be worthwhile to move the assignment of 
> `res_mem_end` outside the conditional block. Then simply use `res_mem_end` 
> itself as the condition. That would eliminate the need to check `is_enabled` 
> directly in this spot and simplify the logic slighly. The benefit may 
> be minor, but it might help unify the logic around how `res_mem_end` is used.

That might indeed end up cleaner.

Jonathan


> 
> I will make an effort to identify the appropriate fixes tags related to this. My 
> guess is that it relates to the patch where the CXL windows were originally 
> introduced and activated in the system.
> 
> > > ---
> > >  hw/i386/pc.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > > index 2f58e73d3347..180bc615f3f0 100644
> > > --- a/hw/i386/pc.c
> > > +++ b/hw/i386/pc.c
> > > @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
> > >  
> > >      rom_set_fw(fw_cfg);
> > >  
> > > -    if (machine->device_memory) {
> > > +    if (machine->device_memory || cxl_resv_end) {
> > >          uint64_t *val = g_malloc(sizeof(*val));
> > >          uint64_t res_mem_end;
> > >    
> 
> 

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[peng guo]

On Fri, Jul 25, 2025 at 02:53:37PM +0100, Jonathan Cameron wrote:
> On Fri, 18 Jul 2025 21:35:45 +0800
> peng guo <engguopeng@buaa.edu.cn> wrote:
> 
> > When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> > 9p device fails to initialize properly. The kernel reports the following:
> > 
> >     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
> >     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> > 
> > Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> > device was overlapped by the memory window allocated for the CXL devices. As a
> > result, the kernel could not correctly access the BAR region, causing the
> > virtio device to malfunction.
> > 
> > An excerpt from /proc/iomem shows:
> > 
> >     480010000-cffffffff : CXL Window 0
> >       480010000-4bfffffff : PCI Bus 0000:00
> >       4c0000000-4c01fffff : PCI Bus 0000:0c
> >         4c0000000-4c01fffff : PCI Bus 0000:0d
> >       4c0200000-cffffffff : PCI Bus 0000:00
> >         4c0200000-4c0203fff : 0000:00:03.0
> >           4c0200000-4c0203fff : virtio-pci-modern
> > 
> > To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> > sufficient address space and ensure that CXL memory windows are allocated
> > beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> > as those used by virtio or other pcie devices, resolving the conflict.
> > 
> > QEMU Build Configuration:
> > 
> >     ./configure --prefix=/home/work/qemu_master/build/ \
> >                 --target-list=x86_64-softmmu \
> >                 --enable-kvm \
> >                 --enable-virtfs
> > 
> > QEMU Boot Command:
> > 
> >     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
> >         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
> >         -hda /home/work/gp_qemu/rootfs.img \
> >         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
> >         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
> >         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
> >         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
> >         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> >         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
> >         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
> >         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> > 
> > Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> > 
> > Signed-off-by: peng guo <engguopeng@buaa.edu.cn>
> Analysis looks good.
> 
> For the patch I wonder if we should match the check that follows
> for pcms->cxl_devices_state.is_enabled rather than checking cxl_resv_end
> (which is only set to non 0 if that is_enabled is set).
> 
> Probably better to use a consistent condition for checking if CXL is
> there or not.
> 
> We also ideally need a suitable fixes tag.  I couldn't immediately find one
> so maybe it goes a long way back.
> 

Would the following be the correct Fixes tag for this patch?

Fixes: 03b39fcf64bc ("hw/cxl: Make the CXL fixed memory window setup a machine parameter")

Although earlier versions already had overlapping memory regions, they didn't 
affect device 64-bit BAR access.

> > ---
> >  hw/i386/pc.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 2f58e73d3347..180bc615f3f0 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
> >  
> >      rom_set_fw(fw_cfg);
> >  
> > -    if (machine->device_memory) {
> > +    if (machine->device_memory || cxl_resv_end) {
> >          uint64_t *val = g_malloc(sizeof(*val));
> >          uint64_t res_mem_end;
> >  

Re: [PATCH] hw/i386/pc: Avoid overlap between CXL window and PCI 64bit BARs in QEMU

[Michael S. Tsirkin]

On Fri, Jul 18, 2025 at 09:35:45PM +0800, peng guo wrote:
> When using a CXL Type 3 device together with a virtio 9p device in QEMU, the
> 9p device fails to initialize properly. The kernel reports the following:
> 
>     virtio: device uses modern interface but does not have VIRTIO_F_VERSION_1
>     9pnet_virtio virtio0: probe with driver 9pnet_virtio failed with error -22
> 
> Further investigation revealed that the 64-bit BAR space assigned to the 9pnet
> device was overlapped by the memory window allocated for the CXL devices. As a
> result, the kernel could not correctly access the BAR region, causing the
> virtio device to malfunction.
> 
> An excerpt from /proc/iomem shows:
> 
>     480010000-cffffffff : CXL Window 0
>       480010000-4bfffffff : PCI Bus 0000:00
>       4c0000000-4c01fffff : PCI Bus 0000:0c
>         4c0000000-4c01fffff : PCI Bus 0000:0d
>       4c0200000-cffffffff : PCI Bus 0000:00
>         4c0200000-4c0203fff : 0000:00:03.0
>           4c0200000-4c0203fff : virtio-pci-modern
> 
> To address this issue, this patch uses the value of `cxl_resv_end` to reserve
> sufficient address space and ensure that CXL memory windows are allocated
> beyond all PCI 64-bit BARs. This prevents overlap with 64-bit BARs regions such 
> as those used by virtio or other pcie devices, resolving the conflict.
> 
> QEMU Build Configuration:
> 
>     ./configure --prefix=/home/work/qemu_master/build/ \
>                 --target-list=x86_64-softmmu \
>                 --enable-kvm \
>                 --enable-virtfs
> 
> QEMU Boot Command:
> 
>     sudo /home/work/qemu_master/qemu/build/qemu-system-x86_64 \
>         -nographic -machine q35,cxl=on -enable-kvm -m 16G -smp 8 \
>         -hda /home/work/gp_qemu/rootfs.img \
>         -virtfs local,path=/home/work/gp_qemu/share,mount_tag=host0,security_model=passthrough,id=host0 \
>         -kernel /home/work/linux_output/arch/x86/boot/bzImage \
>         --append "console=ttyS0 crashkernel=256M root=/dev/sda rootfstype=ext4 rw loglevel=8" \
>         -object memory-backend-ram,id=vmem0,share=on,size=4096M \
>         -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
>         -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
>         -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0,sn=0x123456789 \
>         -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G
> 
> Tested in a QEMU setup with a CXL Type 3 device and a 9pnet virtio device.
> 
> Signed-off-by: peng guo <engguopeng@buaa.edu.cn>


To clarify I am expecting a new version of this.

> ---
>  hw/i386/pc.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 2f58e73d3347..180bc615f3f0 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -975,7 +975,7 @@ void pc_memory_init(PCMachineState *pcms,
>  
>      rom_set_fw(fw_cfg);
>  
> -    if (machine->device_memory) {
> +    if (machine->device_memory || cxl_resv_end) {
>          uint64_t *val = g_malloc(sizeof(*val));
>          uint64_t res_mem_end;
>  
> -- 
> 2.43.0