Re: Problem with PCI-passthrough to PV guest

["Roger Pau Monné" <roger.pau@citrix.com>]

On Tue, May 05, 2026 at 05:53:31PM +0200, Jürgen Groß wrote:
> SUSE QA is seeing a problem with PCI-passthrough of a SR-IOV to a PV guest
> running a 6.4 based kernel, but I can reproduce the problem with upstream
> kernel, too.
> 
> The guest is configured with "e820_host=1", but the PCI region on the host
> isn't marked as "reserved" in the memory map, so it won't be reserved in
> the either.

But that's how it should be, device BARs shouldn't be in reserved
regions on the memory map (albeit we have seen this fairly often).

> As the guest doesn't have the ACPI table available describing the PCI regions,
> /proc/iomem in the guest won't include those regions as used, resulting in
> the kernel's resource management to use those areas for potential memory:
> 
>  resource: avoiding allocation from e820 entry [mem 0x78edc000-0x79868fff]
>  resource: avoiding allocation from e820 entry [mem 0x79d2a000-0x8fffffff]
>  resource: remaining [mem 0x0000000090000000-0x00000000ffdfffff] available
>  resource: avoiding allocation from e820 entry [mem 0xc7ffc000-0xc7ffcfff]
>  resource: remaining [mem 0x0000000090000000-0x00000000c7ffbfff] available
> 
> dom0 /proc/iomem:
>  ...
>  80000000-8fffffff : PCI MMCONFIG 0000 [bus 00-ff]
>  90000000-c7ffbfff : PCI Bus 0000:00    ← PCI MMIO window begins here
>    90000000-900fffff : PCI Bus 0000:01  ← I350 VFs assigned in this range
>    c6000000-c70fffff : PCI Bus 0000:04
> 
> dom0 e820 map:
>  ...
>  Xen: [mem 0x0000000079869000-0x0000000079d29fff] ACPI NVS
>  Xen: [mem 0x0000000079d2a000-0x000000008fffffff] reserved
>  Xen: [mem 0x00000000c7ffc000-0x00000000c7ffcfff] reserved
>  Xen: [mem 0x00000000fbffc000-0x00000000fbffcfff] reserved
>  ...
> 
> domU /proc/iomem:
>  ...
>  00100000-78f06fff : System RAM
>    01000000-01ffffff : Kernel code
>    ...
>  90000000-97ffffff : System RAM

But that's not in the mfn address space, it's just RAM in the pfn
space of the guest?

>  fee00000-fee00fff : Local APIC
> 
> domU e820 map:
>  ...
>  Xen: [mem 0x0000000079869000-0x0000000079d29fff] ACPI NVS
>  Xen: [mem 0x0000000079d2a000-0x000000008fffffff] reserved
>  Xen: [mem 0x00000000c7ffc000-0x00000000c7ffcfff] reserved
>  Xen: [mem 0x00000000fbffc000-0x00000000fbffcfff] reserved
>  ...
> 
> The VF is showing up near 0x90000
>  pci 0000:00:00.4: [8086:1520] type 00 class 0x020000
>  pci 0000:00:00.4: reg 0x10: [mem 0x90004000-0x90007fff 64bit pref]
>  pci 0000:00:00.4: reg 0x1c: [mem 0x90024000-0x90027fff 64bit pref]

While the above addresses are in the mfn address space?  I assume this
causes issues because MMIO is identity mapped in the pfn space.

>  pcifront pci-0: New device on 0000:00:00.4 found.
>  pcifront pci-0: claiming resource 0000:00:00.4/0
>  pci 0000:00:00.4: can't claim BAR 0 [mem 0x90004000-0x90007fff 64bit pref]:
> address conflict with System RAM [mem 0x90000000-0x97ffffff]
>  pcifront pci-0: Could not claim resource 0000:00:00.4/0! Device offline.
> Try using e820_host=1 in the guest config.
>  pcifront pci-0: claiming resource 0000:00:00.4/3
>  pci 0000:00:00.4: can't claim BAR 3 [mem 0x90024000-0x90027fff 64bit pref]:
> address conflict with System RAM [mem 0x90000000-0x97ffffff]
>  pcifront pci-0: Could not claim resource 0000:00:00.4/3! Device offline.
> Try using e820_host=1 in the guest config.
> 
> My first idea for solving this was to add the PCI regions from dom0's
> /proc/iomem to the e820 map of the guest, but this is more a hack than a sane
> solution.
> 
> Thoughts?

I think the issue is that the guest has created a pfn RAM range when
there is none in the provided e820.  When using "e820_host=1" the
guest should be limited to creating pfns only in the ranges marked as
RAM on the host e820 memory map.  Creating a pfn range over a hole in
the e820 shouldn't happen.

Regards, Roger.