QEMU-Arm Archive on lore.kernel.org
 help / color / mirror / Atom feed
* [RFC PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement
@ 2026-08-27  0:40 Tushar Dave
  2026-08-27  0:40 ` [RFC PATCH v2 1/5] hw/pci: add fixed-bar and pci-bars properties Tushar Dave
                   ` (5 more replies)
  0 siblings, 6 replies; 20+ messages in thread
From: Tushar Dave @ 2026-08-27  0:40 UTC (permalink / raw)
  To: qemu-devel
  Cc: alwilliamson, jgg, skolothumtho, qemu-arm, peter.maydell, mst,
	marcel.apfelbaum, devel

This RFC v2 is a follow-up to RFC v1 [1].

On some platforms, peer-to-peer (P2P) DMA between PCIe devices requires
guest physical addresses (GPAs) assigned to device BARs to match their
corresponding host physical addresses (HPAs).

RFC v1 proposed moving PCI enumeration and BAR assignment into QEMU
before firmware execution, with EDK2 operating in discovery-only mode
via PcdPciDisableBusEnumeration.

The feedback was that PCI enumeration and resource assignment should
remain in firmware rather than being performed by the VMM. The suggested
EDK2 mechanism was EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL, which
provides a per-device interception point for PCI resource assignment.

Following the feedback, RFC v2 keeps PCI enumeration and resource
assignment in firmware. QEMU only validates the user-provided fixed
BAR configuration and provides the required metadata to firmware
through the "etc/fixed-bars" fw_cfg file.

Two new PCI properties are introduced:

* fixed-bar=on on a PCIe root port marks its subordinate hierarchy for
  fixed BAR placement. Every device with memory BARs in that hierarchy
  must provide a complete pci-bars= configuration.

* pci-bars=barN@<addr>[,barM@<addr>]... on a PCI endpoint specifies the
  required address for each memory BAR. All memory BARs on the device
  must have an explicitly assigned address.

QEMU validates the user-specified configuration before passing it to
firmware. Validation checks BAR alignment, verifies that addresses are
within the configured PCIe MMIO apertures, and ensures that fixed BAR
ranges do not overlap. QEMU performs neither PCI enumeration nor BAR
allocation.

On the firmware side, a new DXE driver, QemuFixedBarsDxe, installs
EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL before PciBusDxe starts.
When PciBusDxe calls CheckDevice() for a discovered PCI function, the
driver returns ACPI address descriptors with _MIF|_MAF set for fixed
BARs. Two small changes to PciBusDxe preserve these fixed addresses and
program them into the BAR registers during BAR programming.

After PciEnumerationComplete, QemuFixedBarsDxe walks each fixed
root-port hierarchy and programs the bridge memory windows to cover
the fixed BAR ranges assigned to endpoint devices.

Example 1: VFIO passthrough devices under a fixed BAR root port.

    -device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0,fixed-bar=on
    -device x3130-upstream,id=upstream1,bus=pcie.port1
    -device xio3130-downstream,id=downstream1_1,bus=upstream1,chassis=1,slot=2
    -device vfio-pci-nohotplug,host=0018:06:00.0,bus=downstream1_1,id=dev0
    -set device.dev0.pci-bars=bar0@0x48000000000,bar2@0x50000000000,bar4@0x58000000000

Example 2: Extending the PCIe MMIO aperture with highmem-mmio-base
(and highmem-mmio-size), using a mix of emulated and VFIO devices.

    -machine virt,...,highmem-mmio-base=0x400000000000,highmem-mmio-size=0x400000000000
    -device e1000,netdev=net0,bus=pcie.0,pci-bars=bar0@0x10000000
    -device pcie-root-port,id=pcie.port9,bus=pcie.9,chassis=4,io-reserve=0,fixed-bar=on
    -device x3130-upstream,id=upstream9,bus=pcie.port9
    -device xio3130-downstream,id=downstream9_1,bus=upstream9,chassis=4,slot=1
    -device vfio-pci,host=0012:03:00.1,bus=downstream9_1,id=nic1,pci-bars=bar0@0x7000c0000000
    -device xio3130-downstream,id=downstream9_2,bus=upstream9,chassis=4,slot=2
    -device vfio-pci-nohotplug,host=0019:06:00.0,bus=downstream9_2,id=dev1
    -set device.dev1.pci-bars=bar0@0x6d4000000000,bar2@0x6d8000000000,bar4@0x6e0000000000

The pci-bars= property is generic and may be used with any PCI endpoint
device, including emulated devices and devices with 32-bit memory BARs.

The optional highmem-mmio-base machine property allows the operator to
reposition the high PCIe MMIO window so that the requested BAR
addresses can be placed within the configured address space.

Limitations:
- I/O BARs and the expansion ROM BAR are not covered by pci-bars=.
- This has only been tested on the AArch64 virt machine.
- SR-IOV VF BARs are not covered.

Testing:

The series was tested on the AArch64 virt machine using multiple PCI
topologies, including emulated devices, VFIO passthrough devices, PCIe
switches, and fixed BAR root ports.

A git branch with this series applied is available at:
https://github.com/tdavenvidia/upstream-qemu/tree/RFC-v2-fixed-bar-upstream

[1] RFC v1 upstream thread:
https://lore.kernel.org/qemu-devel/20260508183717.193630-1-tdave@nvidia.com/

Tushar Dave (5):
  hw/pci: add fixed-bar and pci-bars properties
  pci: add validation for fixed BAR configuration
  pci: add fixed BAR fw_cfg blob export
  hw/arm/virt: export fixed BAR metadata via fw_cfg
  hw/arm/virt: add highmem-mmio-base property

 hw/arm/virt.c                   |  78 ++++++++-
 hw/pci-bridge/pcie_root_port.c  |   1 +
 hw/pci/meson.build              |   2 +
 hw/pci/pci-fixed-bar-blob.c     | 291 ++++++++++++++++++++++++++++++++
 hw/pci/pci-fixed-bar-validate.c | 279 ++++++++++++++++++++++++++++++
 hw/pci/pci-fixed-bar-validate.h |  21 +++
 hw/pci/pci-fixed-bar.h          |  60 +++++++
 hw/pci/pci.c                    | 129 ++++++++++++++
 include/hw/pci/pci_device.h     |  10 ++
 include/hw/pci/pcie_port.h      |   1 +
 10 files changed, 871 insertions(+), 1 deletion(-)
 create mode 100644 hw/pci/pci-fixed-bar-blob.c
 create mode 100644 hw/pci/pci-fixed-bar-validate.c
 create mode 100644 hw/pci/pci-fixed-bar-validate.h
 create mode 100644 hw/pci/pci-fixed-bar.h

-- 
2.34.1



^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2026-09-04 13:46 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-27  0:40 [RFC PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
2026-08-27  0:40 ` [RFC PATCH v2 1/5] hw/pci: add fixed-bar and pci-bars properties Tushar Dave
2026-08-27  0:40 ` [RFC PATCH v2 2/5] pci: add validation for fixed BAR configuration Tushar Dave
2026-08-27  0:40 ` [RFC PATCH v2 3/5] pci: add fixed BAR fw_cfg blob export Tushar Dave
2026-08-27  0:40 ` [RFC PATCH v2 4/5] hw/arm/virt: export fixed BAR metadata via fw_cfg Tushar Dave
2026-08-27  0:40 ` [RFC PATCH v2 5/5] hw/arm/virt: add highmem-mmio-base property Tushar Dave
2026-08-27  7:18 ` [RFC PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Gerd Hoffmann
2026-08-27 13:47   ` Alex Williamson
2026-08-27 14:38     ` [edk2-devel] " Ard Biesheuvel
2026-08-28 15:49       ` Tushar Dave
2026-08-31 13:42         ` Gerd Hoffmann
2026-09-01 21:58           ` Tushar Dave
2026-08-31 13:24     ` Gerd Hoffmann
2026-09-01 22:27       ` Tushar Dave
2026-09-02  6:15         ` Gerd Hoffmann
2026-09-02 15:24           ` Alex Williamson
2026-09-03  9:34             ` Gerd Hoffmann
2026-09-02 16:30           ` Tushar Dave
2026-09-03  9:47             ` Gerd Hoffmann
2026-09-04 13:46               ` Tushar Dave

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox