* [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement
@ 2026-11-02 18:12 Tushar Dave
2026-11-02 18:12 ` [PATCH v2 1/5] hw/pci: introduce PCI fixed BAR properties Tushar Dave
` (6 more replies)
0 siblings, 7 replies; 11+ messages in thread
From: Tushar Dave @ 2026-11-02 18:12 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-arm, peter.maydell, mst, marcel, devel, alwilliamson, jgg,
skolothumtho, ardb, kraxel, tdave
This patch series introduces a mechanism to specify Guest Physical Addresses
(GPAs) for PCI BARs, allowing explicit placement of guest MMIO BAR addresses
to match host physical addresses for assigned devices.
On some platforms, P2P DMA between devices in the same IOMMU group is required
for the desired data path and performance. The PCI fabric ACS is configured
to permit direct P2P without routing traffic through the host bridge.
To support this multi-device IOMMU group P2P scenario in virtualization, the
VM may need to preserve the host PCI BAR addresses in the guest.
A new fixed-bar-N property on a PCI endpoint device specifies the required
guest address for BAR N (N = 0 through 5):
-device some-device,fixed-bar-0=<addr>[,fixed-bar-1=<addr>]...
Once any fixed-bar-N property is set on a device, every memory BAR on that
device must have a fixed address. Under a PCIe root port, fixed BAR placement
is all-or-nothing within that root port's subtree. If one endpoint below the
root port has a fixed BAR, every other endpoint with a memory BAR below that
same root port must also have all of its memory BARs fixed.
QEMU validates the configuration when the machine is done initializing. Each
address must be aligned to its BAR size, fall within the appropriate PCIe MMIO
aperture, and not overlap any other fixed BAR. QEMU aborts on any validation
failure. QEMU performs neither PCI enumeration nor BAR allocation; the
validated configuration is exported to firmware through the "etc/fixed-bars"
fw_cfg blob, which carries each PCI function's vendor/device identification,
firmware device path, and, for fixed devices, the configured address, size,
and type of each BAR. The blob format is documented in
docs/specs/pci-fixed-bars.rst.
The fixed-bar-N property and its validation are generic and apply to any PCI
device. A machine type opts in by setting the new pci_fixed_bar_supported
MachineClass flag. Machine types that leave it unset reject fixed-bar-N. This
series enables it on the virt machine only, by setting the flag and wiring up
the fw_cfg export. Other machine types would need equivalent wiring to expose
the same mechanism to firmware.
On the firmware side, the existing IncompatiblePciDeviceSupportDxe driver is
extended to consume this blob. During device discovery, its CheckDevice()
implementation returns ACPI address descriptors with _MIF|_MAF set for the
fixed BARs found in the blob. PciBusDxe's resource allocator already supports
fixed BARs, including sizing and pinning the bridge windows above them. It now
treats a BAR as fixed when the descriptor carries _MIF|_MAF, and programs the
fixed address into the BAR register.
Fixed BAR placement requires the guest to preserve the PCI resource
configuration established by firmware. This is requested through the PCI boot
configuration preservation mechanism using ACPI _DSM function 5. Fixed BAR
configuration is rejected when firmware is not loaded or ACPI is disabled.
Example 1: VFIO passthrough device with fixed BARs and the PCIe MMIO
aperture adjusted to accommodate the fixed addresses.
-machine virt,...,highmem-mmio-base=4T,highmem-mmio-size=4T
-device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0
-device x3130-upstream,id=upstream1,bus=pcie.port1
-device xio3130-downstream,id=downstream1_1,bus=upstream1,chassis=1,slot=2
-device vfio-pci-nohotplug,host=0018:06:00.0,bus=downstream1_1,id=dev0,\
fixed-bar-0=0x48000000000,fixed-bar-2=0x50000000000,fixed-bar-4=0x58000000000
Example 2: VFIO and emulated devices under the same root port with fixed BARs
and the PCIe MMIO aperture adjusted using highmem-mmio-base.
-machine virt,...,highmem-mmio-base=64T,highmem-mmio-size=64T
-device pxb-pcie,id=pcie.1,bus_nr=1,bus=pcie.0
-device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0
-device x3130-upstream,id=upstream1,bus=pcie.port1
-device xio3130-downstream,id=downstream1_1,bus=upstream1,chassis=1,slot=1
-device vfio-pci-nohotplug,host=0018:06:00.0,bus=downstream1_1,rombar=0,id=dev0,iommufd=iommufd0,\
fixed-bar-0=0x6b4000000000,fixed-bar-2=0x6b8000000000,fixed-bar-4=0x6c0000000000
-device xio3130-downstream,id=downstream1_2,bus=upstream1,chassis=1,slot=2
-device vfio-pci,host=0012:03:00.1,bus=downstream1_2,id=nic1,iommufd=iommufd0,fixed-bar-0=0x6c8000000000
-device xio3130-downstream,id=downstream1_3,bus=upstream1,chassis=1,slot=3
-device e1000,netdev=net0,bus=downstream1_3,id=nic_emu,fixed-bar-0=0x20000000
Limitations:
* I/O BARs and the expansion ROM BAR are not covered by fixed-bar-N.
* SR-IOV VF BARs are not covered.
Testing:
The series was tested on the AArch64 virt machine with a range of PCIe
configurations, including emulated and VFIO passthrough devices, PCIe
switches, single and multiple fixed-BAR endpoints behind a shared root port,
devices attached directly to the host bridge, and endpoints behind nested
bridges. All rejection paths were also exercised, with the corresponding
error messages verified. These included unsupported machine types,
hot-plugged devices, non-existent BARs, I/O BARs, unaligned, overlapping, or
out-of-aperture BARs, incomplete BAR configurations, highmem-mmio=off,
missing firmware, and highmem-mmio-base values that cause address-space
overflow.
Changes since v1:
* Add a pci_fixed_bar_supported MachineClass flag; fixed-bar-N is rejected
on machine types that do not set it. virt sets it.
* Keep the fixed BAR claims in FixedBarsInfo instead of a static array, and
report validation failures through Error ** to the machine code instead of
calling exit(1).
* Allow a 64-bit non-prefetchable BAR in either aperture when the device is
attached to the host bridge; behind a PCI bridge it must lie below 4 GiB.
A 64-bit prefetchable BAR must still lie in the 64-bit aperture.
* Treat the high PCIe MMIO aperture as absent with highmem-mmio=off, and
reject fixed BAR configuration unless firmware is loaded and ACPI is
enabled.
* Check highmem-mmio-base + highmem-mmio-size against the PA space without
overflow, and document highmem-mmio-base in docs/system/arm/virt.rst.
* Document the etc/fixed-bars format in docs/specs/pci-fixed-bars.rst, and
note that the firmware device path is carried for future matching.
* Move pci-fixed-bar.h to include/hw/pci/.
v1:
https://lore.kernel.org/qemu-arm/20260916185554.917526-1-tdave@nvidia.com/
A git branch with this series applied is available at:
https://github.com/tdavenvidia/upstream-qemu/tree/Upstream-fixed-bar-Oct-29-V2
The related EDK2 change is available at:
https://github.com/tianocore/edk2/pull/13183
Tushar Dave (5):
hw/pci: introduce PCI fixed BAR properties
hw/pci: add validation for fixed BAR configuration
hw/pci: add fixed BAR fw_cfg blob export
hw/arm/virt: export fixed BAR metadata via fw_cfg
hw/arm/virt: add highmem-mmio-base property
docs/specs/index.rst | 1 +
docs/specs/pci-fixed-bars.rst | 75 +++++++
docs/system/arm/virt.rst | 4 +
hw/arm/virt.c | 82 ++++++-
hw/pci/meson.build | 2 +
hw/pci/pci-fixed-bar-blob.c | 308 +++++++++++++++++++++++++++
hw/pci/pci-fixed-bar-validate.c | 365 ++++++++++++++++++++++++++++++++
hw/pci/pci-fixed-bar-validate.h | 22 ++
hw/pci/pci.c | 71 +++++++
include/hw/core/boards.h | 3 +
include/hw/pci/pci-fixed-bar.h | 63 ++++++
include/hw/pci/pci_device.h | 7 +
12 files changed, 1002 insertions(+), 1 deletion(-)
create mode 100644 docs/specs/pci-fixed-bars.rst
create mode 100644 hw/pci/pci-fixed-bar-blob.c
create mode 100644 hw/pci/pci-fixed-bar-validate.c
create mode 100644 hw/pci/pci-fixed-bar-validate.h
create mode 100644 include/hw/pci/pci-fixed-bar.h
--
2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v2 1/5] hw/pci: introduce PCI fixed BAR properties
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
@ 2026-11-02 18:12 ` Tushar Dave
2026-11-02 18:12 ` [PATCH v2 2/5] hw/pci: add validation for fixed BAR configuration Tushar Dave
` (5 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Tushar Dave @ 2026-11-02 18:12 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-arm, peter.maydell, mst, marcel, devel, alwilliamson, jgg,
skolothumtho, ardb, kraxel, tdave
Add fixed-bar-N properties to PCIDevice to allow the guest address
of an individual PCI BAR to be explicitly specified. The properties
are available for PCI BARs 0 through 5 and accept QEMU size
values, for example fixed-bar-0=0x40000000 or fixed-bar-0=64G.
e.g.
-device some-device,fixed-bar-0=<addr>[,fixed-bar-1=<addr>]...
During realize, validate that each specified BAR exists and is a
memory BAR, and reject fixed BAR configuration for hot-plugged
devices.
The fixed BAR properties are only supported on machine types that
implement fixed BAR placement. Add the `pci_fixed_bar_supported`
MachineClass flag to advertise this support, and reject fixed BAR
properties on machine types that do not set it.
Signed-off-by: Tushar Dave <tdave@nvidia.com>
---
hw/pci/pci.c | 71 +++++++++++++++++++++++++++++++++++++
include/hw/core/boards.h | 3 ++
include/hw/pci/pci_device.h | 7 ++++
3 files changed, 81 insertions(+)
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 2d2fe02c68..e9ae0d5648 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -89,6 +89,18 @@ static const Property pci_props[] = {
DEFINE_PROP_STRING("romfile", PCIDevice, romfile),
DEFINE_PROP_UINT32("romsize", PCIDevice, romsize, UINT32_MAX),
DEFINE_PROP_INT32("rombar", PCIDevice, rom_bar, -1),
+ DEFINE_PROP_SIZE("fixed-bar-0", PCIDevice, fixed_bar_addr[0],
+ PCI_BAR_UNMAPPED),
+ DEFINE_PROP_SIZE("fixed-bar-1", PCIDevice, fixed_bar_addr[1],
+ PCI_BAR_UNMAPPED),
+ DEFINE_PROP_SIZE("fixed-bar-2", PCIDevice, fixed_bar_addr[2],
+ PCI_BAR_UNMAPPED),
+ DEFINE_PROP_SIZE("fixed-bar-3", PCIDevice, fixed_bar_addr[3],
+ PCI_BAR_UNMAPPED),
+ DEFINE_PROP_SIZE("fixed-bar-4", PCIDevice, fixed_bar_addr[4],
+ PCI_BAR_UNMAPPED),
+ DEFINE_PROP_SIZE("fixed-bar-5", PCIDevice, fixed_bar_addr[5],
+ PCI_BAR_UNMAPPED),
DEFINE_PROP_BIT("multifunction", PCIDevice, cap_present,
QEMU_PCI_CAP_MULTIFUNCTION_BITNR, false),
DEFINE_PROP_BIT("x-pcie-lnksta-dllla", PCIDevice, cap_present,
@@ -228,6 +240,58 @@ static void pci_bus_unrealize(BusState *qbus)
vmstate_unregister(NULL, &vmstate_pcibus, bus);
}
+static bool pci_dev_has_fixed_bar(PCIDevice *pci_dev)
+{
+ int i;
+
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ if (pci_dev->fixed_bar_addr[i] != PCI_BAR_UNMAPPED) {
+ return true;
+ }
+ }
+ return false;
+}
+
+static void pci_check_fixed_bars(PCIDevice *pci_dev, Error **errp)
+{
+ PCIIORegion *r;
+ int i;
+
+ if (!pci_dev_has_fixed_bar(pci_dev)) {
+ return;
+ }
+
+ if (!MACHINE_GET_CLASS(qdev_get_machine())->pci_fixed_bar_supported) {
+ error_setg(errp, "fixed BAR placement is not supported by this "
+ "machine type");
+ return;
+ }
+
+ if (DEVICE(pci_dev)->hotplugged) {
+ error_setg(errp, "fixed BAR placement is not supported on "
+ "hot-plugged devices");
+ return;
+ }
+
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ if (pci_dev->fixed_bar_addr[i] == PCI_BAR_UNMAPPED) {
+ continue;
+ }
+
+ r = &pci_dev->io_regions[i];
+ if (!r->size) {
+ error_setg(errp, "fixed-bar-%d: bar%d does not exist on %s",
+ i, i, pci_dev->name);
+ return;
+ }
+ if (r->type & PCI_BASE_ADDRESS_SPACE_IO) {
+ error_setg(errp, "fixed-bar-%d: bar%d on %s is an I/O BAR, "
+ "not a memory BAR", i, i, pci_dev->name);
+ return;
+ }
+ }
+}
+
static int pcibus_num(PCIBus *bus)
{
if (pci_bus_is_root(bus)) {
@@ -2399,6 +2463,13 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp)
is_default_rom = true;
}
+ pci_check_fixed_bars(pci_dev, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ pci_qdev_unrealize(DEVICE(pci_dev));
+ return;
+ }
+
pci_add_option_rom(pci_dev, is_default_rom, &local_err);
if (local_err) {
error_propagate(errp, local_err);
diff --git a/include/hw/core/boards.h b/include/hw/core/boards.h
index d0e4ad9820..8092634eb3 100644
--- a/include/hw/core/boards.h
+++ b/include/hw/core/boards.h
@@ -242,6 +242,8 @@ typedef struct {
* is not needed.
* @numa_mem_supported:
* true if '--numa node.mem' option is supported and false otherwise
+ * @pci_fixed_bar_supported:
+ * true if the machine supports fixed PCI BAR placement and false otherwise
* @hotplug_allowed:
* If the hook is provided, then it'll be called for each device
* hotplug to check whether the device hotplug is allowed. Return
@@ -317,6 +319,7 @@ struct MachineClass {
bool ignore_boot_device_suffixes;
bool nvdimm_supported;
bool numa_mem_supported;
+ bool pci_fixed_bar_supported;
bool auto_enable_numa;
bool cpu_cluster_has_numa_boundary;
SMPCompatProps smp_props;
diff --git a/include/hw/pci/pci_device.h b/include/hw/pci/pci_device.h
index 5cac6e1688..db4a003ae3 100644
--- a/include/hw/pci/pci_device.h
+++ b/include/hw/pci/pci_device.h
@@ -187,6 +187,13 @@ struct PCIDevice {
uint32_t max_bounce_buffer_size;
char *sriov_pf;
+
+ /*
+ * fixed_bar_addr[N] holds the guest address configured via the
+ * fixed-bar-N property for memory BAR N. PCI_BAR_UNMAPPED means
+ * the BAR's address is not fixed.
+ */
+ uint64_t fixed_bar_addr[PCI_NUM_REGIONS - 1];
};
static inline int pci_intx(PCIDevice *pci_dev)
--
2.34.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 2/5] hw/pci: add validation for fixed BAR configuration
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
2026-11-02 18:12 ` [PATCH v2 1/5] hw/pci: introduce PCI fixed BAR properties Tushar Dave
@ 2026-11-02 18:12 ` Tushar Dave
2026-11-02 18:12 ` [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export Tushar Dave
` (4 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Tushar Dave @ 2026-11-02 18:12 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-arm, peter.maydell, mst, marcel, devel, alwilliamson, jgg,
skolothumtho, ardb, kraxel, tdave
Validate the fixed BAR configuration specified through the
fixed-bar-N properties on PCI endpoint devices.
Configuring any fixed-bar-N property on an endpoint device requires
every memory BAR on that device to have a fixed address. Validate
that each assigned address:
- is aligned to its BAR size;
- is within the appropriate PCIe MMIO aperture; and
- does not overlap any other fixed BAR.
A 32-bit BAR must lie in the 32-bit aperture. A prefetchable 64-bit BAR
must lie in the 64-bit aperture so that all prefetchable BARs can be
accommodated in a single 64-bit prefetchable bridge window. A
non-prefetchable 64-bit BAR may lie in either aperture when attached
directly to the host bridge, but must lie below 4GiB when placed behind
a PCI bridge.
Under a PCIe topology with a root port, fixed BAR placement is
all-or-nothing within that root port's subtree. If any one endpoint
device below the root port has a fixed BAR, every other endpoint
device below that same root port with a memory BAR must have all of
its memory BARs fixed too.
Endpoint devices with no root port in their ancestry, such as those
attached directly to the host bridge, are not required to match
their sibling devices. One such device may use fixed-bar-N for all
of its memory BARs while another sibling device on the same host
bridge uses none at all. Per-device completeness still applies
regardless of ancestry. Once any fixed-bar-N property is set on a
device, every memory BAR on that device must be fixed.
Report a validation failure through the Error ** argument and leave
it to the caller to decide how to handle it.
Signed-off-by: Tushar Dave <tdave@nvidia.com>
---
hw/pci/meson.build | 1 +
hw/pci/pci-fixed-bar-validate.c | 365 ++++++++++++++++++++++++++++++++
hw/pci/pci-fixed-bar-validate.h | 22 ++
3 files changed, 388 insertions(+)
create mode 100644 hw/pci/pci-fixed-bar-validate.c
create mode 100644 hw/pci/pci-fixed-bar-validate.h
diff --git a/hw/pci/meson.build b/hw/pci/meson.build
index 984c7327e5..150085c997 100644
--- a/hw/pci/meson.build
+++ b/hw/pci/meson.build
@@ -19,6 +19,7 @@ endif
pci_ss.add(files('pcie.c', 'pcie_aer.c'))
pci_ss.add(files('pcie_doe.c'))
system_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pcie_port.c', 'pcie_host.c'))
+system_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pci-fixed-bar-validate.c'))
system_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
stub_ss.add(files('pci-stub.c'))
diff --git a/hw/pci/pci-fixed-bar-validate.c b/hw/pci/pci-fixed-bar-validate.c
new file mode 100644
index 0000000000..522d879113
--- /dev/null
+++ b/hw/pci/pci-fixed-bar-validate.c
@@ -0,0 +1,365 @@
+/*
+ * Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * BAR address validation for fixed-BAR placement.
+ *
+ * Written by Tushar Dave
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/pci/pci.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pci_host.h"
+#include "hw/pci/pcie_port.h"
+#include "qapi/error.h"
+#include "qemu/range.h"
+#include "qemu/units.h"
+#include "pci-internal.h"
+#include "pci-fixed-bar-validate.h"
+
+/*
+ * Claimed BAR ranges — detect inter-device and inter-hierarchy overlaps
+ * across all fixed BARs system-wide.
+ */
+
+typedef struct {
+ uint64_t start;
+ uint64_t end;
+ const char *owner; /* device name, for error messages */
+ int bar;
+} FixedClaim;
+
+static void fixed_claims_init(FixedBarsInfo *info)
+{
+ info->claims = g_array_new(false, true, sizeof(FixedClaim));
+}
+
+static void fixed_claims_free(FixedBarsInfo *info)
+{
+ g_array_free(info->claims, true);
+ info->claims = NULL;
+}
+
+static bool fixed_claims_overlap(FixedBarsInfo *info,
+ uint64_t start, uint64_t end,
+ const char **owner_out, int *bar_out,
+ uint64_t *start_out, uint64_t *end_out)
+{
+ FixedClaim *c;
+ guint i;
+
+ for (i = 0; i < info->claims->len; i++) {
+ c = &g_array_index(info->claims, FixedClaim, i);
+ if (ranges_overlap(start, end - start + 1,
+ c->start, c->end - c->start + 1)) {
+ *owner_out = c->owner;
+ *bar_out = c->bar;
+ *start_out = c->start;
+ *end_out = c->end;
+ return true;
+ }
+ }
+ return false;
+}
+
+static void fixed_claims_add(FixedBarsInfo *info,
+ uint64_t start, uint64_t end,
+ const char *owner, int bar)
+{
+ FixedClaim cl;
+
+ cl.start = start;
+ cl.end = end;
+ cl.owner = owner;
+ cl.bar = bar;
+ g_array_append_val(info->claims, cl);
+}
+
+/*
+ * Check that the BAR [start..end] fits in the window [base..limit].
+ * Both ranges are inclusive.
+ */
+static bool bar_in_window(uint64_t start, uint64_t end,
+ uint64_t base, uint64_t limit)
+{
+ return start >= base && end <= limit;
+}
+
+static bool validate_bars(PCIDevice *pdev, FixedBarsInfo *info,
+ bool is_fixed_subtree, Error **errp)
+{
+ const char *devname = DEVICE(pdev)->id ? DEVICE(pdev)->id : pdev->name;
+ const char *overlap_owner;
+ bool has_fixed_bars = false;
+ bool is_64bit;
+ uint64_t addr, end, overlap_start, overlap_end;
+ PCIIORegion *r;
+ int overlap_bar;
+ int i;
+
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ if (pdev->fixed_bar_addr[i] != PCI_BAR_UNMAPPED) {
+ has_fixed_bars = true;
+ break;
+ }
+ }
+
+ if (is_fixed_subtree && !has_fixed_bars) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] under fixed-bar root "
+ "port has memory BARs but no fixed-bar-N= specified",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn));
+ return false;
+ }
+
+ if (!has_fixed_bars) {
+ return true;
+ }
+
+ /* Completeness: every memory BAR must have an address. */
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ r = &pdev->io_regions[i];
+ if (!r->size || (r->type & PCI_BASE_ADDRESS_SPACE_IO)) {
+ continue;
+ }
+ if (pdev->fixed_bar_addr[i] == PCI_BAR_UNMAPPED) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "missing a fixed-bar-N= value",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn), i);
+ return false;
+ }
+ }
+
+ /* Per-BAR checks: alignment, MMIO window, overlap. */
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ r = &pdev->io_regions[i];
+ if (!r->size || (r->type & PCI_BASE_ADDRESS_SPACE_IO)) {
+ continue;
+ }
+
+ is_64bit = !!(r->type & PCI_BASE_ADDRESS_MEM_TYPE_64);
+ addr = (uint64_t)pdev->fixed_bar_addr[i];
+
+ if (r->size - 1 > UINT64_MAX - addr) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "addr=0x%"PRIx64" + size=0x%"PRIx64" overflows",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn),
+ i, addr, r->size);
+ return false;
+ }
+ end = addr + r->size - 1;
+
+ if (is_64bit && !(r->type & PCI_BASE_ADDRESS_MEM_PREFETCH) &&
+ !pci_bus_is_root(pci_get_bus(pdev)) && end >= 4 * GiB) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d is a "
+ "non-prefetchable 64-bit BAR behind a PCI bridge "
+ "and cannot be placed above 4GB",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn), i);
+ return false;
+ }
+
+ if (addr & (r->size - 1)) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "addr=0x%"PRIx64" not aligned to size=0x%"PRIx64,
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn),
+ i, addr, r->size);
+ return false;
+ }
+
+ /*
+ * A 32-bit BAR must fit in the 32-bit window; a 64-bit BAR in either,
+ * except a prefetchable one, which must be above 4GiB.
+ */
+ if (is_64bit && (r->type & PCI_BASE_ADDRESS_MEM_PREFETCH)) {
+ if (!bar_in_window(addr, end,
+ info->mmio64_base, info->mmio64_limit)) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "[0x%"PRIx64"..0x%"PRIx64"] outside the 64-bit "
+ "window [0x%"PRIx64"..0x%"PRIx64"]",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn),
+ i, addr, end,
+ info->mmio64_base, info->mmio64_limit);
+ return false;
+ }
+ } else if (is_64bit) {
+ if (!bar_in_window(addr, end,
+ info->mmio32_base, info->mmio32_limit) &&
+ !bar_in_window(addr, end,
+ info->mmio64_base, info->mmio64_limit)) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "[0x%"PRIx64"..0x%"PRIx64"] outside both the "
+ "32-bit window [0x%"PRIx64"..0x%"PRIx64"] and "
+ "the 64-bit window [0x%"PRIx64"..0x%"PRIx64"]",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn),
+ i, addr, end,
+ info->mmio32_base, info->mmio32_limit,
+ info->mmio64_base, info->mmio64_limit);
+ return false;
+ }
+ } else if (!bar_in_window(addr, end,
+ info->mmio32_base, info->mmio32_limit)) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "[0x%"PRIx64"..0x%"PRIx64"] outside the 32-bit "
+ "window [0x%"PRIx64"..0x%"PRIx64"]",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn),
+ i, addr, end,
+ info->mmio32_base, info->mmio32_limit);
+ return false;
+ }
+
+ if (fixed_claims_overlap(info, addr, end, &overlap_owner,
+ &overlap_bar, &overlap_start, &overlap_end)) {
+ error_setg(errp, "fixed-bar: %s [%02x:%02x.%x] BAR%d "
+ "[0x%"PRIx64"..0x%"PRIx64"] overlaps %s BAR%d "
+ "[0x%"PRIx64"..0x%"PRIx64"]",
+ devname, pci_dev_bus_num(pdev),
+ PCI_SLOT(pdev->devfn), PCI_FUNC(pdev->devfn),
+ i, addr, end, overlap_owner, overlap_bar,
+ overlap_start, overlap_end);
+ return false;
+ }
+
+ fixed_claims_add(info, addr, end, devname, i);
+ }
+
+ info->any_fixed = true;
+ return true;
+}
+
+typedef struct {
+ FixedBarsInfo *info;
+ bool is_fixed_subtree;
+ Error **errp;
+} ScanState;
+
+static void scan_bus(PCIBus *bus, ScanState *state);
+
+static void mark_if_fixed(PCIBus *bus, PCIDevice *pdev, void *opaque)
+{
+ bool *found = opaque;
+ PCIBus *sec;
+ int i;
+
+ if (*found) {
+ return;
+ }
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ if (pdev->fixed_bar_addr[i] != PCI_BAR_UNMAPPED) {
+ *found = true;
+ return;
+ }
+ }
+ if (object_dynamic_cast(OBJECT(pdev), TYPE_PCI_BRIDGE)) {
+ sec = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
+ if (sec) {
+ pci_for_each_device_under_bus(sec, mark_if_fixed, found);
+ }
+ }
+}
+
+/*
+ * True if any device in rp_bus's subtree (the bus behind a root port)
+ * has a fixed BAR.
+ */
+static bool root_port_subtree_has_fixed_bar(PCIBus *rp_bus)
+{
+ bool found = false;
+
+ pci_for_each_device_under_bus(rp_bus, mark_if_fixed, &found);
+ return found;
+}
+
+static void scan_bus_device(PCIBus *bus, PCIDevice *pdev, void *opaque)
+{
+ ScanState *state = opaque;
+ bool has_mem_bar = false;
+ PCIIORegion *r;
+ PCIBus *sec;
+ int i;
+
+ if (*state->errp) {
+ return;
+ }
+
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ r = &pdev->io_regions[i];
+ if (r->size && !(r->type & PCI_BASE_ADDRESS_SPACE_IO)) {
+ has_mem_bar = true;
+ break;
+ }
+ }
+
+ if (has_mem_bar &&
+ !validate_bars(pdev, state->info, state->is_fixed_subtree,
+ state->errp)) {
+ return;
+ }
+
+ if (!object_dynamic_cast(OBJECT(pdev), TYPE_PCI_BRIDGE)) {
+ return;
+ }
+ sec = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
+ if (!sec) {
+ return;
+ }
+
+ if (object_dynamic_cast(OBJECT(pdev), TYPE_PCIE_ROOT_PORT)) {
+ ScanState child = {
+ .info = state->info,
+ .is_fixed_subtree = root_port_subtree_has_fixed_bar(sec),
+ .errp = state->errp,
+ };
+ scan_bus(sec, &child);
+ } else {
+ scan_bus(sec, state);
+ }
+}
+
+static void scan_bus(PCIBus *bus, ScanState *state)
+{
+ pci_for_each_device_under_bus(bus, scan_bus_device, state);
+}
+
+static void scan_all_host_bridges(FixedBarsInfo *info, Error **errp)
+{
+ PCIHostState *hb;
+ ScanState state = {
+ .info = info,
+ .is_fixed_subtree = false,
+ .errp = errp,
+ };
+
+ fixed_claims_init(info);
+ QLIST_FOREACH(hb, &pci_host_bridges, next) {
+ if (hb->bus) {
+ scan_bus(hb->bus, &state);
+ }
+ }
+ fixed_claims_free(info);
+}
+
+/*
+ * fixed_bars_validate - scan all PCI devices, validate fixed BAR addresses.
+ *
+ * @info: carries MMIO window bounds for validation.
+ * @errp: set on a validation error.
+ *
+ * Returns true if every BAR validated, false with @errp set otherwise.
+ * Whether any fixed BAR device was found is left in info->any_fixed.
+ */
+bool fixed_bars_validate(FixedBarsInfo *info, Error **errp)
+{
+ ERRP_GUARD();
+
+ info->any_fixed = false;
+ scan_all_host_bridges(info, errp);
+ return !*errp;
+}
diff --git a/hw/pci/pci-fixed-bar-validate.h b/hw/pci/pci-fixed-bar-validate.h
new file mode 100644
index 0000000000..f4996f27d9
--- /dev/null
+++ b/hw/pci/pci-fixed-bar-validate.h
@@ -0,0 +1,22 @@
+/*
+ * Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ *
+ * Written by Tushar Dave
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#ifndef HW_PCI_FIXED_BAR_VALIDATE_H
+#define HW_PCI_FIXED_BAR_VALIDATE_H
+
+typedef struct {
+ bool any_fixed;
+ uint64_t mmio32_base;
+ uint64_t mmio32_limit;
+ uint64_t mmio64_base;
+ uint64_t mmio64_limit;
+ GArray *claims;
+} FixedBarsInfo;
+
+bool fixed_bars_validate(FixedBarsInfo *info, Error **errp);
+
+#endif /* HW_PCI_FIXED_BAR_VALIDATE_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
2026-11-02 18:12 ` [PATCH v2 1/5] hw/pci: introduce PCI fixed BAR properties Tushar Dave
2026-11-02 18:12 ` [PATCH v2 2/5] hw/pci: add validation for fixed BAR configuration Tushar Dave
@ 2026-11-02 18:12 ` Tushar Dave
2026-11-04 9:59 ` Ard Biesheuvel
2026-11-02 18:12 ` [PATCH v2 4/5] hw/arm/virt: export fixed BAR metadata via fw_cfg Tushar Dave
` (3 subsequent siblings)
6 siblings, 1 reply; 11+ messages in thread
From: Tushar Dave @ 2026-11-02 18:12 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-arm, peter.maydell, mst, marcel, devel, alwilliamson, jgg,
skolothumtho, ardb, kraxel, tdave
Introduce a fixed BAR fw_cfg blob exporter that serializes PCI
device and BAR information into the "etc/fixed-bars" fw_cfg file.
The blob contains an entry for every PCI function with memory BARs.
Each entry carries the device's vendor and device ID, whether it
has a fixed BAR configuration, and its firmware device path. Fixed
BAR entries also include the configured BAR address, size, and type
information for each BAR.
Document the format in docs/specs/pci-fixed-bars.rst.
Signed-off-by: Tushar Dave <tdave@nvidia.com>
---
docs/specs/index.rst | 1 +
docs/specs/pci-fixed-bars.rst | 75 ++++++++
hw/pci/meson.build | 1 +
hw/pci/pci-fixed-bar-blob.c | 308 +++++++++++++++++++++++++++++++++
include/hw/pci/pci-fixed-bar.h | 63 +++++++
5 files changed, 448 insertions(+)
create mode 100644 docs/specs/pci-fixed-bars.rst
create mode 100644 hw/pci/pci-fixed-bar-blob.c
create mode 100644 include/hw/pci/pci-fixed-bar.h
diff --git a/docs/specs/index.rst b/docs/specs/index.rst
index 44e108db02..15cdf4698b 100644
--- a/docs/specs/index.rst
+++ b/docs/specs/index.rst
@@ -11,6 +11,7 @@ guest hardware that is specific to QEMU.
pci-ids
pci-serial
pci-testdev
+ pci-fixed-bars
ppc-xive
ppc-spapr-xive
ppc-spapr-numa
diff --git a/docs/specs/pci-fixed-bars.rst b/docs/specs/pci-fixed-bars.rst
new file mode 100644
index 0000000000..708566550e
--- /dev/null
+++ b/docs/specs/pci-fixed-bars.rst
@@ -0,0 +1,75 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+..
+ Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+
+PCI fixed BAR fw_cfg
+====================
+
+A PCI device can be given a fixed guest address for each of its memory BARs
+with the ``fixed-bar-N`` device properties, where ``N`` is the BAR number,
+for example ``-device e1000,fixed-bar-0=0x20000000``. A machine that
+supports this validates the addresses and passes them to the firmware in the
+``etc/fixed-bars`` fw_cfg file, so that the firmware can place each BAR at
+the requested address instead of assigning one.
+
+The file is present only when at least one device has a fixed BAR. All
+fields are little-endian and the structures are packed.
+
+Layout
+------
+
+The file is a header followed by one device record for every PCI function
+that has a memory BAR, fixed or not. A device record is followed by one BAR
+record for each of its fixed BARs. A device with any fixed BAR has all of its
+memory BARs fixed, so it has a BAR record for each of them.
+
+The device records are in the order in which the firmware discovers the
+devices. Host bridges come in ascending bus number order, the devices of a
+bus in devfn order, and the devices behind a bridge follow the bridge.
+
+Header
+~~~~~~
+
+====== ======= ====================================================
+Offset Size Description
+====== ======= ====================================================
+0 4 ``version``, currently 1
+4 4 ``num_devices``, the number of device records
+====== ======= ====================================================
+
+Device record
+~~~~~~~~~~~~~
+
+====== ======= ====================================================
+Offset Size Description
+====== ======= ====================================================
+0 2 ``vendor_id``
+2 2 ``device_id``
+4 1 ``dev_flags``, bit 0 is set if the device has fixed BARs
+5 1 ``rp_bus``, the primary bus number of the root port the
+ device is under, or the bus number of its host bridge
+ if it is not under a root port
+6 1 ``num_bars``, the number of BAR records that follow,
+ 0 if the device has no fixed BARs
+7 1 reserved, 0
+8 128 ``fw_path``, the firmware device path of the device,
+ NUL-terminated
+====== ======= ====================================================
+
+BAR record
+~~~~~~~~~~
+
+====== ======= ====================================================
+Offset Size Description
+====== ======= ====================================================
+0 1 ``bar``, the BAR number
+1 3 reserved, 0
+4 4 ``flags``, bit 0 indicates the BAR type (0 for 32-bit,
+ 1 for 64-bit), and bit 1 indicates whether the BAR is
+ prefetchable (0 for non-prefetchable, 1 for
+ prefetchable). All other bits are reserved and must
+ be zero.
+8 8 ``address``, the fixed guest address of the BAR
+16 8 ``size``, the size of the BAR
+====== ======= ====================================================
diff --git a/hw/pci/meson.build b/hw/pci/meson.build
index 150085c997..b81cc056c0 100644
--- a/hw/pci/meson.build
+++ b/hw/pci/meson.build
@@ -20,6 +20,7 @@ pci_ss.add(files('pcie.c', 'pcie_aer.c'))
pci_ss.add(files('pcie_doe.c'))
system_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pcie_port.c', 'pcie_host.c'))
system_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pci-fixed-bar-validate.c'))
+system_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pci-fixed-bar-blob.c'))
system_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
stub_ss.add(files('pci-stub.c'))
diff --git a/hw/pci/pci-fixed-bar-blob.c b/hw/pci/pci-fixed-bar-blob.c
new file mode 100644
index 0000000000..dc94debfad
--- /dev/null
+++ b/hw/pci/pci-fixed-bar-blob.c
@@ -0,0 +1,308 @@
+/*
+ * Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * Write validated fixed-BAR device info to the etc/fixed-bars fw_cfg blob.
+ *
+ * Written by Tushar Dave
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/nvram/fw_cfg.h"
+#include "hw/pci/pci.h"
+#include "hw/pci/pci_bridge.h"
+#include "hw/pci/pci_host.h"
+#include "hw/pci/pcie_port.h"
+#include "hw/core/qdev.h"
+#include "qemu/error-report.h"
+#include "pci-internal.h"
+#include "hw/pci/pci-fixed-bar.h"
+#include "pci-fixed-bar-validate.h"
+
+typedef struct {
+ uint8_t bar;
+ uint32_t flags;
+ uint64_t addr;
+ uint64_t size;
+} BarInfo;
+
+/* One PCI function entry to be written to the blob. */
+typedef struct {
+ uint8_t rp_bus; /* primary bus of the root port this device is under */
+ uint16_t vendor_id;
+ uint16_t device_id;
+ bool is_fixed;
+ char fw_path[QEMU_FIXED_BARS_MAX_PATH];
+ GArray *bars; /* array of BarInfo */
+} DeviceInfo;
+
+typedef struct {
+ GArray *devices;
+ PCIBus *root_bus;
+} CollectCtx;
+
+static void collect_device(PCIBus *bus, PCIDevice *pdev, void *opaque);
+
+/* Visit every device on this bus, in devfn order. */
+static void collect_bus(PCIBus *bus, void *opaque)
+{
+ pci_for_each_device_under_bus(bus, collect_device, opaque);
+}
+
+/*
+ * Primary bus number of the nearest TYPE_PCIE_ROOT_PORT ancestor of bus
+ * — i.e. the bus the root port device itself lives on, not its
+ * secondary bus. Falls back to the host bridge's own bus number if bus
+ * isn't under any root port (e.g. a device directly on the host
+ * bridge's primary bus).
+ */
+static uint8_t find_root_port_bus(PCIBus *bus, PCIBus *host_bus)
+{
+ PCIDevice *parent;
+
+ while (bus) {
+ parent = bus->parent_dev;
+ if (!parent) {
+ break;
+ }
+ if (object_dynamic_cast(OBJECT(parent), TYPE_PCIE_ROOT_PORT)) {
+ return pci_bus_num(pci_get_bus(parent));
+ }
+ bus = pci_get_bus(parent);
+ }
+ return pci_bus_num(host_bus);
+}
+
+static void collect_device(PCIBus *bus, PCIDevice *pdev, void *opaque)
+{
+ CollectCtx *ctx = opaque;
+ bool has_mem_bar = false;
+ bool is_64bit, is_pref;
+ DeviceInfo dev;
+ PCIIORegion *r;
+ PCIBus *sec;
+ BarInfo bi;
+ int i;
+
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ r = &pdev->io_regions[i];
+ if (r->size && !(r->type & PCI_BASE_ADDRESS_SPACE_IO)) {
+ has_mem_bar = true;
+ break;
+ }
+ }
+
+ /*
+ * Every device with a memory BAR gets an entry, fixed or not — not
+ * just ones with fixed-bar-N= set. This preserves the exact device
+ * sequence CheckDevice() needs for positional VID:DID matching (see
+ * collect_all_host_bridges()); only fixed devices carry BAR records
+ * (dev.bars stays empty, dev.is_fixed false, otherwise).
+ */
+ if (has_mem_bar) {
+ g_autofree char *fw_path = qdev_get_fw_dev_path(DEVICE(pdev));
+
+ memset(&dev, 0, sizeof(dev));
+ dev.rp_bus = find_root_port_bus(bus, ctx->root_bus);
+ dev.vendor_id = pci_get_word(pdev->config + PCI_VENDOR_ID);
+ dev.device_id = pci_get_word(pdev->config + PCI_DEVICE_ID);
+ dev.is_fixed = false;
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ if (pdev->fixed_bar_addr[i] != PCI_BAR_UNMAPPED) {
+ dev.is_fixed = true;
+ break;
+ }
+ }
+ if (fw_path) {
+ g_strlcpy(dev.fw_path, fw_path, sizeof(dev.fw_path));
+ }
+ dev.bars = g_array_new(false, true, sizeof(BarInfo));
+
+ if (dev.is_fixed) {
+ for (i = 0; i < PCI_NUM_REGIONS - 1; i++) {
+ r = &pdev->io_regions[i];
+ if (!r->size || (r->type & PCI_BASE_ADDRESS_SPACE_IO)) {
+ continue;
+ }
+ is_64bit = !!(r->type & PCI_BASE_ADDRESS_MEM_TYPE_64);
+ is_pref = !!(r->type & PCI_BASE_ADDRESS_MEM_PREFETCH);
+
+ memset(&bi, 0, sizeof(bi));
+ bi.bar = i;
+ bi.addr = (uint64_t)pdev->fixed_bar_addr[i];
+ bi.size = r->size;
+ bi.flags = (is_64bit ? QEMU_FIXED_BAR_F_MEM64 : 0) |
+ (is_pref ? QEMU_FIXED_BAR_F_PREF : 0);
+ g_array_append_val(dev.bars, bi);
+ }
+ }
+ g_array_append_val(ctx->devices, dev);
+ }
+
+ if (!object_dynamic_cast(OBJECT(pdev), TYPE_PCI_BRIDGE)) {
+ return;
+ }
+ sec = pci_bridge_get_sec_bus(PCI_BRIDGE(pdev));
+ if (sec) {
+ collect_bus(sec, opaque);
+ }
+}
+
+static gint cmp_host_bus_num(gconstpointer a, gconstpointer b)
+{
+ PCIHostState *ha = *(PCIHostState **)a;
+ PCIHostState *hb = *(PCIHostState **)b;
+ return (gint)pci_bus_num(ha->bus) - (gint)pci_bus_num(hb->bus);
+}
+
+/*
+ * EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL's CheckDevice interface
+ * (UEFI PI Spec 1.2 Vol 5) only passes VendorId/DeviceId/RevisionId/
+ * SubsystemVendorId/SubsystemDeviceId, no BDF — so in EDK2 we can't
+ * match a blob entry to a specific device directly using BDF. To work
+ * around that, we prepare the blob entries in the same order PciBusDxe
+ * discovers devices, so matching by VID:DID inherently works. Each entry
+ * also carries the firmware device path. The CheckDevice interface cannot
+ * use it today, but it could if the interface is extended.
+ */
+static void collect_all_host_bridges(CollectCtx *ctx)
+{
+ PCIHostState *hb;
+ GPtrArray *sorted;
+ guint i;
+
+ sorted = g_ptr_array_new();
+ QLIST_FOREACH(hb, &pci_host_bridges, next) {
+ if (hb->bus) {
+ g_ptr_array_add(sorted, hb);
+ }
+ }
+ g_ptr_array_sort(sorted, cmp_host_bus_num);
+
+ for (i = 0; i < sorted->len; i++) {
+ hb = g_ptr_array_index(sorted, i);
+ ctx->root_bus = hb->bus;
+ collect_bus(hb->bus, ctx);
+ }
+
+ g_ptr_array_free(sorted, true);
+}
+
+static uint8_t *create_blob(CollectCtx *ctx, size_t *blob_size)
+{
+ QemuFixedBarsDevice *dout;
+ QemuFixedBarsBar *bout;
+ QemuFixedBarsHdr *hdr;
+ DeviceInfo *dev;
+ uint8_t *blob;
+ uint8_t *ptr;
+ BarInfo *bi;
+ size_t sz;
+ guint d, b;
+
+ sz = sizeof(QemuFixedBarsHdr);
+ for (d = 0; d < ctx->devices->len; d++) {
+ dev = &g_array_index(ctx->devices, DeviceInfo, d);
+ sz += sizeof(QemuFixedBarsDevice);
+ sz += dev->bars->len * sizeof(QemuFixedBarsBar);
+ }
+
+ blob = g_malloc0(sz);
+ hdr = (QemuFixedBarsHdr *)blob;
+ hdr->version = cpu_to_le32(QEMU_FIXED_BARS_VERSION);
+ hdr->num_devices = cpu_to_le32(ctx->devices->len);
+
+ ptr = blob + sizeof(*hdr);
+ for (d = 0; d < ctx->devices->len; d++) {
+ dev = &g_array_index(ctx->devices, DeviceInfo, d);
+ dout = (QemuFixedBarsDevice *)ptr;
+
+ dout->vendor_id = cpu_to_le16(dev->vendor_id);
+ dout->device_id = cpu_to_le16(dev->device_id);
+ dout->dev_flags = dev->is_fixed ? QEMU_FIXED_BARS_DEV_F_FIXED : 0;
+ dout->rp_bus = dev->rp_bus;
+ dout->num_bars = dev->bars->len;
+ dout->reserved = 0;
+ memcpy(dout->fw_path, dev->fw_path, sizeof(dout->fw_path));
+ ptr += sizeof(QemuFixedBarsDevice);
+
+ for (b = 0; b < dev->bars->len; b++) {
+ bi = &g_array_index(dev->bars, BarInfo, b);
+ bout = (QemuFixedBarsBar *)ptr;
+
+ bout->bar = bi->bar;
+ bout->flags = cpu_to_le32(bi->flags);
+ bout->address = cpu_to_le64(bi->addr);
+ bout->size = cpu_to_le64(bi->size);
+ ptr += sizeof(QemuFixedBarsBar);
+ }
+ }
+
+ *blob_size = sz;
+ return blob;
+}
+
+/*
+ * fixed_bars_write_blob - validate fixed BAR addresses and write fw_cfg blob.
+ *
+ * @fw_cfg: fw_cfg state to write the etc/fixed-bars blob into.
+ * @mmio32_base: base of the machine's 32-bit PCIe MMIO aperture.
+ * @mmio32_size: size of the machine's 32-bit PCIe MMIO aperture.
+ * @mmio64_base: base of the machine's 64-bit PCIe MMIO aperture.
+ * @mmio64_size: size of the machine's 64-bit PCIe MMIO aperture, or 0
+ * if the machine has none.
+ * @errp: set if the apertures or any BAR address fail validation.
+ *
+ * Returns true if any fixed BAR device was found and the blob was
+ * written, false if there was nothing to do or on error.
+ */
+bool fixed_bars_write_blob(FWCfgState *fw_cfg,
+ uint64_t mmio32_base,
+ uint64_t mmio32_size,
+ uint64_t mmio64_base,
+ uint64_t mmio64_size,
+ Error **errp)
+{
+ FixedBarsInfo info;
+ CollectCtx ctx;
+ uint8_t *blob;
+ size_t blob_size;
+ guint d;
+
+ if (!mmio32_size) {
+ error_setg(errp, "fixed-bar: zero-size 32-bit PCIe MMIO aperture");
+ return false;
+ }
+ if (mmio32_size - 1 > UINT64_MAX - mmio32_base ||
+ (mmio64_size && mmio64_size - 1 > UINT64_MAX - mmio64_base)) {
+ error_setg(errp, "fixed-bar: PCIe MMIO aperture base+size overflows "
+ "(32-bit base=0x%"PRIx64" size=0x%"PRIx64", "
+ "64-bit base=0x%"PRIx64" size=0x%"PRIx64")",
+ mmio32_base, mmio32_size, mmio64_base, mmio64_size);
+ return false;
+ }
+
+ info.mmio32_base = mmio32_base;
+ info.mmio32_limit = mmio32_base + mmio32_size - 1;
+ /* An absent 64-bit aperture is an empty window, no BAR fits in it. */
+ info.mmio64_base = mmio64_size ? mmio64_base : 1;
+ info.mmio64_limit = mmio64_size ? mmio64_base + mmio64_size - 1 : 0;
+
+ if (!fixed_bars_validate(&info, errp) || !info.any_fixed) {
+ return false;
+ }
+
+ ctx.devices = g_array_new(false, true, sizeof(DeviceInfo));
+ ctx.root_bus = NULL;
+ collect_all_host_bridges(&ctx);
+
+ blob = create_blob(&ctx, &blob_size);
+ fw_cfg_add_file(fw_cfg, FW_CFG_FIXED_BARS, blob, blob_size);
+
+ for (d = 0; d < ctx.devices->len; d++) {
+ g_array_free(g_array_index(ctx.devices, DeviceInfo, d).bars, true);
+ }
+ g_array_free(ctx.devices, true);
+
+ return true;
+}
diff --git a/include/hw/pci/pci-fixed-bar.h b/include/hw/pci/pci-fixed-bar.h
new file mode 100644
index 0000000000..2c5af536dc
--- /dev/null
+++ b/include/hw/pci/pci-fixed-bar.h
@@ -0,0 +1,63 @@
+/*
+ * Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * Fixed-BAR — QEMU side declarations.
+ *
+ * Written by Tushar Dave
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#ifndef HW_PCI_FIXED_BAR_H
+#define HW_PCI_FIXED_BAR_H
+
+#include "hw/nvram/fw_cfg.h"
+
+#define FW_CFG_FIXED_BARS "etc/fixed-bars"
+#define QEMU_FIXED_BARS_VERSION 1
+#define QEMU_FIXED_BARS_MAX_PATH 128
+
+/*
+ * On-wire layout (little-endian, packed):
+ *
+ * QemuFixedBarsHdr
+ *
+ * For each device (num_devices total):
+ * QemuFixedBarsDevice -- device header
+ * QemuFixedBarsBar[num_bars] -- one record per BAR (FIXED only)
+ */
+
+typedef struct {
+ uint32_t version; /* QEMU_FIXED_BARS_VERSION */
+ uint32_t num_devices;
+} QEMU_PACKED QemuFixedBarsHdr; /* 8 bytes */
+
+typedef struct {
+ uint16_t vendor_id;
+ uint16_t device_id;
+ uint8_t dev_flags; /* QEMU_FIXED_BARS_DEV_F_* */
+ uint8_t rp_bus; /* primary bus of the root port this device is under */
+ uint8_t num_bars;
+ uint8_t reserved;
+ /* qdev_get_fw_dev_path(), NUL-terminated */
+ char fw_path[QEMU_FIXED_BARS_MAX_PATH];
+} QEMU_PACKED QemuFixedBarsDevice; /* 8 + QEMU_FIXED_BARS_MAX_PATH bytes */
+
+typedef struct {
+ uint8_t bar;
+ uint8_t reserved[3];
+ uint32_t flags; /* QEMU_FIXED_BAR_F_* */
+ uint64_t address;
+ uint64_t size;
+} QEMU_PACKED QemuFixedBarsBar; /* 24 bytes */
+
+#define QEMU_FIXED_BAR_F_MEM64 (1u << 0)
+#define QEMU_FIXED_BAR_F_PREF (1u << 1)
+#define QEMU_FIXED_BARS_DEV_F_FIXED (1u << 0)
+
+bool fixed_bars_write_blob(FWCfgState *fw_cfg,
+ uint64_t mmio32_base,
+ uint64_t mmio32_size,
+ uint64_t mmio64_base,
+ uint64_t mmio64_size,
+ Error **errp);
+
+#endif /* HW_PCI_FIXED_BAR_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 4/5] hw/arm/virt: export fixed BAR metadata via fw_cfg
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
` (2 preceding siblings ...)
2026-11-02 18:12 ` [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export Tushar Dave
@ 2026-11-02 18:12 ` Tushar Dave
2026-11-02 18:12 ` [PATCH v2 5/5] hw/arm/virt: add highmem-mmio-base property Tushar Dave
` (2 subsequent siblings)
6 siblings, 0 replies; 11+ messages in thread
From: Tushar Dave @ 2026-11-02 18:12 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-arm, peter.maydell, mst, marcel, devel, alwilliamson, jgg,
skolothumtho, ardb, kraxel, tdave
Export fixed BAR metadata through the "etc/fixed-bars" fw_cfg file
and set pci_fixed_bar_supported to allow the fixed-bar-N properties
on the virt machine.
Fixed BAR placement requires the guest to preserve the PCI resource
configuration established by firmware, preventing the fixed BAR
addresses from being reassigned during guest boot. ACPI provides
this through the PCI boot configuration preservation mechanism
(_DSM function 5).
Reject fixed BAR configuration if firmware is not loaded or ACPI
is disabled, since PCI boot configuration preservation cannot
otherwise be guaranteed.
Signed-off-by: Tushar Dave <tdave@nvidia.com>
---
hw/arm/virt.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 3ab3ad8c3b..4bacb57770 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -94,6 +94,7 @@
#include "hw/cxl/cxl_host.h"
#include "qemu/guest-random.h"
#include "hw/watchdog/sbsa_gwdt.h"
+#include "hw/pci/pci-fixed-bar.h"
static GlobalProperty arm_virt_compat_defaults[] = {
{ TYPE_VIRTIO_IOMMU_PCI, "aw-bits", "48" },
@@ -2435,6 +2436,23 @@ void virt_machine_done(Notifier *notifier, void *data)
vms->memmap[VIRT_PLATFORM_BUS].size,
vms->irqmap[VIRT_PLATFORM_BUS]);
}
+ if (fixed_bars_write_blob(vms->fw_cfg,
+ vms->memmap[VIRT_PCIE_MMIO].base,
+ vms->memmap[VIRT_PCIE_MMIO].size,
+ vms->memmap[VIRT_HIGH_PCIE_MMIO].base,
+ vms->highmem_mmio ?
+ vms->memmap[VIRT_HIGH_PCIE_MMIO].size : 0,
+ &error_fatal)) {
+ if (!(vms->bootinfo.firmware_loaded && virt_is_acpi_enabled(vms))) {
+ error_report("fixed-bar: fixed BAR placement requires firmware "
+ "and ACPI to preserve the PCI boot configuration, "
+ "otherwise these BAR assignments could be "
+ "reassigned later with no protection");
+ exit(1);
+ }
+ vms->pci_preserve_config = true;
+ }
+
if (arm_load_dtb(info->dtb_start, info, info->dtb_limit, as, ms, cpu) < 0) {
exit(1);
}
@@ -4192,6 +4210,7 @@ static void virt_machine_class_init(ObjectClass *oc, const void *data)
hc->unplug_request = virt_machine_device_unplug_request_cb;
hc->unplug = virt_machine_device_unplug_cb;
mc->nvdimm_supported = true;
+ mc->pci_fixed_bar_supported = true;
mc->smp_props.clusters_supported = true;
/* Supported caches */
--
2.34.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH v2 5/5] hw/arm/virt: add highmem-mmio-base property
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
` (3 preceding siblings ...)
2026-11-02 18:12 ` [PATCH v2 4/5] hw/arm/virt: export fixed BAR metadata via fw_cfg Tushar Dave
@ 2026-11-02 18:12 ` Tushar Dave
2026-11-04 9:46 ` [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Gerd Hoffmann
2026-11-04 13:11 ` Michael S. Tsirkin
6 siblings, 0 replies; 11+ messages in thread
From: Tushar Dave @ 2026-11-02 18:12 UTC (permalink / raw)
To: qemu-devel
Cc: qemu-arm, peter.maydell, mst, marcel, devel, alwilliamson, jgg,
skolothumtho, ardb, kraxel, tdave
Add a highmem-mmio-base property to override the base of the virt
machine's high PCIe MMIO window, alongside the existing
highmem-mmio-size:
-machine virt,highmem-mmio-base=<addr>,highmem-mmio-size=<size>
Some PCI devices require MMIO resources to be placed within a
specific address range that cannot be satisfied by the default
placement. The configured base is validated to be non-zero, aligned
to highmem-mmio-size, non-overlapping with the existing high memory
layout, and within the configured physical address space.
Document the property in docs/system/arm/virt.rst.
Signed-off-by: Tushar Dave <tdave@nvidia.com>
---
docs/system/arm/virt.rst | 4 +++
hw/arm/virt.c | 63 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst
index 5f6dd17978..7ae51bee13 100644
--- a/docs/system/arm/virt.rst
+++ b/docs/system/arm/virt.rst
@@ -152,6 +152,10 @@ highmem-mmio-size
Set the high memory region size for PCI MMIO. Must be a power of 2 and
greater than or equal to the default size (512G).
+highmem-mmio-base
+ Set the high memory region base for PCI MMIO. Must be non-zero and
+ aligned to ``highmem-mmio-size``.
+
gic-version
Specify the version of the Generic Interrupt Controller (GIC) to provide.
Valid values are:
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 4bacb57770..c04aadb92f 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2507,14 +2507,40 @@ static void virt_set_high_memmap(VirtMachineState *vms,
hwaddr base, int pa_bits)
{
hwaddr region_base, region_size;
+ hwaddr pa_limit = BIT_ULL(pa_bits);
bool *region_enabled, fits;
int i;
for (i = VIRT_LOWMEMMAP_LAST; i < ARRAY_SIZE(extended_memmap); i++) {
region_enabled = virt_get_high_memmap_enabled(vms, i);
- region_base = ROUND_UP(base, extended_memmap[i].size);
region_size = extended_memmap[i].size;
+ if (i == VIRT_HIGH_PCIE_MMIO && extended_memmap[i].base) {
+ region_base = extended_memmap[i].base;
+
+ if (region_base < base) {
+ error_report("highmem-mmio-base 0x%"PRIx64" overlaps prior "
+ "high memory layout (must be >= 0x%"PRIx64")",
+ region_base, base);
+ exit(1);
+ }
+ if (region_base % region_size != 0) {
+ error_report("highmem-mmio-base 0x%"PRIx64" must be "
+ "aligned to highmem-mmio-size 0x%"PRIx64,
+ region_base, region_size);
+ exit(1);
+ }
+ if (region_base >= pa_limit ||
+ region_size > pa_limit - region_base) {
+ error_report("highmem-mmio-base 0x%"PRIx64" + "
+ "highmem-mmio-size 0x%"PRIx64" exceeds %d-bit "
+ "PA space", region_base, region_size, pa_bits);
+ exit(1);
+ }
+ } else {
+ region_base = ROUND_UP(base, region_size);
+ }
+
vms->memmap[i].base = region_base;
vms->memmap[i].size = region_size;
@@ -3415,6 +3441,33 @@ static void virt_set_highmem_mmio_size(Object *obj, Visitor *v,
extended_memmap[VIRT_HIGH_PCIE_MMIO].size = size;
}
+static void virt_get_highmem_mmio_base(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ uint64_t base = extended_memmap[VIRT_HIGH_PCIE_MMIO].base;
+
+ visit_type_size(v, name, &base, errp);
+}
+
+static void virt_set_highmem_mmio_base(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ uint64_t base;
+
+ if (!visit_type_size(v, name, &base, errp)) {
+ return;
+ }
+
+ if (base == 0) {
+ error_setg(errp, "highmem-mmio-base cannot be 0");
+ return;
+ }
+
+ extended_memmap[VIRT_HIGH_PCIE_MMIO].base = base;
+}
+
static char *virt_get_msi(Object *obj, Error **errp)
{
VirtMachineState *vms = VIRT_MACHINE(obj);
@@ -4286,6 +4339,14 @@ static void virt_machine_class_init(ObjectClass *oc, const void *data)
"Set the high memory region size "
"for PCI MMIO");
+ object_class_property_add(oc, "highmem-mmio-base", "size",
+ virt_get_highmem_mmio_base,
+ virt_set_highmem_mmio_base,
+ NULL, NULL);
+ object_class_property_set_description(oc, "highmem-mmio-base",
+ "Set the high memory region base "
+ "for PCI MMIO");
+
object_class_property_add(oc, "virtio-mmio-transports", "uint8",
virt_get_virtio_transports,
virt_set_virtio_transports,
--
2.34.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
` (4 preceding siblings ...)
2026-11-02 18:12 ` [PATCH v2 5/5] hw/arm/virt: add highmem-mmio-base property Tushar Dave
@ 2026-11-04 9:46 ` Gerd Hoffmann
2026-11-04 13:11 ` Michael S. Tsirkin
6 siblings, 0 replies; 11+ messages in thread
From: Gerd Hoffmann @ 2026-11-04 9:46 UTC (permalink / raw)
To: Tushar Dave
Cc: qemu-devel, qemu-arm, peter.maydell, mst, marcel, devel,
alwilliamson, jgg, skolothumtho, ardb
On Mon, Nov 02, 2026 at 12:12:36PM -0600, Tushar Dave wrote:
> This patch series introduces a mechanism to specify Guest Physical Addresses
> (GPAs) for PCI BARs, allowing explicit placement of guest MMIO BAR addresses
> to match host physical addresses for assigned devices.
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export
2026-11-02 18:12 ` [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export Tushar Dave
@ 2026-11-04 9:59 ` Ard Biesheuvel
2026-11-04 12:11 ` Michael S. Tsirkin
2026-11-04 23:05 ` Tushar Dave
0 siblings, 2 replies; 11+ messages in thread
From: Ard Biesheuvel @ 2026-11-04 9:59 UTC (permalink / raw)
To: Tushar Dave, qemu-devel
Cc: qemu-arm, peter.maydell, Michael S. Tsirkin, marcel,
devel@edk2.groups.io, alwilliamson, Jason Gunthorpe,
Shameer Kolothum Thodi, Gerd Hoffmann
On Mon, 2 Nov 2026, at 19:12, Tushar Dave wrote:
> Introduce a fixed BAR fw_cfg blob exporter that serializes PCI
> device and BAR information into the "etc/fixed-bars" fw_cfg file.
>
> The blob contains an entry for every PCI function with memory BARs.
> Each entry carries the device's vendor and device ID, whether it
> has a fixed BAR configuration, and its firmware device path. Fixed
> BAR entries also include the configured BAR address, size, and type
> information for each BAR.
>
> Document the format in docs/specs/pci-fixed-bars.rst.
>
> Signed-off-by: Tushar Dave <tdave@nvidia.com>
> ---
> docs/specs/index.rst | 1 +
> docs/specs/pci-fixed-bars.rst | 75 ++++++++
> hw/pci/meson.build | 1 +
> hw/pci/pci-fixed-bar-blob.c | 308 +++++++++++++++++++++++++++++++++
> include/hw/pci/pci-fixed-bar.h | 63 +++++++
> 5 files changed, 448 insertions(+)
> create mode 100644 docs/specs/pci-fixed-bars.rst
> create mode 100644 hw/pci/pci-fixed-bar-blob.c
> create mode 100644 include/hw/pci/pci-fixed-bar.h
>
...
> diff --git a/hw/pci/pci-fixed-bar-blob.c b/hw/pci/pci-fixed-bar-blob.c
> new file mode 100644
> index 0000000000..dc94debfad
> --- /dev/null
> +++ b/hw/pci/pci-fixed-bar-blob.c
...
> +/*
> + * EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL's CheckDevice interface
> + * (UEFI PI Spec 1.2 Vol 5) only passes VendorId/DeviceId/RevisionId/
> + * SubsystemVendorId/SubsystemDeviceId, no BDF — so in EDK2 we can't
> + * match a blob entry to a specific device directly using BDF. To work
> + * around that, we prepare the blob entries in the same order PciBusDxe
> + * discovers devices, so matching by VID:DID inherently works.
This sounds rather fragile.
Could you elaborate on what order this is, and why it is reasonable to
rely on that approach being stable?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export
2026-11-04 9:59 ` Ard Biesheuvel
@ 2026-11-04 12:11 ` Michael S. Tsirkin
2026-11-04 23:05 ` Tushar Dave
1 sibling, 0 replies; 11+ messages in thread
From: Michael S. Tsirkin @ 2026-11-04 12:11 UTC (permalink / raw)
To: Ard Biesheuvel
Cc: Tushar Dave, qemu-devel, qemu-arm, peter.maydell, marcel,
devel@edk2.groups.io, alwilliamson, Jason Gunthorpe,
Shameer Kolothum Thodi, Gerd Hoffmann
On Wed, Nov 04, 2026 at 10:59:47AM +0100, Ard Biesheuvel wrote:
>
>
> On Mon, 2 Nov 2026, at 19:12, Tushar Dave wrote:
> > Introduce a fixed BAR fw_cfg blob exporter that serializes PCI
> > device and BAR information into the "etc/fixed-bars" fw_cfg file.
> >
> > The blob contains an entry for every PCI function with memory BARs.
> > Each entry carries the device's vendor and device ID, whether it
> > has a fixed BAR configuration, and its firmware device path. Fixed
> > BAR entries also include the configured BAR address, size, and type
> > information for each BAR.
> >
> > Document the format in docs/specs/pci-fixed-bars.rst.
> >
> > Signed-off-by: Tushar Dave <tdave@nvidia.com>
> > ---
> > docs/specs/index.rst | 1 +
> > docs/specs/pci-fixed-bars.rst | 75 ++++++++
> > hw/pci/meson.build | 1 +
> > hw/pci/pci-fixed-bar-blob.c | 308 +++++++++++++++++++++++++++++++++
> > include/hw/pci/pci-fixed-bar.h | 63 +++++++
> > 5 files changed, 448 insertions(+)
> > create mode 100644 docs/specs/pci-fixed-bars.rst
> > create mode 100644 hw/pci/pci-fixed-bar-blob.c
> > create mode 100644 include/hw/pci/pci-fixed-bar.h
> >
> ...
> > diff --git a/hw/pci/pci-fixed-bar-blob.c b/hw/pci/pci-fixed-bar-blob.c
> > new file mode 100644
> > index 0000000000..dc94debfad
> > --- /dev/null
> > +++ b/hw/pci/pci-fixed-bar-blob.c
> ...
> > +/*
> > + * EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL's CheckDevice interface
> > + * (UEFI PI Spec 1.2 Vol 5) only passes VendorId/DeviceId/RevisionId/
> > + * SubsystemVendorId/SubsystemDeviceId, no BDF — so in EDK2 we can't
> > + * match a blob entry to a specific device directly using BDF. To work
> > + * around that, we prepare the blob entries in the same order PciBusDxe
> > + * discovers devices, so matching by VID:DID inherently works.
>
> This sounds rather fragile.
>
> Could you elaborate on what order this is, and why it is reasonable to
> rely on that approach being stable?
>
>
Also, bus numbers can be changed by guest, and I don't get why
would you want to report devices without a fixed bar.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
` (5 preceding siblings ...)
2026-11-04 9:46 ` [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Gerd Hoffmann
@ 2026-11-04 13:11 ` Michael S. Tsirkin
6 siblings, 0 replies; 11+ messages in thread
From: Michael S. Tsirkin @ 2026-11-04 13:11 UTC (permalink / raw)
To: Tushar Dave
Cc: qemu-devel, qemu-arm, peter.maydell, marcel, devel, alwilliamson,
jgg, skolothumtho, ardb, kraxel
On Mon, Nov 02, 2026 at 12:12:36PM -0600, Tushar Dave wrote:
> This patch series introduces a mechanism to specify Guest Physical Addresses
> (GPAs) for PCI BARs, allowing explicit placement of guest MMIO BAR addresses
> to match host physical addresses for assigned devices.
My understanding is that all this does it pass a property to
the firmware asking it to set the bar to a given value?
There are two standard ways to do this:
- EA capability in pci config space
- fixed BAR _DSM in ACPI
If none of these are appropriate, pls document why.
> On some platforms, P2P DMA between devices in the same IOMMU group is required
> for the desired data path and performance. The PCI fabric ACS is configured
> to permit direct P2P without routing traffic through the host bridge.
>
> To support this multi-device IOMMU group P2P scenario in virtualization, the
> VM may need to preserve the host PCI BAR addresses in the guest.
>
> A new fixed-bar-N property on a PCI endpoint device specifies the required
> guest address for BAR N (N = 0 through 5):
>
> -device some-device,fixed-bar-0=<addr>[,fixed-bar-1=<addr>]...
>
> Once any fixed-bar-N property is set on a device, every memory BAR on that
> device must have a fixed address. Under a PCIe root port, fixed BAR placement
> is all-or-nothing within that root port's subtree. If one endpoint below the
> root port has a fixed BAR, every other endpoint with a memory BAR below that
> same root port must also have all of its memory BARs fixed.
>
> QEMU validates the configuration when the machine is done initializing. Each
> address must be aligned to its BAR size, fall within the appropriate PCIe MMIO
> aperture, and not overlap any other fixed BAR. QEMU aborts on any validation
> failure. QEMU performs neither PCI enumeration nor BAR allocation; the
> validated configuration is exported to firmware through the "etc/fixed-bars"
> fw_cfg blob, which carries each PCI function's vendor/device identification,
> firmware device path, and, for fixed devices, the configured address, size,
> and type of each BAR. The blob format is documented in
> docs/specs/pci-fixed-bars.rst.
>
> The fixed-bar-N property and its validation are generic and apply to any PCI
> device. A machine type opts in by setting the new pci_fixed_bar_supported
> MachineClass flag. Machine types that leave it unset reject fixed-bar-N. This
> series enables it on the virt machine only, by setting the flag and wiring up
> the fw_cfg export. Other machine types would need equivalent wiring to expose
> the same mechanism to firmware.
>
> On the firmware side, the existing IncompatiblePciDeviceSupportDxe driver is
> extended to consume this blob. During device discovery, its CheckDevice()
> implementation returns ACPI address descriptors with _MIF|_MAF set for the
> fixed BARs found in the blob. PciBusDxe's resource allocator already supports
> fixed BARs, including sizing and pinning the bridge windows above them. It now
> treats a BAR as fixed when the descriptor carries _MIF|_MAF, and programs the
> fixed address into the BAR register.
>
> Fixed BAR placement requires the guest to preserve the PCI resource
> configuration established by firmware. This is requested through the PCI boot
> configuration preservation mechanism using ACPI _DSM function 5. Fixed BAR
> configuration is rejected when firmware is not loaded or ACPI is disabled.
>
> Example 1: VFIO passthrough device with fixed BARs and the PCIe MMIO
> aperture adjusted to accommodate the fixed addresses.
>
> -machine virt,...,highmem-mmio-base=4T,highmem-mmio-size=4T
> -device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0
> -device x3130-upstream,id=upstream1,bus=pcie.port1
> -device xio3130-downstream,id=downstream1_1,bus=upstream1,chassis=1,slot=2
> -device vfio-pci-nohotplug,host=0018:06:00.0,bus=downstream1_1,id=dev0,\
> fixed-bar-0=0x48000000000,fixed-bar-2=0x50000000000,fixed-bar-4=0x58000000000
>
> Example 2: VFIO and emulated devices under the same root port with fixed BARs
> and the PCIe MMIO aperture adjusted using highmem-mmio-base.
>
> -machine virt,...,highmem-mmio-base=64T,highmem-mmio-size=64T
> -device pxb-pcie,id=pcie.1,bus_nr=1,bus=pcie.0
> -device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0
> -device x3130-upstream,id=upstream1,bus=pcie.port1
> -device xio3130-downstream,id=downstream1_1,bus=upstream1,chassis=1,slot=1
> -device vfio-pci-nohotplug,host=0018:06:00.0,bus=downstream1_1,rombar=0,id=dev0,iommufd=iommufd0,\
> fixed-bar-0=0x6b4000000000,fixed-bar-2=0x6b8000000000,fixed-bar-4=0x6c0000000000
> -device xio3130-downstream,id=downstream1_2,bus=upstream1,chassis=1,slot=2
> -device vfio-pci,host=0012:03:00.1,bus=downstream1_2,id=nic1,iommufd=iommufd0,fixed-bar-0=0x6c8000000000
> -device xio3130-downstream,id=downstream1_3,bus=upstream1,chassis=1,slot=3
> -device e1000,netdev=net0,bus=downstream1_3,id=nic_emu,fixed-bar-0=0x20000000
>
> Limitations:
>
> * I/O BARs and the expansion ROM BAR are not covered by fixed-bar-N.
> * SR-IOV VF BARs are not covered.
>
> Testing:
>
> The series was tested on the AArch64 virt machine with a range of PCIe
> configurations, including emulated and VFIO passthrough devices, PCIe
> switches, single and multiple fixed-BAR endpoints behind a shared root port,
> devices attached directly to the host bridge, and endpoints behind nested
> bridges. All rejection paths were also exercised, with the corresponding
> error messages verified. These included unsupported machine types,
> hot-plugged devices, non-existent BARs, I/O BARs, unaligned, overlapping, or
> out-of-aperture BARs, incomplete BAR configurations, highmem-mmio=off,
> missing firmware, and highmem-mmio-base values that cause address-space
> overflow.
>
> Changes since v1:
>
> * Add a pci_fixed_bar_supported MachineClass flag; fixed-bar-N is rejected
> on machine types that do not set it. virt sets it.
> * Keep the fixed BAR claims in FixedBarsInfo instead of a static array, and
> report validation failures through Error ** to the machine code instead of
> calling exit(1).
> * Allow a 64-bit non-prefetchable BAR in either aperture when the device is
> attached to the host bridge; behind a PCI bridge it must lie below 4 GiB.
> A 64-bit prefetchable BAR must still lie in the 64-bit aperture.
> * Treat the high PCIe MMIO aperture as absent with highmem-mmio=off, and
> reject fixed BAR configuration unless firmware is loaded and ACPI is
> enabled.
> * Check highmem-mmio-base + highmem-mmio-size against the PA space without
> overflow, and document highmem-mmio-base in docs/system/arm/virt.rst.
> * Document the etc/fixed-bars format in docs/specs/pci-fixed-bars.rst, and
> note that the firmware device path is carried for future matching.
> * Move pci-fixed-bar.h to include/hw/pci/.
>
> v1:
> https://lore.kernel.org/qemu-arm/20260916185554.917526-1-tdave@nvidia.com/
>
> A git branch with this series applied is available at:
> https://github.com/tdavenvidia/upstream-qemu/tree/Upstream-fixed-bar-Oct-29-V2
>
> The related EDK2 change is available at:
> https://github.com/tianocore/edk2/pull/13183
>
> Tushar Dave (5):
> hw/pci: introduce PCI fixed BAR properties
> hw/pci: add validation for fixed BAR configuration
> hw/pci: add fixed BAR fw_cfg blob export
> hw/arm/virt: export fixed BAR metadata via fw_cfg
> hw/arm/virt: add highmem-mmio-base property
>
> docs/specs/index.rst | 1 +
> docs/specs/pci-fixed-bars.rst | 75 +++++++
> docs/system/arm/virt.rst | 4 +
> hw/arm/virt.c | 82 ++++++-
> hw/pci/meson.build | 2 +
> hw/pci/pci-fixed-bar-blob.c | 308 +++++++++++++++++++++++++++
> hw/pci/pci-fixed-bar-validate.c | 365 ++++++++++++++++++++++++++++++++
> hw/pci/pci-fixed-bar-validate.h | 22 ++
> hw/pci/pci.c | 71 +++++++
> include/hw/core/boards.h | 3 +
> include/hw/pci/pci-fixed-bar.h | 63 ++++++
> include/hw/pci/pci_device.h | 7 +
> 12 files changed, 1002 insertions(+), 1 deletion(-)
> create mode 100644 docs/specs/pci-fixed-bars.rst
> create mode 100644 hw/pci/pci-fixed-bar-blob.c
> create mode 100644 hw/pci/pci-fixed-bar-validate.c
> create mode 100644 hw/pci/pci-fixed-bar-validate.h
> create mode 100644 include/hw/pci/pci-fixed-bar.h
>
> --
> 2.34.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export
2026-11-04 9:59 ` Ard Biesheuvel
2026-11-04 12:11 ` Michael S. Tsirkin
@ 2026-11-04 23:05 ` Tushar Dave
1 sibling, 0 replies; 11+ messages in thread
From: Tushar Dave @ 2026-11-04 23:05 UTC (permalink / raw)
To: Ard Biesheuvel, qemu-devel
Cc: qemu-arm, peter.maydell, Michael S. Tsirkin, marcel,
devel@edk2.groups.io, alwilliamson, Jason Gunthorpe,
Shameer Kolothum Thodi, Gerd Hoffmann
On 11/4/2026 3:59 AM, Ard Biesheuvel wrote:
>
>
> On Mon, 2 Nov 2026, at 19:12, Tushar Dave wrote:
>> Introduce a fixed BAR fw_cfg blob exporter that serializes PCI
>> device and BAR information into the "etc/fixed-bars" fw_cfg file.
>>
>> The blob contains an entry for every PCI function with memory BARs.
>> Each entry carries the device's vendor and device ID, whether it
>> has a fixed BAR configuration, and its firmware device path. Fixed
>> BAR entries also include the configured BAR address, size, and type
>> information for each BAR.
>>
>> Document the format in docs/specs/pci-fixed-bars.rst.
>>
>> Signed-off-by: Tushar Dave <tdave@nvidia.com>
>> ---
>> docs/specs/index.rst | 1 +
>> docs/specs/pci-fixed-bars.rst | 75 ++++++++
>> hw/pci/meson.build | 1 +
>> hw/pci/pci-fixed-bar-blob.c | 308 +++++++++++++++++++++++++++++++++
>> include/hw/pci/pci-fixed-bar.h | 63 +++++++
>> 5 files changed, 448 insertions(+)
>> create mode 100644 docs/specs/pci-fixed-bars.rst
>> create mode 100644 hw/pci/pci-fixed-bar-blob.c
>> create mode 100644 include/hw/pci/pci-fixed-bar.h
>>
> ...
>> diff --git a/hw/pci/pci-fixed-bar-blob.c b/hw/pci/pci-fixed-bar-blob.c
>> new file mode 100644
>> index 0000000000..dc94debfad
>> --- /dev/null
>> +++ b/hw/pci/pci-fixed-bar-blob.c
> ...
>> +/*
>> + * EFI_INCOMPATIBLE_PCI_DEVICE_SUPPORT_PROTOCOL's CheckDevice interface
>> + * (UEFI PI Spec 1.2 Vol 5) only passes VendorId/DeviceId/RevisionId/
>> + * SubsystemVendorId/SubsystemDeviceId, no BDF — so in EDK2 we can't
>> + * match a blob entry to a specific device directly using BDF. To work
>> + * around that, we prepare the blob entries in the same order PciBusDxe
>> + * discovers devices, so matching by VID:DID inherently works.
>
> This sounds rather fragile.
>
> Could you elaborate on what order this is, and why it is reasonable to
> rely on that approach being stable?
PciBusDxe discovers devices using the conventional PCI enumeration
order: devices and functions in ascending order on each bus, visiting
a bridge's secondary bus immediately after discovering the bridge
(depth-first), and root bridges one after another. CheckDevice() is
called for every function discovered, including bridges. QEMU exports
the entries in the same order. For each VendorId/DeviceId pair,
IncompatiblePciDeviceSupportDxe maintains a counter, and each
CheckDevice() call for a matching device consumes the next
corresponding entry from the blob. Accordingly, the blob contains an
entry for every function with a memory BAR, whether fixed or not, to
keep the matching simple and in sync.
I think relying on this order is reasonable because it is determined
by the PCI topology configured on the QEMU command line. Since devices
with fixed BARs cannot be hot-plugged, they are always present at
boot, and the resulting discovery order remains stable across boots
of a given VM.
-Tushar
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2026-11-04 23:06 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-02 18:12 [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Tushar Dave
2026-11-02 18:12 ` [PATCH v2 1/5] hw/pci: introduce PCI fixed BAR properties Tushar Dave
2026-11-02 18:12 ` [PATCH v2 2/5] hw/pci: add validation for fixed BAR configuration Tushar Dave
2026-11-02 18:12 ` [PATCH v2 3/5] hw/pci: add fixed BAR fw_cfg blob export Tushar Dave
2026-11-04 9:59 ` Ard Biesheuvel
2026-11-04 12:11 ` Michael S. Tsirkin
2026-11-04 23:05 ` Tushar Dave
2026-11-02 18:12 ` [PATCH v2 4/5] hw/arm/virt: export fixed BAR metadata via fw_cfg Tushar Dave
2026-11-02 18:12 ` [PATCH v2 5/5] hw/arm/virt: add highmem-mmio-base property Tushar Dave
2026-11-04 9:46 ` [PATCH v2 0/5] hw/pci, hw/arm/virt: fixed PCI BAR placement Gerd Hoffmann
2026-11-04 13:11 ` Michael S. Tsirkin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.