[PATCH 00/10] PCI: liveupdate: Keep preserved devices accessible across Live Update

[PATCH 00/10] PCI: liveupdate: Keep preserved devices accessible across Live Update

[David Matlack]

This series can be found on GitHub:

  https://github.com/dmatlack/linux/tree/liveupdate/pci/p2pdma/v1

This series builds on the PCI core support for Live Update [1] to allow
memory transactions targeting preserved PCI devices to proceed
uninterrupted across a Live Update. The base series allows preserved
devices to perform DMA to system memory across a Live Update, but the
next kernel still sizes, probes, and may reassign the BARs and bridge
windows of preserved devices during enumeration, which disables
decoding or moves address ranges out from under anything that is still
accessing them.

Use cases include:

  - P2PDMA between preserved devices, e.g. between devices assigned to
    the same VM via VFIO.

  - MMIO from Orphaned VMs [2], whose vCPUs keep running in guest mode
    on dedicated CPUs during kexec, and therefore keep accessing the
    BARs of devices assigned to them throughout the Live Update.

To support this, the BAR addresses and bridge windows of preserved
devices must not change across a Live Update, and the next kernel must
not disable decoding or forwarding on preserved devices or on the
bridges above them while enumerating them and assigning resources.

[1] https://lore.kernel.org/linux-pci/20261013162529.1928093-1-dmatlack@google.com/
[2] https://lore.kernel.org/kvm/20260920193650.3373435-1-pasha.tatashin@soleen.com/

Series Overview
---------------

This series implements the following:

  1. Avoid sizing the BARs of incoming preserved devices. Sizing a BAR
     requires disabling decoding. Instead, have the previous kernel
     pass the BAR masks (the values the BARs return when read after
     writing all 1s) and decode the BARs from those (patches 1-2).

  2. Avoid probing which windows incoming preserved bridges implement,
     which temporarily disables the prefetchable window. Instead, add a
     generic mechanism for passing the results of probes that require
     writing to the device, and use it to pass which windows each
     bridge implements (patches 3-4).

  3. Skip programming bridge windows that already contain the values
     being written, so that pci_setup_bridge() does not temporarily
     disable the windows of preserved bridges. This applies to all
     bridges, not only preserved ones (patch 5).

  4. Claim the BARs and bridge windows of incoming preserved devices,
     as programmed by the previous kernel, while scanning and before
     anything sizes or assigns resources, regardless of how the host
     bridge driver sets up resources. Panic if they cannot be claimed,
     since that means something else may be decoding the same address
     range (patch 6).

  5. Prevent the PCI core from assigning, releasing, or realigning the
     resources of incoming preserved devices (patches 7-9).

  6. Avoid enabling INTx on incoming preserved devices, both while
     probing whether INTx masking works and when the device is enabled
     (patch 10).

All new behavior applies only to incoming preserved devices, i.e. only
after a Live Update and only until pci_liveupdate_finish(), except for
patch 5, which avoids redundant writes for all bridges.

ABI Changes
-----------

This series uses the optional feature flags added to struct pci_ser in
v10 of the base series to append new fields to struct pci_dev_ser:

  - PCI_SER_OPTIONAL_BAR_MASKS: @bar_masks and @rom_mask.

  - PCI_SER_OPTIONAL_PROBED: @probed_supported and @probed, a bitmask
    of probe results (which windows a bridge implements, and whether
    INTx masking is broken).

The ABI only contains hardware state (register values and the results
of probing hardware), not kernel-internal state such as IORESOURCE_*
flags. If the previous kernel did not pass a field, the next kernel
falls back to sizing or probing the device as before.

Merge Plan
----------

This series applies on top of v10 of the PCI core support for Live
Update series [1], which is targeting Linux 7.5. This series is
targeting Linux 7.6, or 7.5 if feasible.

Both series will be merged through liveupdate.git, matching the
agreed-upon tree in the MAINTAINERS entry for PCI LIVE UPDATE.

Testing
-------

This series was tested in conjunction with v5 of the VFIO PCI driver
series:

  https://lore.kernel.org/kvm/20260714151505.3466855-1-vipinsh@google.com/

The full set of patches used for testing can be found on GitHub.

  https://github.com/dmatlack/linux/tree/liveupdate/pci/p2pdma/v1-with-vfio

Testing was performed using vfio_pci_liveupdate_kexec_test with the igb
selftest driver, which keeps the device performing DMA while kexec
happens. QEMU traced every PCI config write, and each run checked that
the next kernel did not write to the BAR, ROM, bridge window, or bus
number registers of the preserved device or any bridge upstream of it,
did not change their decode or Bus Master Enable bits, and did not
enable INTx on them, and that their BARs and windows had the same
values before and after kexec. The following configurations were
tested:

  - x86 (q35): igb behind a root port.

  - x86 (q35): igb behind a PCIe switch (root port, upstream port,
    downstream port), with a non-preserved device on another downstream
    port of the same switch.

  - arm64 (virt, SMMUv3, generic ECAM host bridge): igb behind a root
    port. This exercises the path where bridge windows are not read
    from hardware during scanning.

Future Work
-----------

Please see the PCI core Live Update Roadmap:

  https://lore.kernel.org/linux-pci/20261001232133.560284-1-dmatlack@google.com/


David Matlack (10):
  PCI: Factor out BAR sizing from pci_read_bases()
  PCI: liveupdate: Avoid sizing BARs of incoming devices
  PCI: Factor out bridge window probing from pci_read_bridge_windows()
  PCI: liveupdate: Avoid probing windows of incoming bridges
  PCI: Skip programming bridge windows that are already programmed
  PCI: liveupdate: Claim resources of incoming devices during scanning
  PCI: liveupdate: Never assign resources of incoming devices
  PCI: liveupdate: Prevent releasing resource windows of incoming
    bridges
  PCI: liveupdate: Don't realign resources of incoming devices
  PCI: liveupdate: Avoid enabling INTx on incoming devices

 drivers/pci/liveupdate.c    | 363 +++++++++++++++++++++++++++++++++++-
 drivers/pci/liveupdate.h    |  26 +++
 drivers/pci/pci.c           |  18 ++
 drivers/pci/probe.c         | 110 ++++++++---
 drivers/pci/setup-bus.c     |  62 ++++--
 include/linux/kho/abi/pci.h |  52 +++++-
 6 files changed, 589 insertions(+), 42 deletions(-)


base-commit: 312200e9211ede3f9a03cc17d094805e06e4674e
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 01/10] PCI: Factor out BAR sizing from pci_read_bases()

[David Matlack]

Move the code that sizes the standard BARs and the Expansion ROM BAR,
including disabling decoding while sizing, out of pci_read_bases() and
into a new helper, pci_size_bases().

A subsequent commit will obtain the sizes of the BARs of devices
preserved across Live Update from the previous kernel instead of sizing
them, since sizing requires disabling decoding.

No functional change intended.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/probe.c | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index debe0ad1ef68..9187f1d91766 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -323,22 +323,11 @@ int __pci_read_base(struct pci_dev *dev, enum pci_bar_type type,
 	return (res->flags & IORESOURCE_MEM_64) ? 1 : 0;
 }
 
-static __always_inline void pci_read_bases(struct pci_dev *dev,
-					   unsigned int howmany, int rom)
+static void pci_size_bases(struct pci_dev *dev, unsigned int howmany, int rom,
+			   u32 *stdbars, u32 *rombar)
 {
-	u32 rombar, stdbars[PCI_STD_NUM_BARS];
-	unsigned int pos, reg;
 	u16 orig_cmd;
 
-	BUILD_BUG_ON(statically_true(howmany > PCI_STD_NUM_BARS));
-
-	if (dev->non_compliant_bars)
-		return;
-
-	/* Per PCIe r4.0, sec 9.3.4.1.11, the VF BARs are all RO Zero */
-	if (dev->is_virtfn)
-		return;
-
 	/* No printks while decoding is disabled! */
 	if (!dev->mmio_always_on) {
 		pci_read_config_word(dev, PCI_COMMAND, &orig_cmd);
@@ -350,11 +339,29 @@ static __always_inline void pci_read_bases(struct pci_dev *dev,
 
 	__pci_size_stdbars(dev, howmany, PCI_BASE_ADDRESS_0, stdbars);
 	if (rom)
-		__pci_size_rom(dev, rom, &rombar);
+		__pci_size_rom(dev, rom, rombar);
 
 	if (!dev->mmio_always_on &&
 	    (orig_cmd & PCI_COMMAND_DECODE_ENABLE))
 		pci_write_config_word(dev, PCI_COMMAND, orig_cmd);
+}
+
+static __always_inline void pci_read_bases(struct pci_dev *dev,
+					   unsigned int howmany, int rom)
+{
+	u32 rombar, stdbars[PCI_STD_NUM_BARS];
+	unsigned int pos, reg;
+
+	BUILD_BUG_ON(statically_true(howmany > PCI_STD_NUM_BARS));
+
+	if (dev->non_compliant_bars)
+		return;
+
+	/* Per PCIe r4.0, sec 9.3.4.1.11, the VF BARs are all RO Zero */
+	if (dev->is_virtfn)
+		return;
+
+	pci_size_bases(dev, howmany, rom, stdbars, &rombar);
 
 	for (pos = 0; pos < howmany; pos++) {
 		struct resource *res = &dev->resource[pos];
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 02/10] PCI: liveupdate: Avoid sizing BARs of incoming devices

[David Matlack]

Avoid sizing the BARs of incoming preserved devices during enumeration.
Instead, have the previous kernel pass the BAR masks in struct
pci_dev_ser and use those in place of sizing.

Sizing a BAR requires disabling decoding. While decoding is disabled,
the device does not respond to memory transactions targeting its BARs:
Posted Writes are silently dropped and Reads fail with Unsupported
Request. This would disrupt anything that keeps accessing a preserved
device across Live Update, such as P2PDMA from other preserved devices,
or MMIO from VMs whose vCPUs keep running during kexec.

Sizing is the only part of reading a BAR that requires writing to it.
The address and type of each BAR can be read from the device without
disturbing it. So pass only the result of sizing across Live Update:
for each BAR register, the value it returns when read after writing all
1s to it (its mask). Append the masks to struct pci_dev_ser and
advertise their presence with a new optional feature flag,
PCI_SER_OPTIONAL_BAR_MASKS. The next kernel then decodes the BARs as if
it had sized them, so the ABI does not depend on any kernel-internal
state such as IORESOURCE_* flags.

Compute the masks from the size of each resource in
pci_liveupdate_freeze(), once preservation status can no longer change
and just before kexec, rather than when the device is preserved. A BAR
can change size after the device is preserved, e.g. via Resizable BAR,
in which case the next kernel would decode the BAR with the wrong size.

Cache the optional features of the incoming FLB that this kernel
supports in struct pci_liveupdate_global when the FLB is retrieved, so
they can be checked without getting a reference to the incoming FLB,
and verify that the incoming struct pci_dev_ser entries are large
enough to contain the fields of those features.

If the previous kernel did not pass BAR masks, fall back to sizing the
BARs.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/liveupdate.c    | 138 +++++++++++++++++++++++++++++++++++-
 drivers/pci/liveupdate.h    |   9 +++
 drivers/pci/probe.c         |  10 +++
 include/linux/kho/abi/pci.h |  25 ++++++-
 4 files changed, 178 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c
index 6f92d4ddca2b..f48cab3cc232 100644
--- a/drivers/pci/liveupdate.c
+++ b/drivers/pci/liveupdate.c
@@ -197,6 +197,13 @@
  *  * The PCI core does not disable bus mastering on outgoing preserved devices
  *    during kexec. This allows preserved devices to issue memory transactions
  *    throughout the Live Update.
+ *
+ *  * The PCI core does not size the BARs of incoming preserved devices, since
+ *    sizing requires disabling decoding, which would disrupt memory
+ *    transactions targeting the device (e.g. P2PDMA, or MMIO from VMs whose
+ *    vCPUs keep running during kexec). Instead, the previous kernel passes the
+ *    size of each BAR in struct pci_dev_ser. Everything else about the BARs
+ *    (e.g. their addresses and types) is read from the device as usual.
  */
 
 #define pr_fmt(fmt) "PCI: liveupdate: " fmt
@@ -221,10 +228,14 @@
  * @had_incoming: True if the previous kernel preserved at least one PCI device.
  *                Set when the incoming FLB is retrieved and never cleared, so
  *                it stays true after Live Update finishes.
+ * @incoming_optional_flags: The optional features in the incoming FLB
+ *                           (struct pci_ser::optional_flags) that this kernel
+ *                           supports. Set when the incoming FLB is retrieved.
  */
 struct pci_liveupdate_global {
 	struct rw_semaphore rwsem;
 	bool had_incoming;
+	u64 incoming_optional_flags;
 };
 
 static struct pci_liveupdate_global pci_liveupdate = {
@@ -267,6 +278,26 @@ struct pci_flb_incoming {
  */
 #define PCI_SER_REQUIRED_FLAGS_SUPPORTED	0ULL
 
+/*
+ * Bits in struct pci_ser::optional_flags that this kernel knows how to handle.
+ * This kernel also provides all of them to the next kernel.
+ */
+#define PCI_SER_OPTIONAL_FLAGS_SUPPORTED	PCI_SER_OPTIONAL_BAR_MASKS
+
+/*
+ * The minimum size of struct pci_dev_ser given the optional features that are
+ * present in it.
+ */
+static u32 pci_dev_ser_min_size(u64 optional_flags)
+{
+	u32 size = PCI_DEV_SER_MIN_SIZE;
+
+	if (optional_flags & PCI_SER_OPTIONAL_BAR_MASKS)
+		size = max(size, offsetofend(struct pci_dev_ser, rom_mask));
+
+	return size;
+}
+
 /* PCI_LIVEUPDATE depends on 64BIT, so domain and bdf always fit. */
 static unsigned long pci_ser_xa_key(u32 domain, u16 bdf)
 {
@@ -287,7 +318,7 @@ static int pci_flb_preserve(struct liveupdate_flb_op_args *args)
 		return PTR_ERR(ser);
 
 	ser->required_flags = 0;
-	ser->optional_flags = 0;
+	ser->optional_flags = PCI_SER_OPTIONAL_FLAGS_SUPPORTED;
 	ser->dev_ser_size = sizeof(struct pci_dev_ser);
 	ser->nr_devices = 0;
 	ser->devices = 0;
@@ -325,6 +356,7 @@ static int pci_flb_retrieve(struct liveupdate_flb_op_args *args)
 	struct pci_flb_incoming *incoming;
 	struct pci_dev_ser *dev_ser;
 	struct kho_block_set_it it;
+	u64 optional_flags;
 	int ret;
 
 	pr_debug("Retrieving struct pci_ser (0x%llx)\n", args->data);
@@ -333,12 +365,16 @@ static int pci_flb_retrieve(struct liveupdate_flb_op_args *args)
 		panic("Incoming PCI FLB requires unsupported features (0x%llx)\n",
 		      ser->required_flags & ~PCI_SER_REQUIRED_FLAGS_SUPPORTED);
 
+	/* Ignore any optional features that this kernel does not support. */
+	optional_flags = ser->optional_flags & PCI_SER_OPTIONAL_FLAGS_SUPPORTED;
+
 	/*
 	 * The previous kernel may have appended fields to struct pci_dev_ser,
 	 * which this kernel ignores, but every entry must be at least the
-	 * minimum size.
+	 * minimum size, including the fields of the optional features that
+	 * are present.
 	 */
-	if (ser->dev_ser_size < PCI_DEV_SER_MIN_SIZE)
+	if (ser->dev_ser_size < pci_dev_ser_min_size(optional_flags))
 		panic("Incoming PCI FLB has unsupported struct pci_dev_ser size (%u)\n",
 		      ser->dev_ser_size);
 
@@ -378,6 +414,8 @@ static int pci_flb_retrieve(struct liveupdate_flb_op_args *args)
 	if (!xa_empty(&incoming->xa))
 		pci_liveupdate.had_incoming = true;
 
+	pci_liveupdate.incoming_optional_flags = optional_flags;
+
 	args->obj = incoming;
 	return 0;
 }
@@ -541,6 +579,53 @@ static void pci_liveupdate_unpreserve_path(struct pci_flb_outgoing *outgoing,
 	}
 }
 
+/*
+ * Compute the value that the BAR backing resource @resno of @dev returns when
+ * read after writing all 1s to it, from the size of the resource.
+ */
+static u64 pci_liveupdate_bar_mask(struct pci_dev *dev, int resno)
+{
+	struct resource *res = &dev->resource[resno];
+	resource_size_t size = resource_size(res);
+
+	if (!res->flags || !size)
+		return 0;
+
+	/* Skip resources that are not decoded by the BAR. */
+	if (res->flags & IORESOURCE_PCI_EA_BEI)
+		return 0;
+
+	if (resno == PCI_ROM_RESOURCE && (res->flags & IORESOURCE_ROM_SHADOW))
+		return 0;
+
+	/* The size of a BAR is always a power of 2. */
+	pci_WARN_ONCE(dev, size & (size - 1),
+		      "%s %pR: Size is not a power of 2\n",
+		      pci_resource_name(dev, resno), res);
+
+	return ~((u64)size - 1);
+}
+
+static void pci_liveupdate_save_bar_masks(struct pci_dev *dev,
+					  struct pci_dev_ser *dev_ser)
+{
+	u64 mask;
+	int i;
+
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		mask = pci_liveupdate_bar_mask(dev, i);
+		dev_ser->bar_masks[i] = lower_32_bits(mask);
+
+		/* The upper half of a 64-bit BAR is in the next register. */
+		if ((dev->resource[i].flags & IORESOURCE_MEM_64) &&
+		    i + 1 < PCI_STD_NUM_BARS)
+			dev_ser->bar_masks[++i] = upper_32_bits(mask);
+	}
+
+	mask = pci_liveupdate_bar_mask(dev, PCI_ROM_RESOURCE);
+	dev_ser->rom_mask = lower_32_bits(mask);
+}
+
 static int pci_liveupdate_preserve_device(struct pci_flb_outgoing *outgoing,
 					  struct pci_dev *dev)
 {
@@ -833,6 +918,14 @@ void pci_liveupdate_freeze(struct pci_dev *dev)
 {
 	guard(rwsem_write)(&pci_liveupdate.rwsem);
 	dev->liveupdate.frozen = true;
+
+	/*
+	 * Save the BAR masks of outgoing devices now that preservation status
+	 * is frozen, rather than when the devices are preserved, in case the
+	 * size of a BAR changes in between (e.g. via Resizable BAR).
+	 */
+	if (dev->liveupdate.outgoing)
+		pci_liveupdate_save_bar_masks(dev, dev->liveupdate.outgoing);
 }
 
 static int pci_liveupdate_finish_device(struct pci_ser *ser, struct pci_dev *dev)
@@ -984,6 +1077,45 @@ bool pci_liveupdate_is_outgoing(struct pci_dev *dev)
 	return dev->liveupdate.outgoing;
 }
 
+/**
+ * pci_liveupdate_read_bar_masks() - Get the BAR masks of an incoming device
+ * @dev: The PCI device
+ * @nr_bars: The number of standard BARs to get masks for
+ * @bar_masks: Array of @nr_bars entries to fill with the standard BAR masks
+ * @rom_mask: Filled with the Expansion ROM BAR mask, if not NULL
+ *
+ * Get the BAR masks passed by the previous kernel, i.e. the values that the
+ * BARs of @dev return when read after writing all 1s to them, so that the BARs
+ * do not need to be sized.
+ *
+ * Return: 0 on success, or <0 if @dev is not incoming-preserved or the
+ * previous kernel did not pass BAR masks.
+ */
+int pci_liveupdate_read_bar_masks(struct pci_dev *dev, unsigned int nr_bars,
+				  u32 *bar_masks, u32 *rom_mask)
+{
+	struct pci_dev_ser *dev_ser;
+	unsigned int i;
+
+	guard(rwsem_read)(&pci_liveupdate.rwsem);
+
+	dev_ser = dev->liveupdate.incoming;
+	if (!dev_ser)
+		return -EINVAL;
+
+	if (!(pci_liveupdate.incoming_optional_flags &
+	      PCI_SER_OPTIONAL_BAR_MASKS))
+		return -EOPNOTSUPP;
+
+	for (i = 0; i < nr_bars; i++)
+		bar_masks[i] = dev_ser->bar_masks[i];
+
+	if (rom_mask)
+		*rom_mask = dev_ser->rom_mask;
+
+	return 0;
+}
+
 /**
  * pci_liveupdate_is_incoming() - Check if a device is incoming-preserved
  * @dev: The PCI device to check
diff --git a/drivers/pci/liveupdate.h b/drivers/pci/liveupdate.h
index 333ddd8f36c5..80749f14beea 100644
--- a/drivers/pci/liveupdate.h
+++ b/drivers/pci/liveupdate.h
@@ -19,6 +19,8 @@ bool pci_liveupdate_refuse_bus_numbers(struct pci_bus *bus, struct pci_dev *dev)
 int pci_liveupdate_adopt_acs(struct pci_dev *dev);
 int pci_liveupdate_adopt_ari(struct pci_dev *dev);
 bool pci_liveupdate_is_outgoing(struct pci_dev *dev);
+int pci_liveupdate_read_bar_masks(struct pci_dev *dev, unsigned int nr_bars,
+				  u32 *bar_masks, u32 *rom_mask);
 #else
 static inline void pci_liveupdate_setup_device(struct pci_dev *dev)
 {
@@ -57,6 +59,13 @@ static inline bool pci_liveupdate_is_outgoing(struct pci_dev *dev)
 {
 	return false;
 }
+
+static inline int pci_liveupdate_read_bar_masks(struct pci_dev *dev,
+						unsigned int nr_bars,
+						u32 *bar_masks, u32 *rom_mask)
+{
+	return -EINVAL;
+}
 #endif
 
 #endif /* DRIVERS_PCI_LIVEUPDATE_H */
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 9187f1d91766..3cb2e481eddf 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -328,6 +328,16 @@ static void pci_size_bases(struct pci_dev *dev, unsigned int howmany, int rom,
 {
 	u16 orig_cmd;
 
+	/*
+	 * Sizing BARs requires disabling decoding, which would disrupt memory
+	 * transactions targeting devices preserved across Live Update (e.g.
+	 * P2PDMA, or MMIO from VMs whose vCPUs keep running during kexec), so
+	 * use the BAR masks passed by the previous kernel for those devices.
+	 */
+	if (!pci_liveupdate_read_bar_masks(dev, howmany, stdbars,
+					   rom ? rombar : NULL))
+		return;
+
 	/* No printks while decoding is disabled! */
 	if (!dev->mmio_always_on) {
 		pci_read_config_word(dev, PCI_COMMAND, &orig_cmd);
diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h
index c2e3a5949f73..5fa2ecfe5b6d 100644
--- a/include/linux/kho/abi/pci.h
+++ b/include/linux/kho/abi/pci.h
@@ -8,8 +8,10 @@
 #ifndef _LINUX_KHO_ABI_PCI_H
 #define _LINUX_KHO_ABI_PCI_H
 
+#include <linux/bits.h>
 #include <linux/bug.h>
 #include <linux/compiler.h>
+#include <linux/pci_regs.h>
 #include <linux/stddef.h>
 #include <linux/types.h>
 
@@ -56,11 +58,24 @@
  *            reference count is equal to the number of preserved devices at or
  *            below it in the PCI hierarchy when the struct pci_dev_ser is in
  *            use, and 0 otherwise.
+ * @bar_masks: The value each standard BAR register returns when read after
+ *             writing all 1s to it, i.e. the mask that encodes the size of the
+ *             BAR. Only the address bits are meaningful. A 64-bit BAR uses two
+ *             entries, like it uses two registers. 0 if the BAR is not
+ *             implemented. Used by the next kernel instead of sizing the BARs,
+ *             which requires disabling decoding and would disrupt memory
+ *             transactions targeting the device.
+ * @rom_mask: Same as @bar_masks, but for the Expansion ROM BAR.
+ *
+ * Each field after @refcount is only present if the ``PCI_SER_OPTIONAL_*``
+ * flag that covers it is set in ``pci_ser.optional_flags``.
  */
 struct pci_dev_ser {
 	u32 domain;
 	u16 bdf;
 	u16 refcount;
+	u32 bar_masks[PCI_STD_NUM_BARS];
+	u32 rom_mask;
 } __packed;
 
 /*
@@ -80,7 +95,7 @@ struct pci_dev_ser {
  * @required_flags: Features that the next kernel must support to take over
  *                  the preserved devices. No flags are currently defined.
  * @optional_flags: Features that the next kernel may ignore if it does not
- *                  support them. No flags are currently defined.
+ *                  support them (``PCI_SER_OPTIONAL_*``).
  * @dev_ser_size: The size in bytes of each struct pci_dev_ser in @devices.
  * @nr_devices: The number of devices that were preserved.
  * @devices: Physical address of the first KHO block containing pci_dev_ser.
@@ -93,4 +108,12 @@ struct pci_ser {
 	u64 devices;
 } __packed;
 
+/*
+ * Bits in struct pci_ser::optional_flags. If a bit is set, @dev_ser_size is
+ * large enough to contain the fields of struct pci_dev_ser that it covers:
+ *
+ *  * PCI_SER_OPTIONAL_BAR_MASKS: @bar_masks and @rom_mask
+ */
+#define PCI_SER_OPTIONAL_BAR_MASKS	BIT_ULL(0)
+
 #endif /* _LINUX_KHO_ABI_PCI_H */
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 03/10] PCI: Factor out bridge window probing from pci_read_bridge_windows()

[David Matlack]

Move the code that determines which windows a bridge implements, which
requires temporarily writing to the window registers, out of
pci_read_bridge_windows() and into a new helper,
pci_probe_bridge_windows(). Probe all windows before reading any of
them.

A subsequent commit will obtain which windows are implemented by bridges
preserved across Live Update from the previous kernel instead of probing
them, since probing can temporarily disable windows.

No functional change intended.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/probe.c | 44 ++++++++++++++++++++++++++++----------------
 1 file changed, 28 insertions(+), 16 deletions(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 3cb2e481eddf..473e4389ffae 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -520,19 +520,14 @@ static void pci_read_bridge_mmio_pref(struct pci_dev *dev, struct resource *res,
 	}
 }
 
-static void pci_read_bridge_windows(struct pci_dev *bridge)
+/*
+ * Determine which windows @bridge implements. This requires temporarily
+ * writing to the window registers of windows that read as zero.
+ */
+static void pci_probe_bridge_windows(struct pci_dev *bridge)
 {
-	u32 buses;
 	u16 io;
 	u32 pmem, tmp;
-	struct resource res;
-
-	pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses);
-	res.flags = IORESOURCE_BUS;
-	res.start = FIELD_GET(PCI_SECONDARY_BUS_MASK, buses);
-	res.end = FIELD_GET(PCI_SUBORDINATE_BUS_MASK, buses);
-	pci_info(bridge, "PCI bridge to %pR%s\n", &res,
-		 bridge->transparent ? " (subtractive decode)" : "");
 
 	pci_read_config_word(bridge, PCI_IO_BASE, &io);
 	if (!io) {
@@ -540,12 +535,8 @@ static void pci_read_bridge_windows(struct pci_dev *bridge)
 		pci_read_config_word(bridge, PCI_IO_BASE, &io);
 		pci_write_config_word(bridge, PCI_IO_BASE, 0x0);
 	}
-	if (io) {
+	if (io)
 		bridge->io_window = 1;
-		pci_read_bridge_io(bridge, &res, true);
-	}
-
-	pci_read_bridge_mmio(bridge, &res, true);
 
 	/*
 	 * DECchip 21050 pass 2 errata: the bridge may miss an address
@@ -582,8 +573,29 @@ static void pci_read_bridge_windows(struct pci_dev *bridge)
 		if (tmp)
 			bridge->pref_64_window = 1;
 	}
+}
+
+static void pci_read_bridge_windows(struct pci_dev *bridge)
+{
+	u32 buses;
+	struct resource res;
+
+	pci_read_config_dword(bridge, PCI_PRIMARY_BUS, &buses);
+	res.flags = IORESOURCE_BUS;
+	res.start = FIELD_GET(PCI_SECONDARY_BUS_MASK, buses);
+	res.end = FIELD_GET(PCI_SUBORDINATE_BUS_MASK, buses);
+	pci_info(bridge, "PCI bridge to %pR%s\n", &res,
+		 bridge->transparent ? " (subtractive decode)" : "");
+
+	pci_probe_bridge_windows(bridge);
+
+	if (bridge->io_window)
+		pci_read_bridge_io(bridge, &res, true);
+
+	pci_read_bridge_mmio(bridge, &res, true);
 
-	pci_read_bridge_mmio_pref(bridge, &res, true);
+	if (bridge->pref_window)
+		pci_read_bridge_mmio_pref(bridge, &res, true);
 }
 
 void pci_read_bridge_bases(struct pci_bus *child)
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 04/10] PCI: liveupdate: Avoid probing windows of incoming bridges

[David Matlack]

Avoid probing which windows incoming preserved bridges implement during
enumeration. Instead, add a new optional feature for passing the
results of probing in struct pci_dev_ser, and use it to pass which
windows each preserved bridge implements.

pci_probe_bridge_windows() always writes all 1s to the Prefetchable
Base Upper 32 Bits register of bridges with a 64-bit prefetchable
window to check that it is writable. This temporarily disables the
prefetchable window, so the bridge stops forwarding memory transactions
targeting devices below it, such as P2PDMA from other preserved devices,
or MMIO from VMs whose vCPUs keep running during kexec.

The new optional feature, PCI_SER_OPTIONAL_PROBED, appends two bitmasks
to struct pci_dev_ser: @probed holds the results of probing properties
of the device that require temporarily writing to it, one bit per
property (PCI_DEV_SER_PROBED_*), and @probed_supported indicates which
bits of @probed are valid. This allows passing the results of more
probes in the future (e.g. whether INTx masking is broken) without
adding a new feature flag for each one, and lets the next kernel use
the results it understands and probe the rest.

If the previous kernel did not pass which windows a bridge implements,
fall back to probing the windows.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/liveupdate.c    | 93 ++++++++++++++++++++++++++++++++++++-
 drivers/pci/liveupdate.h    |  6 +++
 drivers/pci/probe.c         |  9 ++++
 include/linux/kho/abi/pci.h | 24 ++++++++++
 4 files changed, 131 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c
index f48cab3cc232..72ad4fd00f7b 100644
--- a/drivers/pci/liveupdate.c
+++ b/drivers/pci/liveupdate.c
@@ -204,6 +204,12 @@
  *    vCPUs keep running during kexec). Instead, the previous kernel passes the
  *    size of each BAR in struct pci_dev_ser. Everything else about the BARs
  *    (e.g. their addresses and types) is read from the device as usual.
+ *
+ *  * The PCI core does not probe properties of incoming preserved devices
+ *    that require temporarily writing to the device (e.g. which windows a
+ *    bridge implements), since that would disrupt memory transactions
+ *    targeting the device or devices below it. Instead, the previous kernel
+ *    passes the results of probing in struct pci_dev_ser.
  */
 
 #define pr_fmt(fmt) "PCI: liveupdate: " fmt
@@ -282,7 +288,13 @@ struct pci_flb_incoming {
  * Bits in struct pci_ser::optional_flags that this kernel knows how to handle.
  * This kernel also provides all of them to the next kernel.
  */
-#define PCI_SER_OPTIONAL_FLAGS_SUPPORTED	PCI_SER_OPTIONAL_BAR_MASKS
+#define PCI_SER_OPTIONAL_FLAGS_SUPPORTED \
+	(PCI_SER_OPTIONAL_BAR_MASKS | PCI_SER_OPTIONAL_PROBED)
+
+#define PCI_DEV_SER_PROBED_WINDOWS		\
+	(PCI_DEV_SER_PROBED_IO_WINDOW |		\
+	 PCI_DEV_SER_PROBED_PREF_WINDOW |	\
+	 PCI_DEV_SER_PROBED_PREF_64_WINDOW)
 
 /*
  * The minimum size of struct pci_dev_ser given the optional features that are
@@ -295,6 +307,9 @@ static u32 pci_dev_ser_min_size(u64 optional_flags)
 	if (optional_flags & PCI_SER_OPTIONAL_BAR_MASKS)
 		size = max(size, offsetofend(struct pci_dev_ser, rom_mask));
 
+	if (optional_flags & PCI_SER_OPTIONAL_PROBED)
+		size = max(size, offsetofend(struct pci_dev_ser, probed));
+
 	return size;
 }
 
@@ -626,6 +641,28 @@ static void pci_liveupdate_save_bar_masks(struct pci_dev *dev,
 	dev_ser->rom_mask = lower_32_bits(mask);
 }
 
+static void pci_liveupdate_preserve_probed(struct pci_dev *dev,
+					   struct pci_dev_ser *dev_ser)
+{
+	u64 supported = 0, probed = 0;
+
+	if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
+		supported |= PCI_DEV_SER_PROBED_WINDOWS;
+
+		if (dev->io_window)
+			probed |= PCI_DEV_SER_PROBED_IO_WINDOW;
+
+		if (dev->pref_window)
+			probed |= PCI_DEV_SER_PROBED_PREF_WINDOW;
+
+		if (dev->pref_64_window)
+			probed |= PCI_DEV_SER_PROBED_PREF_64_WINDOW;
+	}
+
+	dev_ser->probed_supported = supported;
+	dev_ser->probed = probed;
+}
+
 static int pci_liveupdate_preserve_device(struct pci_flb_outgoing *outgoing,
 					  struct pci_dev *dev)
 {
@@ -671,6 +708,8 @@ static int pci_liveupdate_preserve_device(struct pci_flb_outgoing *outgoing,
 		if (IS_ERR(dev_ser))
 			return PTR_ERR(dev_ser);
 
+		pci_liveupdate_preserve_probed(dev, dev_ser);
+
 		pci_info(dev, "Device will be preserved across next Live Update\n");
 		outgoing->ser->nr_devices++;
 		outgoing->ser->devices = kho_block_set_head_pa(&outgoing->block_set);
@@ -1116,6 +1155,58 @@ int pci_liveupdate_read_bar_masks(struct pci_dev *dev, unsigned int nr_bars,
 	return 0;
 }
 
+/*
+ * Get the results of probing @bits on the incoming device @dev, as passed by
+ * the previous kernel. Returns false if the previous kernel did not pass all of
+ * @bits, in which case they must be probed.
+ */
+static bool pci_liveupdate_read_probed(struct pci_dev *dev, u64 bits,
+				       u64 *probed)
+{
+	struct pci_dev_ser *dev_ser = dev->liveupdate.incoming;
+
+	lockdep_assert_held(&pci_liveupdate.rwsem);
+
+	if (!dev_ser)
+		return false;
+
+	if (!(pci_liveupdate.incoming_optional_flags & PCI_SER_OPTIONAL_PROBED))
+		return false;
+
+	if ((dev_ser->probed_supported & bits) != bits)
+		return false;
+
+	*probed = dev_ser->probed & bits;
+	return true;
+}
+
+/**
+ * pci_liveupdate_read_bridge_windows() - Get the windows of an incoming bridge
+ * @bridge: The PCI bridge
+ *
+ * Set which windows @bridge implements (@bridge->io_window, pref_window, and
+ * pref_64_window) as determined by the previous kernel, so that the windows do
+ * not need to be probed.
+ *
+ * Return: 0 on success, or -ENODATA if @bridge is not incoming-preserved or
+ * the previous kernel did not pass which windows @bridge implements.
+ */
+int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge)
+{
+	u64 probed;
+
+	guard(rwsem_read)(&pci_liveupdate.rwsem);
+
+	if (!pci_liveupdate_read_probed(bridge, PCI_DEV_SER_PROBED_WINDOWS,
+					&probed))
+		return -ENODATA;
+
+	bridge->io_window = !!(probed & PCI_DEV_SER_PROBED_IO_WINDOW);
+	bridge->pref_window = !!(probed & PCI_DEV_SER_PROBED_PREF_WINDOW);
+	bridge->pref_64_window = !!(probed & PCI_DEV_SER_PROBED_PREF_64_WINDOW);
+	return 0;
+}
+
 /**
  * pci_liveupdate_is_incoming() - Check if a device is incoming-preserved
  * @dev: The PCI device to check
diff --git a/drivers/pci/liveupdate.h b/drivers/pci/liveupdate.h
index 80749f14beea..4c4104025a7e 100644
--- a/drivers/pci/liveupdate.h
+++ b/drivers/pci/liveupdate.h
@@ -21,6 +21,7 @@ int pci_liveupdate_adopt_ari(struct pci_dev *dev);
 bool pci_liveupdate_is_outgoing(struct pci_dev *dev);
 int pci_liveupdate_read_bar_masks(struct pci_dev *dev, unsigned int nr_bars,
 				  u32 *bar_masks, u32 *rom_mask);
+int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge);
 #else
 static inline void pci_liveupdate_setup_device(struct pci_dev *dev)
 {
@@ -66,6 +67,11 @@ static inline int pci_liveupdate_read_bar_masks(struct pci_dev *dev,
 {
 	return -EINVAL;
 }
+
+static inline int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge)
+{
+	return -EINVAL;
+}
 #endif
 
 #endif /* DRIVERS_PCI_LIVEUPDATE_H */
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 473e4389ffae..184d93aa8522 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -529,6 +529,15 @@ static void pci_probe_bridge_windows(struct pci_dev *bridge)
 	u16 io;
 	u32 pmem, tmp;
 
+	/*
+	 * Writing to the window registers would disrupt memory transactions
+	 * that bridges preserved across Live Update forward to devices below
+	 * them, so use the windows determined by the previous kernel for those
+	 * bridges.
+	 */
+	if (!pci_liveupdate_read_bridge_windows(bridge))
+		return;
+
 	pci_read_config_word(bridge, PCI_IO_BASE, &io);
 	if (!io) {
 		pci_write_config_word(bridge, PCI_IO_BASE, 0xe0f0);
diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h
index 5fa2ecfe5b6d..84a1bf71eeca 100644
--- a/include/linux/kho/abi/pci.h
+++ b/include/linux/kho/abi/pci.h
@@ -66,6 +66,13 @@
  *             which requires disabling decoding and would disrupt memory
  *             transactions targeting the device.
  * @rom_mask: Same as @bar_masks, but for the Expansion ROM BAR.
+ * @probed_supported: The ``PCI_DEV_SER_PROBED_*`` bits that are valid in
+ *                    @probed.
+ * @probed: The results of probing properties of the device that require
+ *          temporarily writing to it (``PCI_DEV_SER_PROBED_*``). Used by the
+ *          next kernel instead of probing the device again, which would
+ *          disrupt memory transactions targeting the device or devices below
+ *          it.
  *
  * Each field after @refcount is only present if the ``PCI_SER_OPTIONAL_*``
  * flag that covers it is set in ``pci_ser.optional_flags``.
@@ -76,8 +83,23 @@ struct pci_dev_ser {
 	u16 refcount;
 	u32 bar_masks[PCI_STD_NUM_BARS];
 	u32 rom_mask;
+	u64 probed_supported;
+	u64 probed;
 } __packed;
 
+/*
+ * Bits in struct pci_dev_ser::probed_supported and probed:
+ *
+ *  * PCI_DEV_SER_PROBED_IO_WINDOW: The bridge has an I/O window.
+ *  * PCI_DEV_SER_PROBED_PREF_WINDOW: The bridge has a prefetchable memory
+ *    window.
+ *  * PCI_DEV_SER_PROBED_PREF_64_WINDOW: The bridge's prefetchable memory
+ *    window is 64-bit.
+ */
+#define PCI_DEV_SER_PROBED_IO_WINDOW		BIT_ULL(0)
+#define PCI_DEV_SER_PROBED_PREF_WINDOW		BIT_ULL(1)
+#define PCI_DEV_SER_PROBED_PREF_64_WINDOW	BIT_ULL(2)
+
 /*
  * The size of struct pci_dev_ser when this ABI was introduced. Since fields
  * can only be appended, every struct pci_dev_ser is at least this large.
@@ -113,7 +135,9 @@ struct pci_ser {
  * large enough to contain the fields of struct pci_dev_ser that it covers:
  *
  *  * PCI_SER_OPTIONAL_BAR_MASKS: @bar_masks and @rom_mask
+ *  * PCI_SER_OPTIONAL_PROBED: @probed_supported and @probed
  */
 #define PCI_SER_OPTIONAL_BAR_MASKS	BIT_ULL(0)
+#define PCI_SER_OPTIONAL_PROBED		BIT_ULL(1)
 
 #endif /* _LINUX_KHO_ABI_PCI_H */
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 05/10] PCI: Skip programming bridge windows that are already programmed

[David Matlack]

Skip updating the window registers of a bridge in
pci_setup_bridge_io(), pci_setup_bridge_mmio(), and
pci_setup_bridge_mmio_pref() if they already contain the values being
written (ignoring the read-only low bits of the base and limit
registers).

pci_setup_bridge_io() and pci_setup_bridge_mmio_pref() temporarily
disable the I/O and prefetchable memory windows while updating them,
since the base and limit of each window are split across multiple
registers. This happens even if the window is not changing, e.g. for
every bridge whose windows were assigned by firmware and claimed, since
__pci_bus_assign_resources_one() calls pci_setup_bridge() on every
bridge that is not yet enabled during enumeration.
pci_setup_bridge_mmio() updates the memory window with a single write,
but skip unchanged writes there too for consistency.

This prevents disabling the windows of bridges preserved across Live
Update during enumeration, which would disrupt memory transactions
targeting preserved devices below them, such as P2PDMA from other
preserved devices, or MMIO from VMs whose vCPUs keep running during
kexec.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/setup-bus.c | 52 ++++++++++++++++++++++++++++++++---------
 1 file changed, 41 insertions(+), 11 deletions(-)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index e8c94aa1d3c1..4bec76950f46 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -799,8 +799,8 @@ static void pci_setup_bridge_io(struct pci_dev *bridge)
 	struct pci_bus_region region;
 	unsigned long io_mask;
 	u8 io_base_lo, io_limit_lo;
-	u16 l;
-	u32 io_upper16;
+	u16 l, cur_l, l_mask;
+	u32 io_upper16, cur_io_upper16;
 
 	io_mask = PCI_IO_RANGE_MASK;
 	if (bridge->io_window_1k)
@@ -823,6 +823,18 @@ static void pci_setup_bridge_io(struct pci_dev *bridge)
 		io_upper16 = 0;
 		l = 0x00f0;
 	}
+
+	/*
+	 * Leave the I/O range alone if it is already programmed as needed,
+	 * rather than temporarily disabling it below. The low bits of the I/O
+	 * Base and Limit registers are read-only.
+	 */
+	pci_read_config_word(bridge, PCI_IO_BASE, &cur_l);
+	pci_read_config_dword(bridge, PCI_IO_BASE_UPPER16, &cur_io_upper16);
+	l_mask = ((u8)io_mask << 8) | (u8)io_mask;
+	if (!((cur_l ^ l) & l_mask) && cur_io_upper16 == io_upper16)
+		return;
+
 	/* Temporarily disable the I/O range before updating PCI_IO_BASE */
 	pci_write_config_dword(bridge, PCI_IO_BASE_UPPER16, 0x0000ffff);
 	/* Update lower 16 bits of I/O base/limit */
@@ -836,7 +848,7 @@ static void pci_setup_bridge_mmio(struct pci_dev *bridge)
 	struct resource *res;
 	const char *res_name;
 	struct pci_bus_region region;
-	u32 l;
+	u32 l, cur_l;
 
 	/* Set up the top and bottom of the PCI Memory segment for this bus */
 	res = &bridge->resource[PCI_BRIDGE_MEM_WINDOW];
@@ -849,6 +861,12 @@ static void pci_setup_bridge_mmio(struct pci_dev *bridge)
 	} else {
 		l = 0x0000fff0;
 	}
+
+	/* The low bits of the Memory Base and Limit registers are read-only */
+	pci_read_config_dword(bridge, PCI_MEMORY_BASE, &cur_l);
+	if (!((cur_l ^ l) & 0xfff0fff0))
+		return;
+
 	pci_write_config_dword(bridge, PCI_MEMORY_BASE, l);
 }
 
@@ -857,14 +875,7 @@ static void pci_setup_bridge_mmio_pref(struct pci_dev *bridge)
 	struct resource *res;
 	const char *res_name;
 	struct pci_bus_region region;
-	u32 l, bu, lu;
-
-	/*
-	 * Clear out the upper 32 bits of PREF limit.  If
-	 * PCI_PREF_BASE_UPPER32 was non-zero, this temporarily disables
-	 * PREF range, which is ok.
-	 */
-	pci_write_config_dword(bridge, PCI_PREF_LIMIT_UPPER32, 0);
+	u32 l, bu, lu, cur_l, cur_bu, cur_lu;
 
 	/* Set up PREF base/limit */
 	bu = lu = 0;
@@ -882,6 +893,25 @@ static void pci_setup_bridge_mmio_pref(struct pci_dev *bridge)
 	} else {
 		l = 0x0000fff0;
 	}
+
+	/*
+	 * Leave the PREF range alone if it is already programmed as needed,
+	 * rather than temporarily disabling it below. The low bits of the
+	 * Prefetchable Memory Base and Limit registers are read-only.
+	 */
+	pci_read_config_dword(bridge, PCI_PREF_MEMORY_BASE, &cur_l);
+	pci_read_config_dword(bridge, PCI_PREF_BASE_UPPER32, &cur_bu);
+	pci_read_config_dword(bridge, PCI_PREF_LIMIT_UPPER32, &cur_lu);
+	if (!((cur_l ^ l) & 0xfff0fff0) && cur_bu == bu && cur_lu == lu)
+		return;
+
+	/*
+	 * Clear out the upper 32 bits of PREF limit.  If
+	 * PCI_PREF_BASE_UPPER32 was non-zero, this temporarily disables
+	 * PREF range, which is ok.
+	 */
+	pci_write_config_dword(bridge, PCI_PREF_LIMIT_UPPER32, 0);
+
 	pci_write_config_dword(bridge, PCI_PREF_MEMORY_BASE, l);
 
 	/* Set the upper 32 bits of PREF base & limit */
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 06/10] PCI: liveupdate: Claim resources of incoming devices during scanning

[David Matlack]

Claim the BARs and bridge windows of incoming preserved devices, as
programmed by the previous kernel, while scanning. Claim them for each
bus after its devices have been enumerated and pcibios_fixup_bus() has
run, and before the buses below it are scanned.

This ensures that the address ranges decoded by incoming devices are in
the resource tree before anything sizes or assigns resources,
regardless of how the host bridge driver sets up resources. Without
this, whether existing resources are claimed depends on the platform:
x86 claims them in pcibios_resource_survey(), pci_host_probe() and the
generic ACPI host bridge code only claim them if the host bridge has
preserve_config set, and some host bridge drivers (e.g. VMD) never
claim them.

Buses are scanned top-down and all bridges upstream of a preserved
device are also preserved, so the windows that contain the resources of
an incoming device are always claimed before the device's resources.
Outside of x86, bridge windows are not read from hardware during
scanning, so read them first if needed. Code that claims resources
later, such as pcibios_resource_survey() and pci_bus_claim_resources(),
skips resources that are already claimed.

Panic if claiming fails because something else has already claimed part
of the range decoded by an incoming device, since memory transactions
that still target the incoming device (e.g. P2PDMA, or MMIO from VMs
whose vCPUs keep running during kexec) may be corrupting it. Also panic
if a bridge window cannot be claimed for any other reason, e.g. because
it does not fit in the upstream window, since the bridge would keep
forwarding a range that the resource tree considers free, and the PCI
core could assign it to another device. Neither should ever happen,
since the previous kernel had the same resources claimed. A BAR that
fails for any other reason was never assigned an address (e.g. a ROM BAR
that the previous kernel left unassigned), so there is nothing to claim.

pci_liveupdate_claim_bus_resources() does nothing if the previous kernel
did not preserve any devices, so the only overhead outside of Live
Update is checking a flag under the PCI Live Update rwsem.

An alternative that was considered was setting preserve_config on host
bridges with incoming devices, which makes pci_host_probe() and the
generic ACPI code claim all existing resources. However, not all host
bridge drivers honor preserve_config, it is a firmware-provided policy
that applies to every device below the host bridge, and it would have
to be cleared once Live Update finishes.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/liveupdate.c | 99 ++++++++++++++++++++++++++++++++++++++++
 drivers/pci/liveupdate.h |  5 ++
 drivers/pci/probe.c      |  8 ++++
 3 files changed, 112 insertions(+)

diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c
index 72ad4fd00f7b..ac4525568818 100644
--- a/drivers/pci/liveupdate.c
+++ b/drivers/pci/liveupdate.c
@@ -184,6 +184,12 @@
  * The PCI core treats preserved devices differently than non-preserved devices.
  * This section enumerates those differences.
  *
+ *  * The PCI core claims the BARs and bridge windows of incoming preserved
+ *    devices, as programmed by the previous kernel, while scanning and before
+ *    any resources are sized or assigned. This ensures no other device is
+ *    assigned address space that an incoming device is still decoding, and
+ *    that the resources of incoming devices are never reassigned.
+ *
  *  * The PCI core adopts all ACS controls enabled on incoming preserved devices
  *    rather than assigning new ones. This ensures that TLPs are routed the same
  *    way after Live Update and ensures that IOMMU groups do not change. Note
@@ -1207,6 +1213,99 @@ int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge)
 	return 0;
 }
 
+/*
+ * Check if the windows of @bridge have been read from hardware. Outside of x86
+ * that does not happen during scanning.
+ */
+static bool pci_liveupdate_windows_read(struct pci_dev *bridge)
+{
+	int i;
+
+	for (i = PCI_BRIDGE_RESOURCES; i <= PCI_BRIDGE_RESOURCE_END; i++) {
+		if (pci_resource_n(bridge, i)->flags)
+			return true;
+	}
+
+	return false;
+}
+
+static void pci_liveupdate_claim_resource(struct pci_dev *dev, int resno)
+{
+	struct resource *r = pci_resource_n(dev, resno);
+	int ret;
+
+	/*
+	 * Skip resources that are unused, already claimed, or disabled (e.g.
+	 * bridge windows that are turned off).
+	 */
+	if (!r->flags || resource_assigned(r) ||
+	    (r->flags & IORESOURCE_DISABLED))
+		return;
+
+	/*
+	 * Panic if something else has already claimed part of the range, since
+	 * memory transactions still targeting the device may be corrupting it.
+	 * Also panic if a bridge window cannot be claimed at all, since the
+	 * bridge forwards the range even though the resource tree considers it
+	 * free. BARs that fail for other reasons were never assigned an address
+	 * (e.g. a ROM BAR that was left unassigned), so there is nothing to
+	 * claim.
+	 */
+	ret = pci_claim_resource(dev, resno);
+	if (ret == -EBUSY || (ret && pci_resource_is_bridge_win(resno)))
+		panic("%s %s %pR: Failed to claim resource of device preserved across Live Update\n",
+		      pci_name(dev), pci_resource_name(dev, resno), r);
+}
+
+/**
+ * pci_liveupdate_claim_bus_resources() - Claim the resources of incoming
+ *                                        devices on a bus
+ * @bus: The PCI bus
+ *
+ * Claim the windows of the bridge upstream of @bus, if it is
+ * incoming-preserved, and the BARs of the incoming preserved devices on @bus,
+ * as programmed by the previous kernel. The PCI core calls this while
+ * scanning, after enumerating the devices on @bus and before scanning the
+ * buses below it, so that the resources of incoming devices are claimed before
+ * any resources are sized or assigned.
+ *
+ * Buses are scanned top-down and all bridges upstream of a preserved device
+ * are also preserved, so the windows that contain the resources of an incoming
+ * device are always claimed before the device's resources. Failing to claim a
+ * resource because something else has already claimed part of its range is
+ * fatal, since memory transactions still targeting the device may be
+ * corrupting it.
+ *
+ * This does nothing if the previous kernel did not preserve any devices.
+ */
+void pci_liveupdate_claim_bus_resources(struct pci_bus *bus)
+{
+	struct pci_dev *dev;
+	int i;
+
+	guard(rwsem_read)(&pci_liveupdate.rwsem);
+
+	if (!pci_liveupdate.had_incoming)
+		return;
+
+	if (bus->self && bus->self->liveupdate.incoming) {
+		if (!pci_liveupdate_windows_read(bus->self))
+			pci_read_bridge_bases(bus);
+
+		for (i = PCI_BRIDGE_RESOURCES;
+		     i <= PCI_BRIDGE_RESOURCE_END; i++)
+			pci_liveupdate_claim_resource(bus->self, i);
+	}
+
+	list_for_each_entry(dev, &bus->devices, bus_list) {
+		if (!dev->liveupdate.incoming)
+			continue;
+
+		for (i = 0; i < PCI_BRIDGE_RESOURCES; i++)
+			pci_liveupdate_claim_resource(dev, i);
+	}
+}
+
 /**
  * pci_liveupdate_is_incoming() - Check if a device is incoming-preserved
  * @dev: The PCI device to check
diff --git a/drivers/pci/liveupdate.h b/drivers/pci/liveupdate.h
index 4c4104025a7e..98d23e217308 100644
--- a/drivers/pci/liveupdate.h
+++ b/drivers/pci/liveupdate.h
@@ -22,6 +22,7 @@ bool pci_liveupdate_is_outgoing(struct pci_dev *dev);
 int pci_liveupdate_read_bar_masks(struct pci_dev *dev, unsigned int nr_bars,
 				  u32 *bar_masks, u32 *rom_mask);
 int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge);
+void pci_liveupdate_claim_bus_resources(struct pci_bus *bus);
 #else
 static inline void pci_liveupdate_setup_device(struct pci_dev *dev)
 {
@@ -72,6 +73,10 @@ static inline int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge)
 {
 	return -EINVAL;
 }
+
+static inline void pci_liveupdate_claim_bus_resources(struct pci_bus *bus)
+{
+}
 #endif
 
 #endif /* DRIVERS_PCI_LIVEUPDATE_H */
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 184d93aa8522..a53d0077a08b 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -3155,6 +3155,14 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus,
 		bus->is_added = 1;
 	}
 
+	/*
+	 * Claim the resources of devices preserved across Live Update before
+	 * scanning the buses below and before any resources are sized or
+	 * assigned, so that nothing else is assigned the address space they
+	 * decode.
+	 */
+	pci_liveupdate_claim_bus_resources(bus);
+
 	/*
 	 * Calculate how many hotplug bridges and normal bridges there
 	 * are on this bus. We will distribute the additional available
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 07/10] PCI: liveupdate: Never assign resources of incoming devices

[David Matlack]

Prevent the PCI core from sizing or assigning the resources of incoming
preserved devices by adding a check for pci_liveupdate_is_incoming() to
pdev_resource_assignable(). It is consulted for every resource when
sizing bridge windows, in pbus_size_io() and pbus_size_mem(), and when
building the list of resources to assign, in pdev_sort_resources().

The resources of incoming devices are claimed as programmed by the
previous kernel while scanning, and claimed resources are never sized
or reassigned. However, an incoming device may also have resources that
the previous kernel never assigned an address, e.g. a disabled ROM BAR,
which therefore cannot be claimed. Assigning one of those would write
to the device's BARs, and updating a 64-bit BAR temporarily disables
memory decoding (see pci_std_update_resource()), which would disrupt
memory transactions targeting the device's other BARs, such as P2PDMA,
or MMIO from VMs whose vCPUs keep running during kexec. Leave them
unassigned while the device is incoming instead.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/setup-bus.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 4bec76950f46..b82440394ee2 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -311,6 +311,10 @@ static bool pdev_resource_assignable(struct pci_dev *dev, struct resource *res)
 	if (pci_resource_is_bridge_win(idx) && res->flags & IORESOURCE_DISABLED)
 		return false;
 
+	/* Devices preserved across Live Update keep their resources */
+	if (pci_liveupdate_is_incoming(dev))
+		return false;
+
 	return true;
 }
 
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 08/10] PCI: liveupdate: Prevent releasing resource windows of incoming bridges

[David Matlack]

Prevent the windows of bridges preserved across Live Update from being
released during the recursive "realloc" failure paths by adding a check
for pci_liveupdate_is_incoming() to pci_bridge_release_resources(), and
log that the window was not released, similar to the message that
pci_release_resource() logs when it releases a window.

This prevents the PCI core from forcibly releasing a preserved bridge
window to satisfy the resource needs of a failing child device.
Instead, the child's resources simply fail to be allocated. This should
never happen in practice though, since the previous kernel should have
established a sane resource allocation before kexec.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/setup-bus.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index b82440394ee2..7283c4be803e 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1700,6 +1700,12 @@ static void pci_bridge_release_resources(struct pci_bus *bus,
 
 	idx = pci_resource_num(dev, b_win);
 
+	if (pci_liveupdate_is_incoming(dev)) {
+		pci_info(dev, "%s %pR: not releasing window of bridge preserved across Live Update\n",
+			 pci_resource_name(dev, idx), b_win);
+		return;
+	}
+
 	/* If there are children, release them all */
 	release_child_resources(b_win);
 
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 09/10] PCI: liveupdate: Don't realign resources of incoming devices

[David Matlack]

Skip pci_reassigndev_resource_alignment() for incoming preserved devices
and log a warning instead.

When the "pci=resource_alignment=" kernel parameter matches a device,
pci_reassigndev_resource_alignment() disables Memory Space Enable, marks
the device's BARs as unassigned so they get reassigned, and disables
bridge windows. Doing this to a device preserved across Live Update
would disrupt any memory transactions targeting the device and change
the addresses of its BARs out from under anything still accessing them,
such as its driver, peers performing P2PDMA to it, or VMs whose vCPUs
keep running during kexec.

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/pci.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index e5b960ca26e6..853c84a2c782 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -6750,6 +6750,16 @@ void pci_reassigndev_resource_alignment(struct pci_dev *dev)
 		return;
 	}
 
+	/*
+	 * Reassigning resources disables decoding and moves BARs and bridge
+	 * windows, which would break devices preserved across Live Update that
+	 * are still performing memory transactions.
+	 */
+	if (pci_liveupdate_is_incoming(dev)) {
+		pci_warn(dev, "Can't reassign resources of device preserved across Live Update\n");
+		return;
+	}
+
 	pci_read_config_word(dev, PCI_COMMAND, &command);
 	command &= ~PCI_COMMAND_MEMORY;
 	pci_write_config_word(dev, PCI_COMMAND, command);
-- 
2.56.0.212.g129ede20d9-goog

[PATCH 10/10] PCI: liveupdate: Avoid enabling INTx on incoming devices

[David Matlack]

Avoid clearing PCI_COMMAND_INTX_DISABLE on incoming preserved devices.
Pass whether INTx masking is broken on each preserved device to the
next kernel instead of probing it, and skip clearing
PCI_COMMAND_INTX_DISABLE in do_pci_enable_device() for incoming
devices.

Clearing PCI_COMMAND_INTX_DISABLE enables INTx on the device, even if
only temporarily. pci_intx_mask_broken() probes whether INTx masking
works by toggling PCI_COMMAND_INTX_DISABLE during enumeration, and
do_pci_enable_device() clears it when a device that is not using MSI or
MSI-X is enabled. A preserved device may assert INTx at any point
during the Live Update, before its driver in the next kernel has set up
interrupt handling, so enabling INTx could result in interrupts that
no handler claims, on a line that may be shared with other devices.

Pass the result of probing as a new bit in the probed results in
struct pci_dev_ser, PCI_DEV_SER_PROBED_BROKEN_INTX_MASKING, which
reflects struct pci_dev::broken_intx_masking. If the previous kernel
did not pass it, fall back to probing INTx masking.

Drivers bound to incoming devices are responsible for enabling INTx once
they are ready to handle it. pci_alloc_irq_vectors() already does so
with pci_intx() when it falls back to INTx, and vfio-pci clears
PCI_COMMAND_INTX_DISABLE itself in vfio_pci_core_enable().

Assisted-by: LLM
Signed-off-by: David Matlack <dmatlack@google.com>
---
 drivers/pci/liveupdate.c    | 37 ++++++++++++++++++++++++++++++++++++-
 drivers/pci/liveupdate.h    |  6 ++++++
 drivers/pci/pci.c           |  8 ++++++++
 drivers/pci/probe.c         | 10 ++++++++++
 include/linux/kho/abi/pci.h |  3 +++
 5 files changed, 63 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/liveupdate.c b/drivers/pci/liveupdate.c
index ac4525568818..8a74762c3f7d 100644
--- a/drivers/pci/liveupdate.c
+++ b/drivers/pci/liveupdate.c
@@ -216,6 +216,11 @@
  *    bridge implements), since that would disrupt memory transactions
  *    targeting the device or devices below it. Instead, the previous kernel
  *    passes the results of probing in struct pci_dev_ser.
+ *
+ *  * The PCI core does not enable INTx (i.e. clear PCI_COMMAND_INTX_DISABLE)
+ *    on incoming preserved devices, neither while probing whether INTx masking
+ *    works nor when the device is enabled, since the device could then raise
+ *    interrupts before its driver is ready to handle them.
  */
 
 #define pr_fmt(fmt) "PCI: liveupdate: " fmt
@@ -650,7 +655,11 @@ static void pci_liveupdate_save_bar_masks(struct pci_dev *dev,
 static void pci_liveupdate_preserve_probed(struct pci_dev *dev,
 					   struct pci_dev_ser *dev_ser)
 {
-	u64 supported = 0, probed = 0;
+	u64 supported = PCI_DEV_SER_PROBED_BROKEN_INTX_MASKING;
+	u64 probed = 0;
+
+	if (dev->broken_intx_masking)
+		probed |= PCI_DEV_SER_PROBED_BROKEN_INTX_MASKING;
 
 	if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
 		supported |= PCI_DEV_SER_PROBED_WINDOWS;
@@ -1213,6 +1222,32 @@ int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge)
 	return 0;
 }
 
+/**
+ * pci_liveupdate_intx_mask_broken() - Get whether INTx masking is broken on an
+ *                                     incoming device
+ * @dev: The PCI device
+ *
+ * Get whether INTx masking is broken on @dev as determined by the previous
+ * kernel, so that INTx masking does not need to be probed.
+ *
+ * Return: 1 if INTx masking is broken, 0 if it is not, or -ENODATA if @dev is
+ * not incoming-preserved or the previous kernel did not pass whether INTx
+ * masking is broken on @dev.
+ */
+int pci_liveupdate_intx_mask_broken(struct pci_dev *dev)
+{
+	u64 probed;
+
+	guard(rwsem_read)(&pci_liveupdate.rwsem);
+
+	if (!pci_liveupdate_read_probed(dev,
+					PCI_DEV_SER_PROBED_BROKEN_INTX_MASKING,
+					&probed))
+		return -ENODATA;
+
+	return !!probed;
+}
+
 /*
  * Check if the windows of @bridge have been read from hardware. Outside of x86
  * that does not happen during scanning.
diff --git a/drivers/pci/liveupdate.h b/drivers/pci/liveupdate.h
index 98d23e217308..7f86cc85deee 100644
--- a/drivers/pci/liveupdate.h
+++ b/drivers/pci/liveupdate.h
@@ -22,6 +22,7 @@ bool pci_liveupdate_is_outgoing(struct pci_dev *dev);
 int pci_liveupdate_read_bar_masks(struct pci_dev *dev, unsigned int nr_bars,
 				  u32 *bar_masks, u32 *rom_mask);
 int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge);
+int pci_liveupdate_intx_mask_broken(struct pci_dev *dev);
 void pci_liveupdate_claim_bus_resources(struct pci_bus *bus);
 #else
 static inline void pci_liveupdate_setup_device(struct pci_dev *dev)
@@ -74,6 +75,11 @@ static inline int pci_liveupdate_read_bridge_windows(struct pci_dev *bridge)
 	return -EINVAL;
 }
 
+static inline int pci_liveupdate_intx_mask_broken(struct pci_dev *dev)
+{
+	return -EINVAL;
+}
+
 static inline void pci_liveupdate_claim_bus_resources(struct pci_bus *bus)
 {
 }
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 853c84a2c782..43af72d1e5c9 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2184,6 +2184,14 @@ static int do_pci_enable_device(struct pci_dev *dev, int bars)
 	if (dev->msi_enabled || dev->msix_enabled)
 		return 0;
 
+	/*
+	 * Leave INTx disabled on devices preserved across Live Update, since
+	 * the device could raise interrupts before its driver is ready. It is
+	 * up to the driver to enable INTx once it is ready.
+	 */
+	if (pci_liveupdate_is_incoming(dev))
+		return 0;
+
 	pci_read_config_byte(dev, PCI_INTERRUPT_PIN, &pin);
 	if (pin) {
 		pci_read_config_word(dev, PCI_COMMAND, &cmd);
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index a53d0077a08b..badce31b34f8 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1976,6 +1976,16 @@ static u8 pci_hdr_type(struct pci_dev *dev)
 static int pci_intx_mask_broken(struct pci_dev *dev)
 {
 	u16 orig, toggle, new;
+	int ret;
+
+	/*
+	 * Toggling PCI_COMMAND_INTX_DISABLE could let a device preserved across
+	 * Live Update raise interrupts before its driver is ready, so use the
+	 * result determined by the previous kernel for those devices.
+	 */
+	ret = pci_liveupdate_intx_mask_broken(dev);
+	if (ret >= 0)
+		return ret;
 
 	pci_read_config_word(dev, PCI_COMMAND, &orig);
 	toggle = orig ^ PCI_COMMAND_INTX_DISABLE;
diff --git a/include/linux/kho/abi/pci.h b/include/linux/kho/abi/pci.h
index 84a1bf71eeca..badcf2f01be7 100644
--- a/include/linux/kho/abi/pci.h
+++ b/include/linux/kho/abi/pci.h
@@ -95,10 +95,13 @@ struct pci_dev_ser {
  *    window.
  *  * PCI_DEV_SER_PROBED_PREF_64_WINDOW: The bridge's prefetchable memory
  *    window is 64-bit.
+ *  * PCI_DEV_SER_PROBED_BROKEN_INTX_MASKING: INTx masking cannot be used on
+ *    the device, i.e. struct pci_dev::broken_intx_masking is set.
  */
 #define PCI_DEV_SER_PROBED_IO_WINDOW		BIT_ULL(0)
 #define PCI_DEV_SER_PROBED_PREF_WINDOW		BIT_ULL(1)
 #define PCI_DEV_SER_PROBED_PREF_64_WINDOW	BIT_ULL(2)
+#define PCI_DEV_SER_PROBED_BROKEN_INTX_MASKING	BIT_ULL(3)
 
 /*
  * The size of struct pci_dev_ser when this ABI was introduced. Since fields
-- 
2.56.0.212.g129ede20d9-goog