Linux Documentation
 help / color / mirror / Atom feed
* [PATCH v11 00/13] PCI: liveupdate: PCI core support for Live Update
@ 2026-11-18 22:36 David Matlack
  2026-11-18 22:36 ` [PATCH v11 01/13] PCI: liveupdate: Set up FLB handler for the PCI core David Matlack
                   ` (12 more replies)
  0 siblings, 13 replies; 14+ messages in thread
From: David Matlack @ 2026-11-18 22:36 UTC (permalink / raw)
  To: kexec, linux-doc, linux-kernel, linux-mm, linux-pci
  Cc: Adithya Jayachandran, Alexander Graf, Alex Williamson,
	Alex Williamson, Bjorn Helgaas, Chris Down, Chris Li,
	David Matlack, David Rientjes, Jacob Pan, Jason Gunthorpe,
	Jonathan Corbet, Josh Hilke, Leon Romanovsky, Lukas Wunner,
	Mike Rapoport, Parav Pandit, Pasha Tatashin, Pranjal Shrivastava,
	Pratyush Yadav, Randy Dunlap, Saeed Mahameed, Samiullah Khawaja,
	Shuah Khan, Vipin Sharma, William Tu, Yi Liu

This series can be found on GitHub:

  https://github.com/dmatlack/linux/tree/liveupdate/pci/base/v11

This series introduces support in the PCI core for Live Update, enabling
drivers to preserve PCI devices across a kexec-based kernel update
without interrupting the device. This lets devices, such as ones
assigned to VMs through VFIO, keep running across a host kernel upgrade.

Preserved devices can keep doing DMA to system memory throughout the
Live Update. Devices may be behind bridges, but VFs, devices that need
device-specific ACS quirks, and CardBus are not supported. Support for
P2P and VF preservation will be addressed in future series.

Series Overview
---------------

This series implements the following to support PCI device preservation
across Live Update:

  1. Set up a File-Lifecycle-Bound (FLB) handler to track and preserve
     PCI-specific state (struct pci_ser) across Live Update using Kexec
     Handover (KHO).

  2. Add APIs for drivers to register devices for preservation
     (outgoing) and for the PCI core to identify preserved devices
     during enumeration (incoming).

  3. Automatically preserve all upstream bridges for any preserved
     endpoint. Use reference counting to ensure bridges remain preserved
     as long as any downstream device is preserved.

  4. Keep the Requester ID (bus, device, function) of preserved devices
     stable for the duration of their preservation by adopting the
     secondary bus numbers, subordinate bus numbers, and ARI Forwarding
     Enable of preserved bridges. After a Live Update, pci=assign-busses
     and pci=noari do not override this adopted state.

  5. Save and restore the ACS Control register and Egress Control Vector
     across resets for all devices (patch 8), so that adopted ACS
     controls survive a reset.

  6. Keep memory transactions to/from preserved devices routed
     consistently by adopting Access Control Services (ACS) controls
     across a Live Update, from the endpoint up to the root port.

  7. Freeze each device's preservation status when it is shut down, and
     avoid disabling bus mastering on preserved devices in the PCI
     shutdown path, so preserved devices can keep doing DMA during
     kexec.

  8. Add Documentation/PCI/liveupdate.rst, built from the kernel-doc in
     drivers/pci/liveupdate.c and include/linux/kho/abi/pci.h, describing
     the FLB API, device tracking, and the division of responsibilities
     between the PCI core, drivers, and userspace.

Merge Plan
----------

This series is intended to go through liveupdate.git, the tree listed in
the new PCI LIVE UPDATE MAINTAINERS entry.

v10 and a follow-up patch were applied to the lu-pci-preservation branch
of liveupdate.git. This version is meant to replace them, and applies on
v7.4-rc1, the base of that branch.

Testing
-------

VFIO
~~~~

This series was tested with v5 of the vfio-pci Live Update series,
rebased onto this series and the luo-internal-api branch of
liveupdate.git (which the VFIO series depends on):

  https://lore.kernel.org/kvm/20260714151505.3466855-1-vipinsh@google.com/

The full set of patches used for testing can be found on GitHub.

  https://github.com/dmatlack/linux/tree/liveupdate/pci/base/v11-with-vfio

Testing was performed using the VFIO selftests, including the new Live
Update selftests:

  - vfio_pci_liveupdate_uapi_test
  - vfio_pci_liveupdate_kexec_test

The tests were run in a QEMU-based VM environment (using an emulated
Intel IGB device connected to a PCIe Root Port to exercise bridge
support) and on bare metal using an Intel Emerald Rapids server with 8x
Intel DSA PCIe devices and 1x NVMe device, all of which were preserved
across a Live Update by vfio_pci_liveupdate_kexec_test.

PCI Selftests
~~~~~~~~~~~~~

This series was also tested with a new set of PCI Live Update
selftests:

  https://github.com/dmatlack/linux/tree/liveupdate/pci/base/v11-with-selftests

Each test preserves a PCI device across a Live Update in a QEMU VM and
checks that the PCI core keeps the device, and every bridge above it,
running: same BDF and bus numbers, Bus Master Enable left set, ARI
Forwarding Enable and ACS controls adopted from the previous kernel, and
no PCI config writes that disrupt the device during the Live Update. The
tests cover endpoints on the root bus, below a Root Port, below a
switch, and below two levels of switches, as well as booting the next
kernel with command lines that would otherwise renumber buses or change
ARI and ACS. All tests pass on x86_64 (q35) and arm64 (virt), except
for the pci=assign-busses test, which is skipped on arm64 (the option is
x86-only).

Future Work
-----------

Please see the PCI core Live Update Roadmap:

  https://lore.kernel.org/linux-pci/20261001232133.560284-1-dmatlack@google.com/

Changelog
---------

v11:
 - Rebase onto v7.4-rc1, the base of lu-pci-preservation in
   liveupdate.git
 - Fold in "PCI: liveupdate: Factor out pci_dev_ser allocation and
   release", which was posted and applied as a follow-up to v10 [1]:
   introduce the struct pci_dev_ser allocation and release helpers in
   "Track outgoing preserved PCI devices", and split
   pci_liveupdate_preserve_device() into separate paths for new and
   already-preserved devices in "Auto-preserve upstream bridges across
   Live Update" (suggested by Bjorn; the follow-up was reviewed and
   tested by Chris Down)
 - Don't allocate an ACS save buffer for devices that rely on
   device-specific ACS quirks (Alex)
 - Adopt the ACS controls of preserved devices even if they match a
   device-specific ACS quirk, logging a warning instead of programming
   ACS from scratch, and panic if a preserved device has no ACS save
   buffer (Alex)
 - Clean up commit messages, comments and documentation: fix
   inaccuracies, remove filler and duplicated text, document that
   pci=noari does not override adopted ARI Forwarding Enable, and note
   in the documentation that devices cannot be preserved or unpreserved
   once they are shut down
 - Panic if any outgoing device was never unpreserved when the outgoing
   FLB is freed, instead of leaving dangling dev->liveupdate.outgoing
   pointers, and share the check with the incoming side (Sashiko)

[1] https://lore.kernel.org/linux-pci/20261109174009.735309-1-dmatlack@google.com/

v10: https://lore.kernel.org/linux-pci/20261013162529.1928093-1-dmatlack@google.com/
v9: https://lore.kernel.org/linux-pci/20260918200640.887030-1-dmatlack@google.com/
v8: https://lore.kernel.org/linux-pci/20260728221007.2098560-1-dmatlack@google.com/
v7: https://lore.kernel.org/linux-pci/20260710212616.1351130-1-dmatlack@google.com/
v6: https://lore.kernel.org/linux-pci/20260522202410.3104264-1-dmatlack@google.com/
v5: https://lore.kernel.org/linux-pci/20260512184846.119396-1-dmatlack@google.com/
v4: https://lore.kernel.org/linux-pci/20260423212316.3431746-1-dmatlack@google.com/
v3: https://lore.kernel.org/kvm/20260323235817.1960573-1-dmatlack@google.com/
v2: https://lore.kernel.org/kvm/20260129212510.967611-1-dmatlack@google.com/
v1: https://lore.kernel.org/kvm/20251126193608.2678510-1-dmatlack@google.com/
rfc: https://lore.kernel.org/kvm/20251018000713.677779-1-vipinsh@google.com/


David Matlack (13):
  PCI: liveupdate: Set up FLB handler for the PCI core
  PCI: liveupdate: Track outgoing preserved PCI devices
  PCI: liveupdate: Track incoming preserved PCI devices
  PCI: liveupdate: Document driver binding responsibilities
  PCI: liveupdate: Auto-preserve upstream bridges across Live Update
  PCI: liveupdate: Preserve bus numbers during Live Update
  PCI: Refactor matching logic for pci_dev_acs_ops
  PCI: Save and restore the ACS Control register and Egress Control
    Vector
  PCI: liveupdate: Adopt ACS controls in incoming preserved devices
  PCI: liveupdate: Adopt ARI Forwarding Enable on preserved bridges
  PCI: liveupdate: Freeze preservation status during shutdown
  PCI: liveupdate: Do not disable bus mastering on preserved devices
    during kexec
  Documentation: PCI: Add documentation for Live Update

 Documentation/PCI/index.rst                   |    1 +
 Documentation/PCI/liveupdate.rst              |   35 +
 .../admin-guide/kernel-parameters.txt         |   11 +-
 Documentation/core-api/liveupdate.rst         |    5 +
 MAINTAINERS                                   |   15 +
 drivers/pci/Kconfig                           |   14 +
 drivers/pci/Makefile                          |    1 +
 drivers/pci/liveupdate.c                      | 1011 +++++++++++++++++
 drivers/pci/liveupdate.h                      |   72 ++
 drivers/pci/pci-driver.c                      |    8 +-
 drivers/pci/pci.c                             |  169 ++-
 drivers/pci/pci.h                             |    5 +
 drivers/pci/probe.c                           |   16 +-
 drivers/pci/quirks.c                          |   58 +-
 include/linux/kho/abi/pci.h                   |   96 ++
 include/linux/pci.h                           |    7 +
 include/linux/pci_liveupdate.h                |   72 ++
 17 files changed, 1556 insertions(+), 40 deletions(-)
 create mode 100644 Documentation/PCI/liveupdate.rst
 create mode 100644 drivers/pci/liveupdate.c
 create mode 100644 drivers/pci/liveupdate.h
 create mode 100644 include/linux/kho/abi/pci.h
 create mode 100644 include/linux/pci_liveupdate.h


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.56.1.703.gf848ed807c-goog


^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2026-11-18 22:37 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 22:36 [PATCH v11 00/13] PCI: liveupdate: PCI core support for Live Update David Matlack
2026-11-18 22:36 ` [PATCH v11 01/13] PCI: liveupdate: Set up FLB handler for the PCI core David Matlack
2026-11-18 22:36 ` [PATCH v11 02/13] PCI: liveupdate: Track outgoing preserved PCI devices David Matlack
2026-11-18 22:36 ` [PATCH v11 03/13] PCI: liveupdate: Track incoming " David Matlack
2026-11-18 22:36 ` [PATCH v11 04/13] PCI: liveupdate: Document driver binding responsibilities David Matlack
2026-11-18 22:36 ` [PATCH v11 05/13] PCI: liveupdate: Auto-preserve upstream bridges across Live Update David Matlack
2026-11-18 22:36 ` [PATCH v11 06/13] PCI: liveupdate: Preserve bus numbers during " David Matlack
2026-11-18 22:36 ` [PATCH v11 07/13] PCI: Refactor matching logic for pci_dev_acs_ops David Matlack
2026-11-18 22:36 ` [PATCH v11 08/13] PCI: Save and restore the ACS Control register and Egress Control Vector David Matlack
2026-11-18 22:37 ` [PATCH v11 09/13] PCI: liveupdate: Adopt ACS controls in incoming preserved devices David Matlack
2026-11-18 22:37 ` [PATCH v11 10/13] PCI: liveupdate: Adopt ARI Forwarding Enable on preserved bridges David Matlack
2026-11-18 22:37 ` [PATCH v11 11/13] PCI: liveupdate: Freeze preservation status during shutdown David Matlack
2026-11-18 22:37 ` [PATCH v11 12/13] PCI: liveupdate: Do not disable bus mastering on preserved devices during kexec David Matlack
2026-11-18 22:37 ` [PATCH v11 13/13] Documentation: PCI: Add documentation for Live Update David Matlack

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox