Hello,
This series can be found on Github:
https://github.com/shvipin/linux vfio/liveupdate/v5
This series provides the base support to enable Live Update feature for
VFIO PCI devices using the Live Update Orchestrator (LUO) and Kexec
Handover (KHO).
Overview
--------
This series makes the following changes to support VFIO Live Update:
- Puts VFIO Live Update support behind the CONFIG_VFIO_PCI_LIVEUPDATE
config option (which requires CONFIG_VFIO_DEVICE_CDEV).
- Preserves VFIO cdev device FDs across a kexec Live Update session via
LUO/KHO.
- Enforces that incoming preserved devices can only be retrieved via
LIVEUPDATE_SESSION_RETRIEVE_FD post-kexec (preventing legacy
group/cdev opens from accessing uninitialized state).
- Quiesces the device prior to kexec in freeze() by resetting hardware,
disabling bus mastering, and marking the device frozen to reject
post-freeze IO/MMIO.
- Includes VFIO selftests (uapi & kexec tests with continuous DMA
checks) and kernel documentation.
VFIO PCI Live Update Roadmap
----------------------------
We are planning to add VFIO PCI Live Update support in phases. Since full
support of this feature will come in phase, VFIO PCI Liveupdate is
controlled by CONFIG_VFIO_PCI_LIVEUPDATE config option.
This series is phase 1 below:
1. PCIe function (non-VF) preserved across Live Update without changing
BDF
2. PCIe function (non-VF) preserved across Live Update without resetting
3. PCIe function (non-VF) preserved across Live Update with continuous
DMA (noiommu)
4. PCIe function (non-VF) preserved across Live Update with continuous
DMA (iommufd)
5. PCIe VF preserved across Live Update with continuous DMA (iommufd)
Dependencies
------------
This series has following dependencies as shown in the diagram
|---------------------------|
| VFIO v5 | <--- Sent in this series.
|---------------------------|
| Liveupdate Selftests[1] |
|---------------------------|
| PCI v7 [2] |
|---------------------------|
[1] https://lore.kernel.org/all/20260511201155.1488670-1-vipinsh@google.com/
These selftests changes are needed by other selftests also (iommu,
kvm), so, I have sent them separately.
[2] https://lore.kernel.org/linux-pci/20260710212616.1351130-1-dmatlack@google.com/
This is the PCI v7, which adds the live update support in PCI core.
Github branch I have mentioned at the top has all of these dependencies.
Testing
-------
Tested using VFIO Live Update selftests in both QEMU and bare-metal
environment (Intel DSA PCIe device).
I want to make tests VFIO Live Update selftests in QEMU. But how to select a PCIe device?
what device do the PCI IDÂ 0000:00:04.0 and 0000:6a:01.0 mean in the following testcase?
Zhu Yanjun
$ ./setup.sh 0000:00:04.0
$ ./vfio_pci_liveupdate_kexec_test --stage 1 0000:6a:01.0
$ kexec --reuse-cmdline -l -s /boot/vmlinuz
$ kexec -e
$ ./setup.sh 0000:00:04.0
$ ./vfio_pci_liveupdate_kexec_test --stage 2 0000:6a:01.0
Kernel command line options used:
- liveupdate=on
- kho=on
- vfio_iommu_type1.allow_unsafe_interrupts=Y
- vfio_pci.enable_sriov=Y
- vfio_pci.disable_denylist=Y
- vfio.enable_unsafe_noiommu_mode=Y
Changelog
---------
v5:
- Addressed Sashiko feedback.
- Removed condition to disable CONFIG_VFIO_PCI_DMABUF.
- Expanded vfio_find_device() helper to safely discover a registered
vfio_device by matching criteria. Internal Sashiko instance found
data race.
- Updated selftests to latest changes in VFIO selftests.
- Rebased on top of PCI Live Update series v7.
v4: https://lore.kernel.org/kvm/20260511234802.2280368-1-vipinsh@google.com/
v3: https://lore.kernel.org/kvm/20260323235817.1960573-1-dmatlack@google.com/
v2: https://lore.kernel.org/kvm/20260129212510.967611-1-dmatlack@google.com/
v1: https://lore.kernel.org/kvm/20251126193608.2678510-1-dmatlack@google.com/
rfc: https://lore.kernel.org/kvm/20251018000713.677779-1-vipinsh@google.com/
David Matlack (7):
vfio: Enforce preserved devices are retrieved via
LIVEUPDATE_SESSION_RETRIEVE_FD
docs: liveupdate: Add documentation for VFIO PCI
vfio: selftests: Add vfio_pci_liveupdate_uapi_test
vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED
vfio: selftests: Expose iommu_modes to tests
vfio: selftests: Verify that opening VFIO device fails during Live
Update
vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test
Vipin Sharma (13):
vfio/pci: Factor out the reset logic in VFIO PCI device close path
vfio: Export various helpers from VFIO
vfio/pci: Export vfio_pci_dma_buf_move for vfio-pci module
liveupdate: Export symbols needed by modules
vfio/pci: Register a file handler with Live Update Orchestrator
vfio/pci: Preserve vfio-pci device files across Live Update
vfio: Introduce vfio_find_device() helper
vfio: Refactor vfio_device_fops_cdev_open()
vfio: Add API to open cdev device for Live Update restore
vfio/pci: Retrieve preserved device files after Live Update
vfio: selftests: Build liveupdate library in VFIO selftests
vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD
vfio: selftests: Add vfio_pci_liveupdate_kexec_test
Documentation/core-api/liveupdate.rst | 1 +
.../driver-api/vfio_pci_liveupdate.rst | 23 ++
MAINTAINERS | 2 +
drivers/vfio/device_cdev.c | 67 +++-
drivers/vfio/group.c | 9 +
drivers/vfio/pci/Kconfig | 12 +
drivers/vfio/pci/Makefile | 1 +
drivers/vfio/pci/vfio_pci.c | 13 +-
drivers/vfio/pci/vfio_pci_config.c | 3 +
drivers/vfio/pci/vfio_pci_core.c | 119 +++---
drivers/vfio/pci/vfio_pci_dmabuf.c | 1 +
drivers/vfio/pci/vfio_pci_liveupdate.c | 340 ++++++++++++++++++
drivers/vfio/pci/vfio_pci_priv.h | 28 ++
drivers/vfio/vfio.h | 18 +
drivers/vfio/vfio_main.c | 67 +++-
include/linux/kho/abi/vfio_pci.h | 43 +++
include/linux/vfio.h | 12 +
include/linux/vfio_pci_core.h | 3 +
kernel/liveupdate/luo_file.c | 2 +
tools/testing/selftests/vfio/Makefile | 23 +-
.../vfio/lib/include/libvfio/iommu.h | 2 +
.../lib/include/libvfio/vfio_pci_device.h | 3 +
tools/testing/selftests/vfio/lib/iommu.c | 4 +-
.../selftests/vfio/lib/vfio_pci_device.c | 25 +-
.../vfio/vfio_pci_liveupdate_kexec_test.c | 256 +++++++++++++
.../vfio/vfio_pci_liveupdate_uapi_test.c | 97 +++++
26 files changed, 1097 insertions(+), 77 deletions(-)
create mode 100644 Documentation/driver-api/vfio_pci_liveupdate.rst
create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c
create mode 100644 include/linux/kho/abi/vfio_pci.h
create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c
create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c
base-commit: cbf37fff757e87376e80e11a8378393a8f25145f
prerequisite-patch-id: 85705fb54d3065efe1d87ab4b69e828a9f3404e7
prerequisite-patch-id: a33ae79b5fc0259bd013b430c18cc3d14a678599