[PATCH v4 00/16] vfio/pci: Base Live Update support for VFIO

[Vipin Sharma <vipinsh@google.com>]

Hello,

This series can be found on Github:

  https://github.com/shvipin/linux vfio/liveupdate/v4

It adds the base support to enable Live Update feature for VFIO devices.

In v3 of this series, both core PCI subsystem and VFIO changes were
together. Based on the feedbacks on the patches, increasing
complexities and the overall number of patches, we have split the effort
in two parts, PCI and VFIO.

v4 version of PCI work is here:
  https://lore.kernel.org/linux-pci/20260423212316.3431746-1-dmatlack@google.com/

This series is the VFIO part, built on top of PCI v4. Going forward,
work on both subsystem will be independent and we may not be making same
version releases in sync. However, VFIO changes will always be on the top of
some version of PCI changes.

Overview
--------

This series made following changes to support VFIO live update:

  1. VFIO Live Update feature is put behind VFIO_PCI_LIVEUPDATE config
     option. It also requires that VFIO_DEVICE_CDEV config is enabled.
  1. To keep things simple in this series, enabling this option will
     disable VFIO_PCI_DMABUF. This is temporary.
  1. A VFIO cdev FD can be preserved via Live Update session and
     retrieved after kexec via the same session.
  2. Getting the preserved device FD after kexec via legacy VFIO group
     APIs or by directly opening /dev/vfio/devices/vfioX will error out.
     User has to get the FD from Live Update subsystem.
  3. For now, only FD is preserved, underlying device and its states are
     not.
  4. To keep the next kernel safe, just before the kexec, VFIO resets
     the device, disables BME and restores the original state of the
     device. This is temporary and will be removed when we have added
     full support of the Live Update.
  5. New VFIO selftests have been added which interacts with VFIO
     - vfio_pci_liveupdate_uapi_test
     - vfio_pci_liveupdate_kexec_test
     liveupdate.
  6. Documentations have been added for kernel docs.

Dependencies
------------

This series has following dependencies as shown in the diagram

    |---------------------------|
    |         VFIO v4           |
    |---------------------------|
    |  Liveupdate Selftests[1]  |
    |---------------------------|
    |        PCI v4 [2]         |
    |---------------------------|
    |  Liveupdate Refcount [3]  |
    |---------------------------|

[1] https://lore.kernel.org/all/20260511201155.1488670-1-vipinsh@google.com/
  These selftests changes are needed by other selftests also (iommu, kvm),
  so, I have sent them separately.

[2] https://lore.kernel.org/linux-pci/20260423212316.3431746-1-dmatlack@google.com/
  This is the PCI v4, which adds the live update support in PCI core.

[3] https://lore.kernel.org/lkml/20260423174032.3140399-1-dmatlack@google.com/
  PCI v4 is dependent on this.

Github branch I have mentioned at the top has all of these dependencies.

Testing
-------

Series has been tested using new VFIO live update selftests in both qemu and
bare-metal environment. In bare-metal environment, it was ran using the
Intel DSA PCIe device.

vfio_pci_liveupdate_kexec_test test is executed as following

  $ ./setup.sh 0000:00:04.0
  $ ./vfio_pci_liveupdate_kexec_test --stage 1 0000:6a:01.0
  $ kexec --reuse-cmdline -l -s /boot/vmlinuz
  $ kexec -e
  $ ./setup.sh 0000:00:04.0
  $ ./vfio_pci_liveupdate_kexec_test --stage 2 0000:6a:01.0

Kernel commandline used for booting the VM and testing had following
options passed
  - liveupdate=on
  - kho=on
  - vfio_iommu_type1.allow_unsafe_interrupts=Y
  - vfio_pci.enable_sriov=Y
  - vfio_pci.disable_denylist=Y
  - vfio.enable_unsafe_noiommu_mode=Y

Changelog
---------
v4:
 - Addressed sashiko feedback regarding issues in resetting the devize
   in freeze
 - Dropped the patch which was skipping reset of preserved VFIO device
   after kexec when user opens the device. This will be added back in
   future but for now not needed as freeze() is also resetting the
   device before kexec.
 - Kernel doc and comment updates based on feedbacks.

v3: https://lore.kernel.org/kvm/20260323235817.1960573-1-dmatlack@google.com/

v2: https://lore.kernel.org/kvm/20260129212510.967611-1-dmatlack@google.com/

v1: https://lore.kernel.org/kvm/20251126193608.2678510-1-dmatlack@google.com/

rfc: https://lore.kernel.org/kvm/20251018000713.677779-1-vipinsh@google.com/


David Matlack (10):
  vfio/pci: Notify PCI subsystem about devices preserved across Live
    Update
  vfio: Enforce preserved devices are retrieved via
    LIVEUPDATE_SESSION_RETRIEVE_FD
  vfio/pci: Store incoming Live Update state in struct
    vfio_pci_core_device
  docs: liveupdate: Add documentation for VFIO PCI
  vfio: selftests: Add vfio_pci_liveupdate_uapi_test
  vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED
  vfio: selftests: Expose iommu_modes to tests
  vfio: selftests: Expose low-level helper routines for setting up
    struct vfio_pci_device
  vfio: selftests: Verify that opening VFIO device fails during Live
    Update
  vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test

Vipin Sharma (6):
  vfio/pci: Register a file handler with Live Update Orchestrator
  vfio/pci: Preserve vfio-pci device files across Live Update
  vfio/pci: Retrieve preserved device files after Live Update
  vfio: selftests: Build liveupdate library in VFIO selftests
  vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD
  vfio: selftests: Add vfio_pci_liveupdate_kexec_test

 Documentation/core-api/liveupdate.rst         |   1 +
 .../driver-api/vfio_pci_liveupdate.rst        |  23 ++
 MAINTAINERS                                   |   2 +
 drivers/vfio/device_cdev.c                    |  67 +++-
 drivers/vfio/group.c                          |   9 +
 drivers/vfio/pci/Kconfig                      |  16 +-
 drivers/vfio/pci/Makefile                     |   1 +
 drivers/vfio/pci/vfio_pci.c                   |  13 +-
 drivers/vfio/pci/vfio_pci_core.c              |  62 ++--
 drivers/vfio/pci/vfio_pci_liveupdate.c        | 349 ++++++++++++++++++
 drivers/vfio/pci/vfio_pci_priv.h              |  19 +
 drivers/vfio/vfio.h                           |  18 +
 drivers/vfio/vfio_main.c                      |  16 +-
 include/linux/kho/abi/vfio_pci.h              |  43 +++
 include/linux/vfio.h                          |  13 +
 include/linux/vfio_pci_core.h                 |   2 +
 tools/testing/selftests/vfio/Makefile         |  23 +-
 .../vfio/lib/include/libvfio/iommu.h          |   2 +
 .../lib/include/libvfio/vfio_pci_device.h     |   8 +
 tools/testing/selftests/vfio/lib/iommu.c      |   4 +-
 .../selftests/vfio/lib/vfio_pci_device.c      |  60 ++-
 .../vfio/vfio_pci_liveupdate_kexec_test.c     | 256 +++++++++++++
 .../vfio/vfio_pci_liveupdate_uapi_test.c      |  93 +++++
 23 files changed, 1041 insertions(+), 59 deletions(-)
 create mode 100644 Documentation/driver-api/vfio_pci_liveupdate.rst
 create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c
 create mode 100644 include/linux/kho/abi/vfio_pci.h
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c


base-commit: 2a4c0c11c0193889446cdb6f1540cc2b9aff97dd
-- 
2.54.0.563.g4f69b47b94-goog