[PATCH v7 00/10] vfio/hisilicon: add ACC live migration driver

Shameer Kolothum
2022-03-02T17:30:25Z

Hi,

This series attempts to add vfio live migration support for HiSilicon
ACC VF devices based on the new v2 migration protocol definition and
mlx5 v9 series discussed here[0].

v6 --> v7
-Renamed MIG_PRECOPY ioctl name and struct name. Updated ioctl descriptions
regarding ioctl validity (patch #7).
- Adressed comments from Jason and Alex on PRE_COPY read() and ioctl() fns
(patch #9).
- Moved only VF PCI ids to pci_ids.h(patch #3).

This is sanity tested on a HiSilicon platform using the Qemu branch
provided here[1].

Please take a look and let me know your feedback.

Thanks,
Shameer
[0] https://lore.kernel.org/kvm/20220224142024.147653-1-yishaih@nvidia.com/
[1] https://github.com/jgunthorpe/qemu/commits/vfio_migration_v2

v5 --> v6
-Report PRE_COPY support and use that for early compatibility check
between src and dst devices.
-For generic PRE_COPY support, included patch #7 from Jason(Thanks!).
-Addressed comments from Alex(Thanks!).
-Added the QM state register update to QM driver(patch #8) since that
is being used in migration driver to decide whether the device is
ready to save the state.

RFCv4 --> v5
- Dropped RFC tag as v2 migration APIs are more stable now.
- Addressed review comments from Jason and Alex (Thanks!).

v3 --> RFCv4
-Based on migration v2 protocol and mlx5 v7 series.
-Added RFC tag again as migration v2 protocol is still under discussion.
-Added new patch #6 to retrieve the PF QM data.
-PRE_COPY compatibility check is now done after the migration data
transfer. This is not ideal and needs discussion.

RFC v2 --> v3
-Dropped RFC tag as the vfio_pci_core subsystem framework is now
part of 5.15-rc1.
-Added override methods for vfio_device_ops read/write/mmap calls
to limit the access within the functional register space.
-Patches 1 to 3 are code refactoring to move the common ACC QM
definitions and header around.

RFCv1 --> RFCv2

-Adds a new vendor-specific vfio_pci driver(hisi-acc-vfio-pci)
for HiSilicon ACC VF devices based on the new vfio-pci-core
framework proposal.

-Since HiSilicon ACC VF device MMIO space contains both the
functional register space and migration control register space,
override the vfio_device_ops ioctl method to report only the
functional space to VMs.

-For a successful migration, we still need access to VF dev
functional register space mainly to read the status registers.
But accessing these while the Guest vCPUs are running may leave
a security hole. To avoid any potential security issues, we
map/unmap the MMIO regions on a need basis and is safe to do so.
(Please see hisi_acc_vf_ioremap/unmap() fns in patch #4).

-Dropped debugfs support for now.
-Uses common QM functions for mailbox access(patch #3).

Jason Gunthorpe (1):
vfio: Extend the device migration protocol with PRE_COPY

Longfang Liu (3):
crypto: hisilicon/qm: Move few definitions to common header
crypto: hisilicon/qm: Set the VF QM state register
hisi_acc_vfio_pci: Add support for VFIO live migration

Shameer Kolothum (6):
crypto: hisilicon/qm: Move the QM header to include/linux
hisi_acc_qm: Move VF PCI device IDs to common header
hisi_acc_vfio_pci: add new vfio_pci driver for HiSilicon ACC devices
hisi_acc_vfio_pci: Restrict access to VF dev BAR2 migration region
hisi_acc_vfio_pci: Add helper to retrieve the struct pci_driver
hisi_acc_vfio_pci: Use its own PCI reset_done error handler

drivers/crypto/hisilicon/hpre/hpre.h | 2 +-
drivers/crypto/hisilicon/hpre/hpre_main.c | 19 +-
drivers/crypto/hisilicon/qm.c | 42 +-
drivers/crypto/hisilicon/sec2/sec.h | 2 +-
drivers/crypto/hisilicon/sec2/sec_main.c | 21 +-
drivers/crypto/hisilicon/sgl.c | 2 +-
drivers/crypto/hisilicon/zip/zip.h | 2 +-
drivers/crypto/hisilicon/zip/zip_main.c | 17 +-
drivers/vfio/pci/Kconfig | 2 +
drivers/vfio/pci/Makefile | 2 +
drivers/vfio/pci/hisilicon/Kconfig | 16 +
drivers/vfio/pci/hisilicon/Makefile | 4 +
.../vfio/pci/hisilicon/hisi_acc_vfio_pci.c | 1369 +++++++++++++++++
.../vfio/pci/hisilicon/hisi_acc_vfio_pci.h | 114 ++
drivers/vfio/vfio.c | 71 +-
.../qm.h => include/linux/hisi_acc_qm.h | 49 +
include/linux/pci_ids.h | 3 +
include/uapi/linux/vfio.h | 113 +-
18 files changed, 1792 insertions(+), 58 deletions(-)
create mode 100644 drivers/vfio/pci/hisilicon/Kconfig
create mode 100644 drivers/vfio/pci/hisilicon/Makefile
create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h
rename drivers/crypto/hisilicon/qm.h => include/linux/hisi_acc_qm.h (87%)

--
2.25.1