RE: [PATCH v9 0/9] vfio/hisilicon: add ACC live migration driver

[Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>]

Hi Alex,

> -----Original Message-----
> From: Shameerali Kolothum Thodi
> Sent: 08 March 2022 18:49
> To: kvm@vger.kernel.org; linux-kernel@vger.kernel.org;
> linux-crypto@vger.kernel.org
> Cc: linux-pci@vger.kernel.org; alex.williamson@redhat.com; jgg@nvidia.com;
> cohuck@redhat.com; mgurtovoy@nvidia.com; yishaih@nvidia.com;
> kevin.tian@intel.com; Linuxarm <linuxarm@huawei.com>; liulongfang
> <liulongfang@huawei.com>; Zengtao (B) <prime.zeng@hisilicon.com>;
> Jonathan Cameron <jonathan.cameron@huawei.com>; Wangzhou (B)
> <wangzhou1@hisilicon.com>
> Subject: [PATCH v9 0/9] vfio/hisilicon: add ACC live migration driver
> 
> Hi,
> 
> This series attempts to add vfio live migration support for HiSilicon
> ACC VF devices based on the new v2 migration protocol definition and
> mlx5 v9 series discussed here[0].
> 
> v8 --> v9
>  - Added acks by Wangzhou/Longfang/Yekai
>  - Added R-by tags by Jason.
>  - Addressed comments by Alex on v8.
>  - Fixed the pf_queue pointer assignment error in patch #8.
>  - Addressed comments from Kevin,
>     -Updated patch #5 commit log msg with a clarification that VF
>      migration BAR assignment is fine if migration support is not there.
>     -Added QM description to patch #8 commit msg.

Hope there is nothing pending for this series now to make it to next.
I know ack from Bjorn is still pending for patch #3, and I have
sent a ping last week and also CCd him on that patch.

Please let me know if there is anything I missed. 

Thanks,
Shameer


> 
> This is sanity tested on a HiSilicon platform using the Qemu branch
> provided here[1].
> 
> Please take a look and let me know your feedback.
> 
> Thanks,
> Shameer
> [0]
> https://lore.kernel.org/kvm/20220224142024.147653-1-yishaih@nvidia.com/
> [1] https://github.com/jgunthorpe/qemu/commits/vfio_migration_v2
> 
> v7 --> v8
>  - Dropped PRE_COPY support and early compatibility checking based on
>    the discussion here[1].
>  - Addressed comments from John, Jason & Alex (Thanks!).
> 
> v6 --> v7
>  -Renamed MIG_PRECOPY ioctl name and struct name. Updated ioctl
> descriptions
>   regarding ioctl validity (patch #7).
> - Adressed comments from Jason and Alex on PRE_COPY read() and ioctl() fns
>   (patch #9).
> - Moved only VF PCI ids to pci_ids.h(patch #3).
> 
> v5 --> v6
>  -Report PRE_COPY support and use that for early compatibility check
>   between src and dst devices.
>  -For generic PRE_COPY support, included patch #7 from Jason(Thanks!).
>  -Addressed comments from Alex(Thanks!).
>  -Added the QM state register update to QM driver(patch #8) since that
>   is being used in migration driver to decide whether the device is
>   ready to save the state.
> 
> RFCv4 --> v5
>   - Dropped RFC tag as v2 migration APIs are more stable now.
>   - Addressed review comments from Jason and Alex (Thanks!).
> 
> v3 --> RFCv4
> -Based on migration v2 protocol and mlx5 v7 series.
> -Added RFC tag again as migration v2 protocol is still under discussion.
> -Added new patch #6 to retrieve the PF QM data.
> -PRE_COPY compatibility check is now done after the migration data
>  transfer. This is not ideal and needs discussion.
> 
> RFC v2 --> v3
>  -Dropped RFC tag as the vfio_pci_core subsystem framework is now
>   part of 5.15-rc1.
>  -Added override methods for vfio_device_ops read/write/mmap calls
>   to limit the access within the functional register space.
>  -Patches 1 to 3 are code refactoring to move the common ACC QM
>   definitions and header around.
> 
> RFCv1 --> RFCv2
> 
>  -Adds a new vendor-specific vfio_pci driver(hisi-acc-vfio-pci)
>   for HiSilicon ACC VF devices based on the new vfio-pci-core
>   framework proposal.
> 
>  -Since HiSilicon ACC VF device MMIO space contains both the
>   functional register space and migration control register space,
>   override the vfio_device_ops ioctl method to report only the
>   functional space to VMs.
> 
>  -For a successful migration, we still need access to VF dev
>   functional register space mainly to read the status registers.
>   But accessing these while the Guest vCPUs are running may leave
>   a security hole. To avoid any potential security issues, we
>   map/unmap the MMIO regions on a need basis and is safe to do so.
>   (Please see hisi_acc_vf_ioremap/unmap() fns in patch #4).
> 
>  -Dropped debugfs support for now.
>  -Uses common QM functions for mailbox access(patch #3).
> 
> Longfang Liu (3):
>   crypto: hisilicon/qm: Move few definitions to common header
>   crypto: hisilicon/qm: Set the VF QM state register
>   hisi_acc_vfio_pci: Add support for VFIO live migration
> 
> Shameer Kolothum (6):
>   crypto: hisilicon/qm: Move the QM header to include/linux
>   hisi_acc_qm: Move VF PCI device IDs to common header
>   hisi_acc_vfio_pci: add new vfio_pci driver for HiSilicon ACC devices
>   hisi_acc_vfio_pci: Restrict access to VF dev BAR2 migration region
>   hisi_acc_vfio_pci: Add helper to retrieve the struct pci_driver
>   hisi_acc_vfio_pci: Use its own PCI reset_done error handler
> 
>  MAINTAINERS                                   |    7 +
>  drivers/crypto/hisilicon/hpre/hpre.h          |    2 +-
>  drivers/crypto/hisilicon/hpre/hpre_main.c     |   19 +-
>  drivers/crypto/hisilicon/qm.c                 |   68 +-
>  drivers/crypto/hisilicon/sec2/sec.h           |    2 +-
>  drivers/crypto/hisilicon/sec2/sec_main.c      |   21 +-
>  drivers/crypto/hisilicon/sgl.c                |    2 +-
>  drivers/crypto/hisilicon/zip/zip.h            |    2 +-
>  drivers/crypto/hisilicon/zip/zip_main.c       |   17 +-
>  drivers/vfio/pci/Kconfig                      |    2 +
>  drivers/vfio/pci/Makefile                     |    2 +
>  drivers/vfio/pci/hisilicon/Kconfig            |   15 +
>  drivers/vfio/pci/hisilicon/Makefile           |    4 +
>  .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c    | 1326 +++++++++++++++++
>  .../vfio/pci/hisilicon/hisi_acc_vfio_pci.h    |  116 ++
>  .../qm.h => include/linux/hisi_acc_qm.h       |   49 +
>  include/linux/pci_ids.h                       |    3 +
>  17 files changed, 1591 insertions(+), 66 deletions(-)
>  create mode 100644 drivers/vfio/pci/hisilicon/Kconfig
>  create mode 100644 drivers/vfio/pci/hisilicon/Makefile
>  create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
>  create mode 100644 drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.h
>  rename drivers/crypto/hisilicon/qm.h => include/linux/hisi_acc_qm.h (87%)
> 
> --
> 2.25.1