Re: [PATCH Kernel v19 7/8] vfio iommu: Add migration capability to report supported features

Alex Williamson
2020-05-14T05:02:06Z

On Thu, 14 May 2020 01:34:38 +0530
Kirti Wankhede <kwankhede@nvidia.com> wrote:

> Added migration capability in IOMMU info chain.
> User application should check IOMMU info chain for migration capability
> to use dirty page tracking feature provided by kernel module.
> User application must check page sizes supported and maximum dirty
> bitmap size returned by this capability structure for ioctls used to get
> dirty bitmap.
>
> Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> ---
> drivers/vfio/vfio_iommu_type1.c | 24 +++++++++++++++++++++++-
> include/uapi/linux/vfio.h | 21 +++++++++++++++++++++
> 2 files changed, 44 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c
> index 4358be26ff80..77351497a9c2 100644
> --- a/drivers/vfio/vfio_iommu_type1.c
> +++ b/drivers/vfio/vfio_iommu_type1.c
> @@ -2389,6 +2389,22 @@ static int vfio_iommu_iova_build_caps(struct vfio_iommu *iommu,
> return ret;
> }
>
> +static int vfio_iommu_migration_build_caps(struct vfio_iommu *iommu,
> + struct vfio_info_cap *caps)
> +{
> + struct vfio_iommu_type1_info_cap_migration cap_mig;
> +
> + cap_mig.header.id = VFIO_IOMMU_TYPE1_INFO_CAP_MIGRATION;
> + cap_mig.header.version = 1;
> + cap_mig.flags = VFIO_IOMMU_INFO_CAPS_MIGRATION_DIRTY_PAGE_TRACK;
> +
> + /* support minimum pgsize */
> + cap_mig.pgsize_bitmap = (size_t)1 << __ffs(iommu->pgsize_bitmap);
> + cap_mig.max_dirty_bitmap_size = DIRTY_BITMAP_SIZE_MAX;
> +
> + return vfio_info_add_capability(caps, &cap_mig.header, sizeof(cap_mig));
> +}
> +
> static long vfio_iommu_type1_ioctl(void *iommu_data,
> unsigned int cmd, unsigned long arg)
> {
> @@ -2433,10 +2449,16 @@ static long vfio_iommu_type1_ioctl(void *iommu_data,
> mutex_lock(&iommu->lock);
> info.flags = VFIO_IOMMU_INFO_PGSIZES;
>
> + vfio_pgsize_bitmap(iommu);

Why is it necessary to rebuild the bitmap here? The user can't get to
this ioctl until they've added a group to the container and set the
IOMMU model.

> info.iova_pgsizes = iommu->pgsize_bitmap;
>
> - ret = vfio_iommu_iova_build_caps(iommu, &caps);
> + ret = vfio_iommu_migration_build_caps(iommu, &caps);
> +
> + if (!ret)
> + ret = vfio_iommu_iova_build_caps(iommu, &caps);
> +
> mutex_unlock(&iommu->lock);
> +
> if (ret)
> return ret;
>
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index e3cbf8b78623..c90604322798 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -1013,6 +1013,27 @@ struct vfio_iommu_type1_info_cap_iova_range {
> struct vfio_iova_range iova_ranges[];
> };
>
> +/*
> + * The migration capability allows to report supported features for migration.
> + *
> + * The structures below define version 1 of this capability.
> + *
> + * pgsize_bitmap: Kernel driver returns supported page sizes bitmap for dirty
> + * page tracking.
> + * max_dirty_bitmap_size: Kernel driver returns maximum supported dirty bitmap
> + * size in bytes to be used by user application for ioctls to get dirty bitmap.
> + */
> +#define VFIO_IOMMU_TYPE1_INFO_CAP_MIGRATION 1
> +
> +struct vfio_iommu_type1_info_cap_migration {
> + struct vfio_info_cap_header header;
> + __u32 flags;
> + /* supports dirty page tracking */
> +#define VFIO_IOMMU_INFO_CAPS_MIGRATION_DIRTY_PAGE_TRACK (1 << 0)

This flag is a bit redundant to the purpose of this capability, isn't
it? I think exposing the capability itself is indicating support for
dirty page tracking. We should probably be explicit in the comment
about exactly what interface this capability implies. Thanks,

Alex

> + __u64 pgsize_bitmap;
> + __u64 max_dirty_bitmap_size; /* in bytes */
> +};
> +
> #define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12)
>
> /**