Re: [PATCH vhost v4 00/15] vdpa/mlx5: Add support for resumable vqs

Dragos Tatulea
2023-12-25T15:05:36Z

On Mon, 2023-12-25 at 09:41 -0500, Michael S. Tsirkin wrote:
> On Tue, Dec 19, 2023 at 08:08:43PM +0200, Dragos Tatulea wrote:
> > Add support for resumable vqs in the mlx5_vdpa driver. This is a
> > firmware feature that can be used for the following benefits:
> > - Full device .suspend/.resume.
> > - .set_map doesn't need to destroy and create new vqs anymore just to
> > update the map. When resumable vqs are supported it is enough to
> > suspend the vqs, set the new maps, and then resume the vqs.
> >
> > The first patch exposes relevant bits for the feature in mlx5_ifc.h.
> > That means it needs to be applied to the mlx5-vhost tree [0] first. Once
> > applied there, the change has to be pulled from mlx5-vhost into the
> > vhost tree and only then the remaining patches can be applied. Same flow
> > as the vq descriptor mappings patchset [1].
> >
> > The second part implements the vdpa backend feature support to allow
> > vq state and address changes when the device is in DRIVER_OK state and
> > suspended.
> >
> > The third part adds support for seletively modifying vq parameters. This
> > is needed to be able to use resumable vqs.
> >
> > Then the actual support for resumable vqs is added.
> >
> > The last part of the series introduces reference counting for mrs which
> > is necessary to avoid freeing mkeys too early or leaking them.
>
>
> I lost track. Are you going to send v5 or not?
>
I was waiting for your answer if I should send it or not. I suppose this means
yes. I will send it in a few minutes.

> > * Changes in v4:
> > - Added vdpa backend feature support for changing vq properties in
> > DRIVER_OK when device is suspended. Added support in the driver as
> > well.
> > - Dropped Acked-by for the patches that had the tag mistakenly
> > added.
> >
> > * Changes in v3:
> > - Faulty version. Please ignore.
> >
> > * Changes in v2:
> > - Added mr refcounting patches.
> > - Deleted unnecessary patch: "vdpa/mlx5: Split function into locked and
> > unlocked variants"
> > - Small print improvement in "Introduce per vq and device resume"
> > patch.
> > - Patch 1/7 has been applied to mlx5-vhost branch.
> >
> >
> > Dragos Tatulea (15):
> > vdpa: Add VHOST_BACKEND_F_CHANGEABLE_VQ_ADDR_IN_SUSPEND flag
> > vdpa: Add VHOST_BACKEND_F_CHANGEABLE_VQ_STATE_IN_SUSPEND flag
> > vdpa: Accept VHOST_BACKEND_F_CHANGEABLE_VQ_ADDR_IN_SUSPEND backend
> > feature
> > vdpa: Accept VHOST_BACKEND_F_CHANGEABLE_VQ_STATE_IN_SUSPEND backend
> > feature
> > vdpa: Track device suspended state
> > vdpa: Block vq address change in DRIVER_OK unless device supports it
> > vdpa: Block vq state change in DRIVER_OK unless device supports it
> > vdpa/mlx5: Expose resumable vq capability
> > vdpa/mlx5: Allow modifying multiple vq fields in one modify command
> > vdpa/mlx5: Introduce per vq and device resume
> > vdpa/mlx5: Mark vq addrs for modification in hw vq
> > vdpa/mlx5: Mark vq state for modification in hw vq
> > vdpa/mlx5: Use vq suspend/resume during .set_map
> > vdpa/mlx5: Introduce reference counting to mrs
> > vdpa/mlx5: Add mkey leak detection
> >
> > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 10 +-
> > drivers/vdpa/mlx5/core/mr.c | 69 +++++++--
> > drivers/vdpa/mlx5/net/mlx5_vnet.c | 218 ++++++++++++++++++++++++++---
> > drivers/vhost/vdpa.c | 51 ++++++-
> > include/linux/mlx5/mlx5_ifc.h | 3 +-
> > include/linux/mlx5/mlx5_ifc_vdpa.h | 4 +
> > include/uapi/linux/vhost_types.h | 8 ++
> > 7 files changed, 322 insertions(+), 41 deletions(-)
> >
> > --
> > 2.43.0
>