Re: [PATCH vhost v4 00/15] vdpa/mlx5: Add support for resumable vqs

[Dragos Tatulea <dtatulea@nvidia.com>]

On Mon, 2023-12-25 at 09:41 -0500, Michael S. Tsirkin wrote:
> On Tue, Dec 19, 2023 at 08:08:43PM +0200, Dragos Tatulea wrote:
> > Add support for resumable vqs in the mlx5_vdpa driver. This is a
> > firmware feature that can be used for the following benefits:
> > - Full device .suspend/.resume.
> > - .set_map doesn't need to destroy and create new vqs anymore just to
> >   update the map. When resumable vqs are supported it is enough to
> >   suspend the vqs, set the new maps, and then resume the vqs.
> > 
> > The first patch exposes relevant bits for the feature in mlx5_ifc.h.
> > That means it needs to be applied to the mlx5-vhost tree [0] first. Once
> > applied there, the change has to be pulled from mlx5-vhost into the
> > vhost tree and only then the remaining patches can be applied. Same flow
> > as the vq descriptor mappings patchset [1].
> > 
> > The second part implements the vdpa backend feature support to allow
> > vq state and address changes when the device is in DRIVER_OK state and
> > suspended.
> > 
> > The third part adds support for seletively modifying vq parameters. This
> > is needed to be able to use resumable vqs.
> > 
> > Then the actual support for resumable vqs is added.
> > 
> > The last part of the series introduces reference counting for mrs which
> > is necessary to avoid freeing mkeys too early or leaking them.
> 
> 
> I lost track. Are you going to send v5 or not?
> 
I was waiting for your answer if I should send it or not. I suppose this means
yes. I will send it in a few minutes.

> > * Changes in v4:
> > - Added vdpa backend feature support for changing vq properties in
> >   DRIVER_OK when device is suspended. Added support in the driver as
> >   well.
> > - Dropped Acked-by for the patches that had the tag mistakenly
> >   added.
> > 
> > * Changes in v3:
> > - Faulty version. Please ignore.
> > 
> > * Changes in v2:
> > - Added mr refcounting patches.
> > - Deleted unnecessary patch: "vdpa/mlx5: Split function into locked and
> >   unlocked variants"
> > - Small print improvement in "Introduce per vq and device resume"
> >   patch.
> > - Patch 1/7 has been applied to mlx5-vhost branch.
> > 
> > 
> > Dragos Tatulea (15):
> >   vdpa: Add VHOST_BACKEND_F_CHANGEABLE_VQ_ADDR_IN_SUSPEND flag
> >   vdpa: Add VHOST_BACKEND_F_CHANGEABLE_VQ_STATE_IN_SUSPEND flag
> >   vdpa: Accept VHOST_BACKEND_F_CHANGEABLE_VQ_ADDR_IN_SUSPEND backend
> >     feature
> >   vdpa: Accept VHOST_BACKEND_F_CHANGEABLE_VQ_STATE_IN_SUSPEND backend
> >     feature
> >   vdpa: Track device suspended state
> >   vdpa: Block vq address change in DRIVER_OK unless device supports it
> >   vdpa: Block vq state change in DRIVER_OK unless device supports it
> >   vdpa/mlx5: Expose resumable vq capability
> >   vdpa/mlx5: Allow modifying multiple vq fields in one modify command
> >   vdpa/mlx5: Introduce per vq and device resume
> >   vdpa/mlx5: Mark vq addrs for modification in hw vq
> >   vdpa/mlx5: Mark vq state for modification in hw vq
> >   vdpa/mlx5: Use vq suspend/resume during .set_map
> >   vdpa/mlx5: Introduce reference counting to mrs
> >   vdpa/mlx5: Add mkey leak detection
> > 
> >  drivers/vdpa/mlx5/core/mlx5_vdpa.h |  10 +-
> >  drivers/vdpa/mlx5/core/mr.c        |  69 +++++++--
> >  drivers/vdpa/mlx5/net/mlx5_vnet.c  | 218 ++++++++++++++++++++++++++---
> >  drivers/vhost/vdpa.c               |  51 ++++++-
> >  include/linux/mlx5/mlx5_ifc.h      |   3 +-
> >  include/linux/mlx5/mlx5_ifc_vdpa.h |   4 +
> >  include/uapi/linux/vhost_types.h   |   8 ++
> >  7 files changed, 322 insertions(+), 41 deletions(-)
> > 
> > -- 
> > 2.43.0
>