* [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
@ 2023-09-28 16:33 Dragos Tatulea
0 siblings, 0 replies; 9+ messages in thread
From: Dragos Tatulea @ 2023-09-28 16:33 UTC (permalink / raw)
To: eperezma, gal, Jason Wang, Leon Romanovsky, Michael S. Tsirkin,
Saeed Mahameed, Xuan Zhuo
Cc: Dragos Tatulea, kvm, linux-kernel, linux-rdma, netdev,
virtualization
This patch series adds support for vq descriptor table mappings which
are used to improve vdpa live migration downtime. The improvement comes
from using smaller mappings which take less time to create and destroy
in hw.
The first part adds the vdpa core changes from Si-Wei [0].
The second part adds support in mlx5_vdpa:
- Refactor the mr code to be able to cleanly add descriptor mappings.
- Add hardware descriptor mr support.
- Properly update iotlb for cvq during ASID switch.
Changes in v2:
- The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change
was split off into two patches to avoid merge conflicts into the tree
of Linus.
The first patch contains only changes for mlx5_ifc.h. This must be
applied into the mlx5-next tree [1] first. Once this patch is applied
on mlx5-next, the change has to be pulled fom mlx5-next into the vhost
tree and only then the remaining patches can be applied.
[0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
[1] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next
Dragos Tatulea (13):
vdpa/mlx5: Expose descriptor group mkey hw capability
vdpa/mlx5: Create helper function for dma mappings
vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
vdpa/mlx5: Take cvq iotlb lock during refresh
vdpa/mlx5: Collapse "dvq" mr add/delete functions
vdpa/mlx5: Rename mr destroy functions
vdpa/mlx5: Allow creation/deletion of any given mr struct
vdpa/mlx5: Move mr mutex out of mr struct
vdpa/mlx5: Improve mr update flow
vdpa/mlx5: Introduce mr for vq descriptor
vdpa/mlx5: Enable hw support for vq descriptor mapping
vdpa/mlx5: Make iotlb helper functions more generic
vdpa/mlx5: Update cvq iotlb mapping on ASID change
Si-Wei Liu (3):
vdpa: introduce dedicated descriptor group for virtqueue
vhost-vdpa: introduce descriptor group backend feature
vhost-vdpa: uAPI to get dedicated descriptor group id
drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
drivers/vdpa/mlx5/core/resources.c | 6 +-
drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
drivers/vhost/vdpa.c | 27 ++++
include/linux/mlx5/mlx5_ifc.h | 8 +-
include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
include/linux/vdpa.h | 11 ++
include/uapi/linux/vhost.h | 8 ++
include/uapi/linux/vhost_types.h | 5 +
10 files changed, 264 insertions(+), 130 deletions(-)
--
2.41.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
@ 2023-09-28 16:45 Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature Dragos Tatulea
` (4 more replies)
0 siblings, 5 replies; 9+ messages in thread
From: Dragos Tatulea @ 2023-09-28 16:45 UTC (permalink / raw)
To: eperezma, gal, Michael S . Tsirkin, Jason Wang, Saeed Mahameed,
Leon Romanovsky, Xuan Zhuo
Cc: virtualization, Dragos Tatulea, netdev, linux-rdma, linux-kernel,
kvm
This patch series adds support for vq descriptor table mappings which
are used to improve vdpa live migration downtime. The improvement comes
from using smaller mappings which take less time to create and destroy
in hw.
The first part adds the vdpa core changes from Si-Wei [0].
The second part adds support in mlx5_vdpa:
- Refactor the mr code to be able to cleanly add descriptor mappings.
- Add hardware descriptor mr support.
- Properly update iotlb for cvq during ASID switch.
Changes in v2:
- The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change
was split off into two patches to avoid merge conflicts into the tree
of Linus.
The first patch contains only changes for mlx5_ifc.h. This must be
applied into the mlx5-next tree [1] first. Once this patch is applied
on mlx5-next, the change has to be pulled fom mlx5-next into the vhost
tree and only then the remaining patches can be applied.
[0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
[1] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next
Dragos Tatulea (13):
vdpa/mlx5: Expose descriptor group mkey hw capability
vdpa/mlx5: Create helper function for dma mappings
vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
vdpa/mlx5: Take cvq iotlb lock during refresh
vdpa/mlx5: Collapse "dvq" mr add/delete functions
vdpa/mlx5: Rename mr destroy functions
vdpa/mlx5: Allow creation/deletion of any given mr struct
vdpa/mlx5: Move mr mutex out of mr struct
vdpa/mlx5: Improve mr update flow
vdpa/mlx5: Introduce mr for vq descriptor
vdpa/mlx5: Enable hw support for vq descriptor mapping
vdpa/mlx5: Make iotlb helper functions more generic
vdpa/mlx5: Update cvq iotlb mapping on ASID change
Si-Wei Liu (3):
vdpa: introduce dedicated descriptor group for virtqueue
vhost-vdpa: introduce descriptor group backend feature
vhost-vdpa: uAPI to get dedicated descriptor group id
drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
drivers/vdpa/mlx5/core/resources.c | 6 +-
drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
drivers/vhost/vdpa.c | 27 ++++
include/linux/mlx5/mlx5_ifc.h | 8 +-
include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
include/linux/vdpa.h | 11 ++
include/uapi/linux/vhost.h | 8 ++
include/uapi/linux/vhost_types.h | 5 +
10 files changed, 264 insertions(+), 130 deletions(-)
--
2.41.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature
2023-09-28 16:45 [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
@ 2023-09-28 16:45 ` Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id Dragos Tatulea
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Dragos Tatulea @ 2023-09-28 16:45 UTC (permalink / raw)
To: eperezma, gal, Michael S . Tsirkin, Jason Wang
Cc: virtualization, Si-Wei Liu, kvm, netdev, linux-kernel
From: Si-Wei Liu <si-wei.liu@oracle.com>
Userspace knows if the device has dedicated descriptor group or not
by checking this feature bit.
It's only exposed if the vdpa driver backend implements the
.get_vq_desc_group() operation callback. Userspace trying to negotiate
this feature when it or the dependent _F_IOTLB_ASID feature hasn't
been exposed will result in an error.
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Eugenio Pérez <eperezma@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/vdpa.c | 17 +++++++++++++++++
include/uapi/linux/vhost_types.h | 5 +++++
2 files changed, 22 insertions(+)
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 78379ffd2336..2f21798a37ee 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -389,6 +389,14 @@ static bool vhost_vdpa_can_resume(const struct vhost_vdpa *v)
return ops->resume;
}
+static bool vhost_vdpa_has_desc_group(const struct vhost_vdpa *v)
+{
+ struct vdpa_device *vdpa = v->vdpa;
+ const struct vdpa_config_ops *ops = vdpa->config;
+
+ return ops->get_vq_desc_group;
+}
+
static long vhost_vdpa_get_features(struct vhost_vdpa *v, u64 __user *featurep)
{
struct vdpa_device *vdpa = v->vdpa;
@@ -690,6 +698,7 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
if (copy_from_user(&features, featurep, sizeof(features)))
return -EFAULT;
if (features & ~(VHOST_VDPA_BACKEND_FEATURES |
+ BIT_ULL(VHOST_BACKEND_F_DESC_ASID) |
BIT_ULL(VHOST_BACKEND_F_SUSPEND) |
BIT_ULL(VHOST_BACKEND_F_RESUME) |
BIT_ULL(VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK)))
@@ -700,6 +709,12 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
if ((features & BIT_ULL(VHOST_BACKEND_F_RESUME)) &&
!vhost_vdpa_can_resume(v))
return -EOPNOTSUPP;
+ if ((features & BIT_ULL(VHOST_BACKEND_F_DESC_ASID)) &&
+ !(features & BIT_ULL(VHOST_BACKEND_F_IOTLB_ASID)))
+ return -EINVAL;
+ if ((features & BIT_ULL(VHOST_BACKEND_F_DESC_ASID)) &&
+ !vhost_vdpa_has_desc_group(v))
+ return -EOPNOTSUPP;
vhost_set_backend_features(&v->vdev, features);
return 0;
}
@@ -753,6 +768,8 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
features |= BIT_ULL(VHOST_BACKEND_F_SUSPEND);
if (vhost_vdpa_can_resume(v))
features |= BIT_ULL(VHOST_BACKEND_F_RESUME);
+ if (vhost_vdpa_has_desc_group(v))
+ features |= BIT_ULL(VHOST_BACKEND_F_DESC_ASID);
features |= vhost_vdpa_get_backend_features(v);
if (copy_to_user(featurep, &features, sizeof(features)))
r = -EFAULT;
diff --git a/include/uapi/linux/vhost_types.h b/include/uapi/linux/vhost_types.h
index 2d827d22cd99..18ad6ae7ab5c 100644
--- a/include/uapi/linux/vhost_types.h
+++ b/include/uapi/linux/vhost_types.h
@@ -185,5 +185,10 @@ struct vhost_vdpa_iova_range {
* DRIVER_OK
*/
#define VHOST_BACKEND_F_ENABLE_AFTER_DRIVER_OK 0x6
+/* Device may expose the virtqueue's descriptor area, driver area and
+ * device area to a different group for ASID binding than where its
+ * buffers may reside. Requires VHOST_BACKEND_F_IOTLB_ASID.
+ */
+#define VHOST_BACKEND_F_DESC_ASID 0x7
#endif
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH vhost 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id
2023-09-28 16:45 [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature Dragos Tatulea
@ 2023-09-28 16:45 ` Dragos Tatulea
2023-10-02 10:16 ` [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Leon Romanovsky
` (2 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Dragos Tatulea @ 2023-09-28 16:45 UTC (permalink / raw)
To: eperezma, gal, Michael S . Tsirkin, Jason Wang
Cc: virtualization, Si-Wei Liu, kvm, netdev, linux-kernel
From: Si-Wei Liu <si-wei.liu@oracle.com>
With _F_DESC_ASID backend feature, the device can now support the
VHOST_VDPA_GET_VRING_DESC_GROUP ioctl, and it may expose the descriptor
table (including avail and used ring) in a different group than the
buffers it contains. This new uAPI will fetch the group ID of the
descriptor table.
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
Acked-by: Eugenio Pérez <eperezma@redhat.com>
Acked-by: Jason Wang <jasowang@redhat.com>
---
drivers/vhost/vdpa.c | 10 ++++++++++
include/uapi/linux/vhost.h | 8 ++++++++
2 files changed, 18 insertions(+)
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 2f21798a37ee..851535f57b95 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -613,6 +613,16 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
else if (copy_to_user(argp, &s, sizeof(s)))
return -EFAULT;
return 0;
+ case VHOST_VDPA_GET_VRING_DESC_GROUP:
+ if (!vhost_vdpa_has_desc_group(v))
+ return -EOPNOTSUPP;
+ s.index = idx;
+ s.num = ops->get_vq_desc_group(vdpa, idx);
+ if (s.num >= vdpa->ngroups)
+ return -EIO;
+ else if (copy_to_user(argp, &s, sizeof(s)))
+ return -EFAULT;
+ return 0;
case VHOST_VDPA_SET_GROUP_ASID:
if (copy_from_user(&s, argp, sizeof(s)))
return -EFAULT;
diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h
index f5c48b61ab62..649560c685f1 100644
--- a/include/uapi/linux/vhost.h
+++ b/include/uapi/linux/vhost.h
@@ -219,4 +219,12 @@
*/
#define VHOST_VDPA_RESUME _IO(VHOST_VIRTIO, 0x7E)
+/* Get the group for the descriptor table including driver & device areas
+ * of a virtqueue: read index, write group in num.
+ * The virtqueue index is stored in the index field of vhost_vring_state.
+ * The group ID of the descriptor table for this specific virtqueue
+ * is returned via num field of vhost_vring_state.
+ */
+#define VHOST_VDPA_GET_VRING_DESC_GROUP _IOWR(VHOST_VIRTIO, 0x7F, \
+ struct vhost_vring_state)
#endif
--
2.41.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
2023-09-28 16:45 [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id Dragos Tatulea
@ 2023-10-02 10:16 ` Leon Romanovsky
2023-10-02 10:16 ` (subset) " Leon Romanovsky
2023-10-05 17:31 ` Michael S. Tsirkin
4 siblings, 0 replies; 9+ messages in thread
From: Leon Romanovsky @ 2023-10-02 10:16 UTC (permalink / raw)
To: Dragos Tatulea
Cc: eperezma, gal, Michael S . Tsirkin, Jason Wang, Saeed Mahameed,
Xuan Zhuo, virtualization, netdev, linux-rdma, linux-kernel, kvm
On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote:
> This patch series adds support for vq descriptor table mappings which
> are used to improve vdpa live migration downtime. The improvement comes
> from using smaller mappings which take less time to create and destroy
> in hw.
>
> The first part adds the vdpa core changes from Si-Wei [0].
>
> The second part adds support in mlx5_vdpa:
> - Refactor the mr code to be able to cleanly add descriptor mappings.
> - Add hardware descriptor mr support.
> - Properly update iotlb for cvq during ASID switch.
>
> Changes in v2:
>
> - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change
> was split off into two patches to avoid merge conflicts into the tree
> of Linus.
>
> The first patch contains only changes for mlx5_ifc.h. This must be
> applied into the mlx5-next tree [1] first. Once this patch is applied
> on mlx5-next, the change has to be pulled fom mlx5-next into the vhost
> tree and only then the remaining patches can be applied.
>
> [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next
>
> Dragos Tatulea (13):
> vdpa/mlx5: Expose descriptor group mkey hw capability
I prepared shared branch with this patch.
https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-vhost
Thanks
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: (subset) [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
2023-09-28 16:45 [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
` (2 preceding siblings ...)
2023-10-02 10:16 ` [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Leon Romanovsky
@ 2023-10-02 10:16 ` Leon Romanovsky
2023-10-05 17:31 ` Michael S. Tsirkin
4 siblings, 0 replies; 9+ messages in thread
From: Leon Romanovsky @ 2023-10-02 10:16 UTC (permalink / raw)
To: eperezma, gal, Michael S . Tsirkin, Jason Wang, Saeed Mahameed,
Xuan Zhuo, Dragos Tatulea
Cc: virtualization, netdev, linux-rdma, linux-kernel, kvm
On Thu, 28 Sep 2023 19:45:11 +0300, Dragos Tatulea wrote:
> This patch series adds support for vq descriptor table mappings which
> are used to improve vdpa live migration downtime. The improvement comes
> from using smaller mappings which take less time to create and destroy
> in hw.
>
> The first part adds the vdpa core changes from Si-Wei [0].
>
> [...]
Applied, thanks!
[01/16] vdpa/mlx5: Expose descriptor group mkey hw capability
https://git.kernel.org/rdma/rdma/c/d424348b060d87
Best regards,
--
Leon Romanovsky <leon@kernel.org>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
2023-09-28 16:45 [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
` (3 preceding siblings ...)
2023-10-02 10:16 ` (subset) " Leon Romanovsky
@ 2023-10-05 17:31 ` Michael S. Tsirkin
2023-10-05 17:44 ` Dragos Tatulea
4 siblings, 1 reply; 9+ messages in thread
From: Michael S. Tsirkin @ 2023-10-05 17:31 UTC (permalink / raw)
To: Dragos Tatulea
Cc: eperezma, gal, Jason Wang, Saeed Mahameed, Leon Romanovsky,
Xuan Zhuo, virtualization, netdev, linux-rdma, linux-kernel, kvm
On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote:
> This patch series adds support for vq descriptor table mappings which
> are used to improve vdpa live migration downtime. The improvement comes
> from using smaller mappings which take less time to create and destroy
> in hw.
>
> The first part adds the vdpa core changes from Si-Wei [0].
>
> The second part adds support in mlx5_vdpa:
> - Refactor the mr code to be able to cleanly add descriptor mappings.
> - Add hardware descriptor mr support.
> - Properly update iotlb for cvq during ASID switch.
>
> Changes in v2:
>
> - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change
> was split off into two patches to avoid merge conflicts into the tree
> of Linus.
>
> The first patch contains only changes for mlx5_ifc.h. This must be
> applied into the mlx5-next tree [1] first. Once this patch is applied
> on mlx5-next, the change has to be pulled fom mlx5-next into the vhost
> tree and only then the remaining patches can be applied.
I get it you plan v3?
> [0] https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next
>
> Dragos Tatulea (13):
> vdpa/mlx5: Expose descriptor group mkey hw capability
> vdpa/mlx5: Create helper function for dma mappings
> vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
> vdpa/mlx5: Take cvq iotlb lock during refresh
> vdpa/mlx5: Collapse "dvq" mr add/delete functions
> vdpa/mlx5: Rename mr destroy functions
> vdpa/mlx5: Allow creation/deletion of any given mr struct
> vdpa/mlx5: Move mr mutex out of mr struct
> vdpa/mlx5: Improve mr update flow
> vdpa/mlx5: Introduce mr for vq descriptor
> vdpa/mlx5: Enable hw support for vq descriptor mapping
> vdpa/mlx5: Make iotlb helper functions more generic
> vdpa/mlx5: Update cvq iotlb mapping on ASID change
>
> Si-Wei Liu (3):
> vdpa: introduce dedicated descriptor group for virtqueue
> vhost-vdpa: introduce descriptor group backend feature
> vhost-vdpa: uAPI to get dedicated descriptor group id
>
> drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
> drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
> drivers/vdpa/mlx5/core/resources.c | 6 +-
> drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
> drivers/vhost/vdpa.c | 27 ++++
> include/linux/mlx5/mlx5_ifc.h | 8 +-
> include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
> include/linux/vdpa.h | 11 ++
> include/uapi/linux/vhost.h | 8 ++
> include/uapi/linux/vhost_types.h | 5 +
> 10 files changed, 264 insertions(+), 130 deletions(-)
>
> --
> 2.41.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
2023-10-05 17:31 ` Michael S. Tsirkin
@ 2023-10-05 17:44 ` Dragos Tatulea
2023-10-05 19:18 ` Michael S. Tsirkin
0 siblings, 1 reply; 9+ messages in thread
From: Dragos Tatulea @ 2023-10-05 17:44 UTC (permalink / raw)
To: mst@redhat.com
Cc: linux-rdma@vger.kernel.org, xuanzhuo@linux.alibaba.com,
virtualization@lists.linux-foundation.org, Gal Pressman,
eperezma@redhat.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org, jasowang@redhat.com, leon@kernel.org,
Saeed Mahameed, kvm@vger.kernel.org
On Thu, 2023-10-05 at 13:31 -0400, Michael S. Tsirkin wrote:
> On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote:
> > This patch series adds support for vq descriptor table mappings which
> > are used to improve vdpa live migration downtime. The improvement comes
> > from using smaller mappings which take less time to create and destroy
> > in hw.
> >
> > The first part adds the vdpa core changes from Si-Wei [0].
> >
> > The second part adds support in mlx5_vdpa:
> > - Refactor the mr code to be able to cleanly add descriptor mappings.
> > - Add hardware descriptor mr support.
> > - Properly update iotlb for cvq during ASID switch.
> >
> > Changes in v2:
> >
> > - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change
> > was split off into two patches to avoid merge conflicts into the tree
> > of Linus.
> >
> > The first patch contains only changes for mlx5_ifc.h. This must be
> > applied into the mlx5-next tree [1] first. Once this patch is applied
> > on mlx5-next, the change has to be pulled fom mlx5-next into the vhost
> > tree and only then the remaining patches can be applied.
>
>
> I get it you plan v3?
There are some very small improvements (commit message in 13/16 and fix in
16/16) that could make a v3. The latter can be addressed as a separate patch
when moving dup_iotlb to vhost/iotlb. What do you think?
>
> > [0]
> > https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
> > [1]
> > https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next
> >
> > Dragos Tatulea (13):
> > vdpa/mlx5: Expose descriptor group mkey hw capability
> > vdpa/mlx5: Create helper function for dma mappings
> > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
> > vdpa/mlx5: Take cvq iotlb lock during refresh
> > vdpa/mlx5: Collapse "dvq" mr add/delete functions
> > vdpa/mlx5: Rename mr destroy functions
> > vdpa/mlx5: Allow creation/deletion of any given mr struct
> > vdpa/mlx5: Move mr mutex out of mr struct
> > vdpa/mlx5: Improve mr update flow
> > vdpa/mlx5: Introduce mr for vq descriptor
> > vdpa/mlx5: Enable hw support for vq descriptor mapping
> > vdpa/mlx5: Make iotlb helper functions more generic
> > vdpa/mlx5: Update cvq iotlb mapping on ASID change
> >
> > Si-Wei Liu (3):
> > vdpa: introduce dedicated descriptor group for virtqueue
> > vhost-vdpa: introduce descriptor group backend feature
> > vhost-vdpa: uAPI to get dedicated descriptor group id
> >
> > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
> > drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
> > drivers/vdpa/mlx5/core/resources.c | 6 +-
> > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
> > drivers/vhost/vdpa.c | 27 ++++
> > include/linux/mlx5/mlx5_ifc.h | 8 +-
> > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
> > include/linux/vdpa.h | 11 ++
> > include/uapi/linux/vhost.h | 8 ++
> > include/uapi/linux/vhost_types.h | 5 +
> > 10 files changed, 264 insertions(+), 130 deletions(-)
> >
> > --
> > 2.41.0
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings
2023-10-05 17:44 ` Dragos Tatulea
@ 2023-10-05 19:18 ` Michael S. Tsirkin
0 siblings, 0 replies; 9+ messages in thread
From: Michael S. Tsirkin @ 2023-10-05 19:18 UTC (permalink / raw)
To: Dragos Tatulea
Cc: linux-rdma@vger.kernel.org, xuanzhuo@linux.alibaba.com,
virtualization@lists.linux-foundation.org, Gal Pressman,
eperezma@redhat.com, linux-kernel@vger.kernel.org,
netdev@vger.kernel.org, jasowang@redhat.com, leon@kernel.org,
Saeed Mahameed, kvm@vger.kernel.org
On Thu, Oct 05, 2023 at 05:44:01PM +0000, Dragos Tatulea wrote:
> On Thu, 2023-10-05 at 13:31 -0400, Michael S. Tsirkin wrote:
> > On Thu, Sep 28, 2023 at 07:45:11PM +0300, Dragos Tatulea wrote:
> > > This patch series adds support for vq descriptor table mappings which
> > > are used to improve vdpa live migration downtime. The improvement comes
> > > from using smaller mappings which take less time to create and destroy
> > > in hw.
> > >
> > > The first part adds the vdpa core changes from Si-Wei [0].
> > >
> > > The second part adds support in mlx5_vdpa:
> > > - Refactor the mr code to be able to cleanly add descriptor mappings.
> > > - Add hardware descriptor mr support.
> > > - Properly update iotlb for cvq during ASID switch.
> > >
> > > Changes in v2:
> > >
> > > - The "vdpa/mlx5: Enable hw support for vq descriptor mapping" change
> > > was split off into two patches to avoid merge conflicts into the tree
> > > of Linus.
> > >
> > > The first patch contains only changes for mlx5_ifc.h. This must be
> > > applied into the mlx5-next tree [1] first. Once this patch is applied
> > > on mlx5-next, the change has to be pulled fom mlx5-next into the vhost
> > > tree and only then the remaining patches can be applied.
> >
> >
> > I get it you plan v3?
> There are some very small improvements (commit message in 13/16 and fix in
> 16/16) that could make a v3. The latter can be addressed as a separate patch
> when moving dup_iotlb to vhost/iotlb. What do you think?
if there's a fix by all means post v3.
> >
> > > [0]
> > > https://lore.kernel.org/virtualization/1694248959-13369-1-git-send-email-si-wei.liu@oracle.com
> > > [1]
> > > https://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git/log/?h=mlx5-next
> > >
> > > Dragos Tatulea (13):
> > > vdpa/mlx5: Expose descriptor group mkey hw capability
> > > vdpa/mlx5: Create helper function for dma mappings
> > > vdpa/mlx5: Decouple cvq iotlb handling from hw mapping code
> > > vdpa/mlx5: Take cvq iotlb lock during refresh
> > > vdpa/mlx5: Collapse "dvq" mr add/delete functions
> > > vdpa/mlx5: Rename mr destroy functions
> > > vdpa/mlx5: Allow creation/deletion of any given mr struct
> > > vdpa/mlx5: Move mr mutex out of mr struct
> > > vdpa/mlx5: Improve mr update flow
> > > vdpa/mlx5: Introduce mr for vq descriptor
> > > vdpa/mlx5: Enable hw support for vq descriptor mapping
> > > vdpa/mlx5: Make iotlb helper functions more generic
> > > vdpa/mlx5: Update cvq iotlb mapping on ASID change
> > >
> > > Si-Wei Liu (3):
> > > vdpa: introduce dedicated descriptor group for virtqueue
> > > vhost-vdpa: introduce descriptor group backend feature
> > > vhost-vdpa: uAPI to get dedicated descriptor group id
> > >
> > > drivers/vdpa/mlx5/core/mlx5_vdpa.h | 31 +++--
> > > drivers/vdpa/mlx5/core/mr.c | 191 ++++++++++++++++-------------
> > > drivers/vdpa/mlx5/core/resources.c | 6 +-
> > > drivers/vdpa/mlx5/net/mlx5_vnet.c | 100 ++++++++++-----
> > > drivers/vhost/vdpa.c | 27 ++++
> > > include/linux/mlx5/mlx5_ifc.h | 8 +-
> > > include/linux/mlx5/mlx5_ifc_vdpa.h | 7 +-
> > > include/linux/vdpa.h | 11 ++
> > > include/uapi/linux/vhost.h | 8 ++
> > > include/uapi/linux/vhost_types.h | 5 +
> > > 10 files changed, 264 insertions(+), 130 deletions(-)
> > >
> > > --
> > > 2.41.0
> >
>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-10-05 19:19 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-28 16:45 [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 03/16] vhost-vdpa: introduce descriptor group backend feature Dragos Tatulea
2023-09-28 16:45 ` [PATCH vhost 04/16] vhost-vdpa: uAPI to get dedicated descriptor group id Dragos Tatulea
2023-10-02 10:16 ` [PATCH vhost v2 00/16] vdpa: Add support for vq descriptor mappings Leon Romanovsky
2023-10-02 10:16 ` (subset) " Leon Romanovsky
2023-10-05 17:31 ` Michael S. Tsirkin
2023-10-05 17:44 ` Dragos Tatulea
2023-10-05 19:18 ` Michael S. Tsirkin
-- strict thread matches above, loose matches on Subject: below --
2023-09-28 16:33 Dragos Tatulea
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).