From: Si-Wei Liu <si-wei.liu@oracle.com>
To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com,
sgarzare@redhat.com, dtatulea@nvidia.com
Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org
Subject: [PATCH v4 4/7] vdpa: introduce .compat_reset operation callback
Date: Sat, 21 Oct 2023 02:25:16 -0700 [thread overview]
Message-ID: <1697880319-4937-5-git-send-email-si-wei.liu@oracle.com> (raw)
In-Reply-To: <1697880319-4937-1-git-send-email-si-wei.liu@oracle.com>
Some device specific IOMMU parent drivers have long standing bogus
behaviour that mistakenly clean up the maps during .reset. By
definition, this is violation to the on-chip IOMMU ops (i.e. .set_map,
or .dma_map & .dma_unmap) in those offending drivers, as the removal of
internal maps is completely agnostic to the upper layer, causing
inconsistent view between the userspace and the kernel. Some userspace
app like QEMU gets around of this brokenness by proactively removing and
adding back all the maps around vdpa device reset, but such workaround
actually penaltize other well-behaved driver setup, where vdpa reset
always comes with the associated mapping cost, especially for kernel
vDPA devices (use_va=false) that have high cost on pinning. It's
imperative to rectify this behaviour and remove the problematic code
from all those non-compliant parent drivers.
However, we cannot unconditionally remove the bogus map-cleaning code
from the buggy .reset implementation, as there might exist userspace
apps that already rely on the behaviour on some setup. Introduce a
.compat_reset driver op to keep compatibility with older userspace. New
and well behaved parent driver should not bother to implement such op,
but only those drivers that are doing or used to do non-compliant
map-cleaning reset will have to.
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
---
include/linux/vdpa.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
index 26ae6ae1eac3..6b8cbf75712d 100644
--- a/include/linux/vdpa.h
+++ b/include/linux/vdpa.h
@@ -252,6 +252,17 @@ struct vdpa_map_file {
* @reset: Reset device
* @vdev: vdpa device
* Returns integer: success (0) or error (< 0)
+ * @compat_reset: Reset device with compatibility quirks to
+ * accommodate older userspace. Only needed by
+ * parent driver which used to have bogus reset
+ * behaviour, and has to maintain such behaviour
+ * for compatibility with older userspace.
+ * Historically compliant driver only has to
+ * implement .reset, Historically non-compliant
+ * driver should implement both.
+ * @vdev: vdpa device
+ * @flags: compatibility quirks for reset
+ * Returns integer: success (0) or error (< 0)
* @suspend: Suspend the device (optional)
* @vdev: vdpa device
* Returns integer: success (0) or error (< 0)
@@ -393,6 +404,8 @@ struct vdpa_config_ops {
u8 (*get_status)(struct vdpa_device *vdev);
void (*set_status)(struct vdpa_device *vdev, u8 status);
int (*reset)(struct vdpa_device *vdev);
+ int (*compat_reset)(struct vdpa_device *vdev, u32 flags);
+#define VDPA_RESET_F_CLEAN_MAP 1
int (*suspend)(struct vdpa_device *vdev);
int (*resume)(struct vdpa_device *vdev);
size_t (*get_config_size)(struct vdpa_device *vdev);
--
2.39.3
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
WARNING: multiple messages have this Message-ID (diff)
From: Si-Wei Liu <si-wei.liu@oracle.com>
To: jasowang@redhat.com, mst@redhat.com, eperezma@redhat.com,
sgarzare@redhat.com, dtatulea@nvidia.com
Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Subject: [PATCH v4 4/7] vdpa: introduce .compat_reset operation callback
Date: Sat, 21 Oct 2023 02:25:16 -0700 [thread overview]
Message-ID: <1697880319-4937-5-git-send-email-si-wei.liu@oracle.com> (raw)
In-Reply-To: <1697880319-4937-1-git-send-email-si-wei.liu@oracle.com>
Some device specific IOMMU parent drivers have long standing bogus
behaviour that mistakenly clean up the maps during .reset. By
definition, this is violation to the on-chip IOMMU ops (i.e. .set_map,
or .dma_map & .dma_unmap) in those offending drivers, as the removal of
internal maps is completely agnostic to the upper layer, causing
inconsistent view between the userspace and the kernel. Some userspace
app like QEMU gets around of this brokenness by proactively removing and
adding back all the maps around vdpa device reset, but such workaround
actually penaltize other well-behaved driver setup, where vdpa reset
always comes with the associated mapping cost, especially for kernel
vDPA devices (use_va=false) that have high cost on pinning. It's
imperative to rectify this behaviour and remove the problematic code
from all those non-compliant parent drivers.
However, we cannot unconditionally remove the bogus map-cleaning code
from the buggy .reset implementation, as there might exist userspace
apps that already rely on the behaviour on some setup. Introduce a
.compat_reset driver op to keep compatibility with older userspace. New
and well behaved parent driver should not bother to implement such op,
but only those drivers that are doing or used to do non-compliant
map-cleaning reset will have to.
Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
---
include/linux/vdpa.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
index 26ae6ae1eac3..6b8cbf75712d 100644
--- a/include/linux/vdpa.h
+++ b/include/linux/vdpa.h
@@ -252,6 +252,17 @@ struct vdpa_map_file {
* @reset: Reset device
* @vdev: vdpa device
* Returns integer: success (0) or error (< 0)
+ * @compat_reset: Reset device with compatibility quirks to
+ * accommodate older userspace. Only needed by
+ * parent driver which used to have bogus reset
+ * behaviour, and has to maintain such behaviour
+ * for compatibility with older userspace.
+ * Historically compliant driver only has to
+ * implement .reset, Historically non-compliant
+ * driver should implement both.
+ * @vdev: vdpa device
+ * @flags: compatibility quirks for reset
+ * Returns integer: success (0) or error (< 0)
* @suspend: Suspend the device (optional)
* @vdev: vdpa device
* Returns integer: success (0) or error (< 0)
@@ -393,6 +404,8 @@ struct vdpa_config_ops {
u8 (*get_status)(struct vdpa_device *vdev);
void (*set_status)(struct vdpa_device *vdev, u8 status);
int (*reset)(struct vdpa_device *vdev);
+ int (*compat_reset)(struct vdpa_device *vdev, u32 flags);
+#define VDPA_RESET_F_CLEAN_MAP 1
int (*suspend)(struct vdpa_device *vdev);
int (*resume)(struct vdpa_device *vdev);
size_t (*get_config_size)(struct vdpa_device *vdev);
--
2.39.3
next prev parent reply other threads:[~2023-10-21 9:28 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-21 9:25 [PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
2023-10-21 9:25 ` [PATCH v4 1/7] vdpa: introduce .reset_map operation callback Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
2023-10-21 9:25 ` [PATCH v4 2/7] vhost-vdpa: reset vendor specific mapping to initial state in .release Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
2023-10-21 9:25 ` [PATCH v4 3/7] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu [this message]
2023-10-21 9:25 ` [PATCH v4 4/7] vdpa: introduce .compat_reset operation callback Si-Wei Liu
2023-10-21 9:25 ` [PATCH v4 5/7] vhost-vdpa: clean iotlb map during reset for older userspace Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
[not found] ` <8a3ec6465bc150fba804f34d456115c4f8a29c70.camel@nvidia.com>
2023-10-23 23:03 ` Si-Wei Liu
2023-10-23 23:03 ` Si-Wei Liu
2023-10-24 5:45 ` Jason Wang
2023-10-24 5:45 ` Jason Wang
2023-10-24 16:21 ` Si-Wei Liu
2023-10-24 16:21 ` Si-Wei Liu
2023-10-24 16:25 ` Si-Wei Liu
2023-10-24 16:25 ` Si-Wei Liu
2023-10-25 1:15 ` Jason Wang
2023-10-25 1:15 ` Jason Wang
2023-10-21 9:25 ` [PATCH v4 6/7] vdpa/mlx5: implement .reset_map driver op Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
2023-10-21 9:25 ` [PATCH v4 7/7] vdpa_sim: implement .reset_map support Si-Wei Liu
2023-10-21 9:25 ` Si-Wei Liu
2023-10-23 3:51 ` [PATCH v4 0/7] vdpa: decouple reset of iotlb mapping from device reset Jason Wang
2023-10-23 3:51 ` Jason Wang
2023-10-23 22:00 ` Si-Wei Liu
2023-10-23 22:00 ` Si-Wei Liu
2023-10-24 6:51 ` Lei Yang
2023-10-24 17:27 ` Si-Wei Liu
2023-10-24 17:27 ` Si-Wei Liu
2023-10-25 9:41 ` Lei Yang
2023-10-25 23:31 ` Si-Wei Liu
2023-10-25 23:31 ` Si-Wei Liu
2023-10-26 6:16 ` Lei Yang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1697880319-4937-5-git-send-email-si-wei.liu@oracle.com \
--to=si-wei.liu@oracle.com \
--cc=dtatulea@nvidia.com \
--cc=eperezma@redhat.com \
--cc=jasowang@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mst@redhat.com \
--cc=sgarzare@redhat.com \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.