* [PATCH v3 1/2] iommu/virtio: Make use of ops->iotlb_sync_map
2023-11-20 14:51 [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH Niklas Schnelle
@ 2023-11-20 14:51 ` Niklas Schnelle
2023-11-20 14:51 ` [PATCH v3 2/2] iommu/virtio: Add ops->flush_iotlb_all and enable deferred flush Niklas Schnelle
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Niklas Schnelle @ 2023-11-20 14:51 UTC (permalink / raw)
To: Jean-Philippe Brucker, Joerg Roedel, Will Deacon, Jason Gunthorpe,
Robin Murphy
Cc: Niklas Schnelle, virtualization, iommu, linux-kernel
Pull out the sync operation from viommu_map_pages() by implementing
ops->iotlb_sync_map. This allows the common IOMMU code to map multiple
elements of an sg with a single sync (see iommu_map_sg()).
Link: https://lore.kernel.org/lkml/20230726111433.1105665-1-schnelle@linux.ibm.com/
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
---
drivers/iommu/virtio-iommu.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c
index 379ebe03efb6..3f18f69d70d8 100644
--- a/drivers/iommu/virtio-iommu.c
+++ b/drivers/iommu/virtio-iommu.c
@@ -843,7 +843,7 @@ static int viommu_map_pages(struct iommu_domain *domain, unsigned long iova,
.flags = cpu_to_le32(flags),
};
- ret = viommu_send_req_sync(vdomain->viommu, &map, sizeof(map));
+ ret = viommu_add_req(vdomain->viommu, &map, sizeof(map));
if (ret) {
viommu_del_mappings(vdomain, iova, end);
return ret;
@@ -912,6 +912,20 @@ static void viommu_iotlb_sync(struct iommu_domain *domain,
viommu_sync_req(vdomain->viommu);
}
+static int viommu_iotlb_sync_map(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ /*
+ * May be called before the viommu is initialized including
+ * while creating direct mapping
+ */
+ if (!vdomain->nr_endpoints)
+ return 0;
+ return viommu_sync_req(vdomain->viommu);
+}
+
static void viommu_get_resv_regions(struct device *dev, struct list_head *head)
{
struct iommu_resv_region *entry, *new_entry, *msi = NULL;
@@ -1058,6 +1072,7 @@ static struct iommu_ops viommu_ops = {
.unmap_pages = viommu_unmap_pages,
.iova_to_phys = viommu_iova_to_phys,
.iotlb_sync = viommu_iotlb_sync,
+ .iotlb_sync_map = viommu_iotlb_sync_map,
.free = viommu_domain_free,
}
};
--
2.39.2
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH v3 2/2] iommu/virtio: Add ops->flush_iotlb_all and enable deferred flush
2023-11-20 14:51 [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH Niklas Schnelle
2023-11-20 14:51 ` [PATCH v3 1/2] iommu/virtio: Make use of ops->iotlb_sync_map Niklas Schnelle
@ 2023-11-20 14:51 ` Niklas Schnelle
2023-11-20 18:38 ` [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH Jean-Philippe Brucker
2023-11-27 10:02 ` Joerg Roedel
3 siblings, 0 replies; 5+ messages in thread
From: Niklas Schnelle @ 2023-11-20 14:51 UTC (permalink / raw)
To: Jean-Philippe Brucker, Joerg Roedel, Will Deacon, Jason Gunthorpe,
Robin Murphy
Cc: Niklas Schnelle, virtualization, iommu, linux-kernel
Add ops->flush_iotlb_all operation to enable virtio-iommu for the
dma-iommu deferred flush scheme. This results in a significant increase
in performance in exchange for a window in which devices can still
access previously IOMMU mapped memory when running with
CONFIG_IOMMU_DEFAULT_DMA_LAZY. The previous strict behavior can be
achieved with iommu.strict=1 on the kernel command line or
CONFIG_IOMMU_DEFAULT_DMA_STRICT.
Link: https://lore.kernel.org/lkml/20230802123612.GA6142@myrica/
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
---
drivers/iommu/virtio-iommu.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/drivers/iommu/virtio-iommu.c b/drivers/iommu/virtio-iommu.c
index 3f18f69d70d8..ef6a87445105 100644
--- a/drivers/iommu/virtio-iommu.c
+++ b/drivers/iommu/virtio-iommu.c
@@ -926,6 +926,19 @@ static int viommu_iotlb_sync_map(struct iommu_domain *domain,
return viommu_sync_req(vdomain->viommu);
}
+static void viommu_flush_iotlb_all(struct iommu_domain *domain)
+{
+ struct viommu_domain *vdomain = to_viommu_domain(domain);
+
+ /*
+ * May be called before the viommu is initialized including
+ * while creating direct mapping
+ */
+ if (!vdomain->nr_endpoints)
+ return;
+ viommu_sync_req(vdomain->viommu);
+}
+
static void viommu_get_resv_regions(struct device *dev, struct list_head *head)
{
struct iommu_resv_region *entry, *new_entry, *msi = NULL;
@@ -1051,6 +1064,8 @@ static bool viommu_capable(struct device *dev, enum iommu_cap cap)
switch (cap) {
case IOMMU_CAP_CACHE_COHERENCY:
return true;
+ case IOMMU_CAP_DEFERRED_FLUSH:
+ return true;
default:
return false;
}
@@ -1071,6 +1086,7 @@ static struct iommu_ops viommu_ops = {
.map_pages = viommu_map_pages,
.unmap_pages = viommu_unmap_pages,
.iova_to_phys = viommu_iova_to_phys,
+ .flush_iotlb_all = viommu_flush_iotlb_all,
.iotlb_sync = viommu_iotlb_sync,
.iotlb_sync_map = viommu_iotlb_sync_map,
.free = viommu_domain_free,
--
2.39.2
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH
2023-11-20 14:51 [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH Niklas Schnelle
2023-11-20 14:51 ` [PATCH v3 1/2] iommu/virtio: Make use of ops->iotlb_sync_map Niklas Schnelle
2023-11-20 14:51 ` [PATCH v3 2/2] iommu/virtio: Add ops->flush_iotlb_all and enable deferred flush Niklas Schnelle
@ 2023-11-20 18:38 ` Jean-Philippe Brucker
2023-11-27 10:02 ` Joerg Roedel
3 siblings, 0 replies; 5+ messages in thread
From: Jean-Philippe Brucker @ 2023-11-20 18:38 UTC (permalink / raw)
To: Niklas Schnelle
Cc: Joerg Roedel, Will Deacon, Jason Gunthorpe, Robin Murphy,
virtualization, iommu, linux-kernel
Hi Niklas,
On Mon, Nov 20, 2023 at 03:51:55PM +0100, Niklas Schnelle wrote:
> Hi All,
>
> Previously I used virtio-iommu as a non-s390x test vehicle[0] for the
> single queue flushing scheme introduced by my s390x DMA API conversion
> series[1]. For this I modified virtio-iommu to a) use .iotlb_sync_map
> and b) enable IOMMU_CAP_DEFERRED_FLUSH. It turned out that deferred
> flush and even just the introduction of ops->iotlb_sync_map yield
> performance uplift[2] even with per-CPU queues. So here is a small
> series of these two changes.
>
> The code is also available on the b4/viommu-deferred-flush branch of my
> kernel.org git repository[3].
>
> Note on testing: I tested this series on my AMD Ryzen 3900X workstation
> using QEMU 8.1.2 a pass-through NVMe and Intel 82599 NIC VFs. For the
> NVMe I saw an increase of about 10% in IOPS and 30% in read bandwidth
> compared with v6.7-rc2. One odd thing though is that QEMU seemed to make
> the entire guest resident/pinned once I passed-through a PCI device.
> I seem to remember this wasn't the case with my last version but not
> sure which QEMU version I used back then.
That's probably expected, now that boot-bypass is enabled by default: on
VM boot, endpoints are able to do DMA to the entire guest-physical address
space, until a virtio-iommu driver disables global bypass in the config
space (at which point the pinned memory is hopefully reclaimed by the
host). QEMU enables it by default to mimic other IOMMU implementations,
and to allow running firmware or OS that don't support virtio-iommu. It
can be disabled with boot-bypass=off
> @Jean-Philippe: I didn't include your R-b's as I changed back to the
> nr_endpoints check and this is like 30% of the patches.
Thank you for the patches. For the series:
Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH
2023-11-20 14:51 [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH Niklas Schnelle
` (2 preceding siblings ...)
2023-11-20 18:38 ` [PATCH v3 0/2] iommu/virtio: Enable IOMMU_CAP_DERRED_FLUSH Jean-Philippe Brucker
@ 2023-11-27 10:02 ` Joerg Roedel
3 siblings, 0 replies; 5+ messages in thread
From: Joerg Roedel @ 2023-11-27 10:02 UTC (permalink / raw)
To: Niklas Schnelle
Cc: Jean-Philippe Brucker, Will Deacon, Jason Gunthorpe, Robin Murphy,
virtualization, iommu, linux-kernel
On Mon, Nov 20, 2023 at 03:51:55PM +0100, Niklas Schnelle wrote:
> Niklas Schnelle (2):
> iommu/virtio: Make use of ops->iotlb_sync_map
> iommu/virtio: Add ops->flush_iotlb_all and enable deferred flush
>
> drivers/iommu/virtio-iommu.c | 33 ++++++++++++++++++++++++++++++++-
> 1 file changed, 32 insertions(+), 1 deletion(-)
Applied, thanks.
^ permalink raw reply [flat|nested] 5+ messages in thread