[PATCH] commit 6b080c4e815ceba3c08ffa980c858595c07e786a upstream

[Zhichuang Sun <zhichuang@google.com>]

iommu/amd: fix amd iotlb flush range in unmap

This was fixed in mainline in 6b080c4e815ceba3c08ffa980c858595c07e786a,
but do not backport the full refactor.

Targeting branch lts linux-5.15.y.

AMD IOMMU driver supports power of 2 KB page size, it can be 4K, 8K,
16K, etc. So when VFIO driver ask AMD IOMMU driver to unmap a
IOVA with a page_size 4K, it actually can unmap a page_size of
8K, depending on the page used during mapping. However, the iotlb
gather function use the page_size as the range of unmap range,
instead of the real unmapped page size r.

This miscalculation of iotlb flush range will make the unflushed
IOTLB entry stale. It triggered hard-to-debug silent data corruption
issue as DMA engine who used the stale IOTLB entry will DMA into
unmapped memory region.

The upstream commit aims at changing API from map/unmap_page() to
map/unmap_pages() and changed the gather range calculation along
with it. It accidentally fixed this bug in the mainline since 6.1.
For this backport, we don't backport the API change, only port the
gather range calculation to fix the bug.

Cc: Nadav Amit <namit@vmware.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Will Deacon <will@kernel.org>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org
Fixes: fc65d0acaf23179b94de399c204328fa259acb90
Signed-off-by: Zhichuang Sun <zhichuang@google.com>
---
 drivers/iommu/amd/iommu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 714c78bf69db..d3a11be8d1dd 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -2121,7 +2121,8 @@ static size_t amd_iommu_unmap(struct iommu_domain *dom, unsigned long iova,
 
 	r = (ops->unmap) ? ops->unmap(ops, iova, page_size, gather) : 0;
 
-	amd_iommu_iotlb_gather_add_page(dom, gather, iova, page_size);
+	if (r)
+		amd_iommu_iotlb_gather_add_page(dom, gather, iova, r);
 
 	return r;
 }
-- 
2.51.0.618.g983fd99d29-goog