[PATCH v3] iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode

[PATCH v3] iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode

[Jacob Pan]

Intel IOMMU driver implements IOTLB flush queue with domain selective
or PASID selective invalidations. In this case there's no need to track
IOVA page range and sync IOTLBs, which may cause significant performance
hit.

This patch adds a check to avoid IOVA gather page and IOTLB sync for
the lazy path.

The performance difference on Sapphire Rapids 100Gb NIC is improved by
the following (as measured by iperf send):

w/o this fix~48 Gbits/s. with this fix ~54 Gbits/s

Cc: <stable@vger.kernel.org>
Fixes: 2a2b8eaa5b25 ("iommu: Handle freelists when using deferred flushing in iommu drivers")
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Tested-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
v3: reword comments, add reviewed by tag. (Kevin)
v2: use helper function iommu_iotlb_gather_queued() instead of open
coding
---
 drivers/iommu/intel/iommu.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 59df7e42fd53..2ee270e4d484 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -4346,7 +4346,12 @@ static size_t intel_iommu_unmap(struct iommu_domain *domain,
 	if (dmar_domain->max_addr == iova + size)
 		dmar_domain->max_addr = iova;
 
-	iommu_iotlb_gather_add_page(domain, gather, iova, size);
+	/*
+	 * We do not use page-selective IOTLB invalidation in flush queue,
+	 * so there is no need to track page and sync iotlb.
+	 */
+	if (!iommu_iotlb_gather_queued(gather))
+		iommu_iotlb_gather_add_page(domain, gather, iova, size);
 
 	return size;
 }
-- 
2.25.1

Re: [PATCH v3] iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode

[Ferdinand Schober]

Hi,

I've stumbled upon this patch trying to figure out how lazy invalidation is implemented in intel IOMMUs.
The patch suggests that lazy invalidation is active whenever iotlb_gather.queued is set.

However, the only place in which gather.queued is written seems to be in dma-iommu.c:

-- drivers/iommu/dma-iommu.c
 820:	iotlb_gather.queued = READ_ONCE(cookie->fq_domain);
2038:	iotlb_gather.queued = free_iova && READ_ONCE(cookie->fq_domain);


Both of these depend on fq_domain but fq_domain is always NULL for intel iommus,
since iommu/intel/iommu.c reports IOMMU_CAP_DEFERRED_FLUSH:

-- drivers/iommu/dma-iommu.c:708
	if (domain->type == IOMMU_DOMAIN_DMA_FQ &&
	    (!device_iommu_capable(dev, IOMMU_CAP_DEFERRED_FLUSH) || iommu_dma_init_fq(domain)))
		domain->type = IOMMU_DOMAIN_DMA;


(Above line numbers are from Kernel 6.17).

So I'm not sure, this commit does what it should?
Please let me know what I'm missing here.

Best Regards,
Ferdinand Schober

Re: [PATCH v3] iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode

[Ferdinand Schober]

Hi,

I've stumbled upon this patch trying to figure out how lazy invalidation is implemented in intel IOMMUs.
The patch suggests that lazy invalidation is active whenever iotlb_gather.queued is set.

However, the only place in which gather.queued is written seems to be in dma-iommu.c:

-- drivers/iommu/dma-iommu.c
 820:	iotlb_gather.queued = READ_ONCE(cookie->fq_domain);
2038:	iotlb_gather.queued = free_iova && READ_ONCE(cookie->fq_domain);


Both of these depend on fq_domain but fq_domain is always NULL for intel iommus,
since iommu/intel/iommu.c reports IOMMU_CAP_DEFERRED_FLUSH:

-- drivers/iommu/dma-iommu.c:708
	if (domain->type == IOMMU_DOMAIN_DMA_FQ &&
	    (!device_iommu_capable(dev, IOMMU_CAP_DEFERRED_FLUSH) || iommu_dma_init_fq(domain)))
		domain->type = IOMMU_DOMAIN_DMA;


(Above line numbers are from Kernel 6.17).

So I'm not sure, this commit does what it should?
Please let me know what I'm missing here.

Best Regards,
Ferdinand Schober

Re: [PATCH v3] iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode

[Robin Murphy]

On 2026-02-20 1:52 am, Ferdinand Schober wrote:
> 
> Hi,
> 
> I've stumbled upon this patch trying to figure out how lazy invalidation is implemented in intel IOMMUs.
> The patch suggests that lazy invalidation is active whenever iotlb_gather.queued is set.
> 
> However, the only place in which gather.queued is written seems to be in dma-iommu.c:
> 
> -- drivers/iommu/dma-iommu.c
>   820:	iotlb_gather.queued = READ_ONCE(cookie->fq_domain);
> 2038:	iotlb_gather.queued = free_iova && READ_ONCE(cookie->fq_domain);
> 
> 
> Both of these depend on fq_domain but fq_domain is always NULL for intel iommus,
> since iommu/intel/iommu.c reports IOMMU_CAP_DEFERRED_FLUSH:
> 
> -- drivers/iommu/dma-iommu.c:708
> 	if (domain->type == IOMMU_DOMAIN_DMA_FQ &&
> 	    (!device_iommu_capable(dev, IOMMU_CAP_DEFERRED_FLUSH) || iommu_dma_init_fq(domain)))
> 		domain->type = IOMMU_DOMAIN_DMA;
> 
> 
> (Above line numbers are from Kernel 6.17).
> 
> So I'm not sure, this commit does what it should?
> Please let me know what I'm missing here.

IOMMU_CAP_DEFERRED_FLUSH *is* the "I can usefully support flush queues" 
capability; if that is reported then iommu_dma_init_fq() is called, and 
if that succeeds then fq_domain will have been set. If it fails, or if 
the IOMMU doesn't support flush queues in the first place, then we fall 
back to the regular strict domain type.

Thanks,
Robin.

> 
> Best Regards,
> Ferdinand Schober
> 
> 
> 
>