Re: [PATCH 2/2] drivers/arm-smmu-v3: Implement .iotlb_sync_map callback

[Pranjal Shrivastava <praan@google.com>]

On Tue, Sep 30, 2025 at 09:27:21AM +0000, Mostafa Saleh wrote:
> On Tue, Sep 30, 2025 at 12:23:50AM +0000, Pranjal Shrivastava wrote:
> > On Mon, Sep 29, 2025 at 09:47:19AM -0300, Jason Gunthorpe wrote:
> > > On Mon, Sep 29, 2025 at 12:24:28PM +0000, Mostafa Saleh wrote:
> > > > On Mon, Sep 29, 2025 at 08:58:03AM -0300, Jason Gunthorpe wrote:
> > > > > On Sat, Sep 27, 2025 at 10:39:53PM +0000, Daniel Mentz wrote:
> > > > > > @@ -3700,6 +3713,7 @@ static const struct iommu_ops arm_smmu_ops = {
> > > > > >  		.map_pages		= arm_smmu_map_pages,
> > > > > >  		.unmap_pages		= arm_smmu_unmap_pages,
> > > > > >  		.flush_iotlb_all	= arm_smmu_flush_iotlb_all,
> > > > > > +		.iotlb_sync_map		= arm_smmu_iotlb_sync_map,
> > > > > 
> > > > > Shouldn't this avoid defining the op on coherent systems?
> > > > 
> > > > Does that mean we need to have 2 iommu_ops, one for
> > > > coherent/non-coherent SMMUs, as both can be mixed in the same system.
> > > 
> > > Yes, I think you'd have to do it with two ops..
> > > 
> > > It just seems wrong to penalize the normal fast case for these
> > > systems.
> > > 
> > 
> > I see we plan to set defer_sync_pte = true always. What if we invoke the
> > ops->iotlb_sync_map() only for incoherent IOMMUs? Maybe something like:
> > 
> > static int arm_smmu_iotlb_sync_map(struct iommu_domain *domain,
> > 				    unsigned long iova, size_t size)
> > {
> > 	struct io_pgtable_ops *ops = to_smmu_domain(domain)->pgtbl_ops;
> > 	struct arm_smmu_device *smmu = to_smmu_domain(domain)->smmu;
> > 	bool is_coherent = smmu->features & ARM_SMMU_FEAT_COHERENCY;
> > 
> > 
> > 	if (!ops || !ops->iotlb_sync_map || is_coherent)
> > 		return 0;
> > 
> > 	ops->iotlb_sync_map(ops, iova, size);
> > 	return 0;
> > }
> > 
> > If needed we can push the coherency check to the io-pgtable op
> > iotlb_sync_map() as well. Just an idea..
> > 
> 
> iotlb_sync_map is already NULL for coherent SMMUs, I beleive
> Jason's point is about that the iommu_ops.default_domain_ops
> will have the extra pointer which will be called by the core code
> anyway, which immediatly returns; wasting some cylces.

Ohh okay, I see.

> To avoid this we can have 2 sets of the default_domain_ops for
> coherent and non-coherent devices, to be chosen at domain alloc time.
> 

I guess it'd be better to have non-coherent def domain ops then.

> Though, It would be intersting to measure how much overhead does the
> current approach have in practice, maybe through dma_map_benchmark?
>

Yes, dma_map_benchmark can be used but its results won't reflect the
impact on scatter-gather workloads since the benchmark doesn't cover
dma_map_sg IIRC. I believe even a small per-call regression may get
amplified at scale though.

Thanks,
Praan