Re: [PATCH v2 3/8] iommu/arm-smmu-v3: Optimize range invalidation for latency

[Jason Gunthorpe <jgg@nvidia.com>]

> Sorry I lost track of this thread and I just saw v4.
> 
> In the mobile space, I haven't seen an SMMUv3 that does not support
> RIL.

Oh? That's very surprising, AFAIK none of our embedded chips support it
yet.. Even the server chips are only just getting it. Are you sure?

I gather it wasn't even available in ARM IP until recently ish?

> However, I have seen workloads that are really sensitive to translation
> latency (display, camera...). And I'd be concerned about those
> regressing.

Yes, those exist, but again, they are already facing these problems if
running without RIL.

And, for the common case of putting something into a carve out region
it is not so likely even an expanded RIL will intersect with a
reserved IOVA that has a high alignment.

At least the things we have built are calibrated to handle a TLB
reload occasionally. The isochronous TLB's are not even sized to be
never-miss for all cases because things like 4k media require such a
large amount of IOVA the area cost is too high.

While others can do something else you are reaching into a pretty
narrow condition to hit a problem:
 - HW that must have a never-miss TLB to work
 - HW that doesn't have a carve out, or has a badly aligned carve out
 - A SMMU that has RIL (non RIL is already worse)
 - A non-isochronos workload that regularly exceeds the RIL/single
   expansion thresholds
 - Unlucky IOVA allocation that places isochronous near other
   workloads in the IOVA space.

> There is a clear trade-off here as you mentioned with TLBI latency,
> would it be make sense to make that behviour configurable from a
> module param?

I think it makes sense for a driver to indicate to the core code that
it needs isochronous and we can do more global things like change how
single works as well. Having an isochronous flag on the domain, for
example, would be a good overall direction.

I'm inclined to leave this as is and let someone come with a specific
problematic HW, rather that try to badly guess without much
information if it might popssibly be a problem.

Then we will know the HW and can mark the driver as I suggest above.

Jason