Re: Cache Invalidation Solution for Nested IOMMU

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Apr 03, 2023 at 07:26:28AM +0000, Liu, Yi L wrote:

> For VT-d, except for the device-TLB invalidation, there is no RID
> information in the invalidation command submitted by iommu driver.
> Device-TLB invalidation would be somehow an included operation in
> IOTLB invalidation if the page table is used by devices that has
> enabled ATS. So guest VT-d iommu driver may not use the Device-TLB
> invalidation. So vRID->pRID conversion may not happen for VT-d side.

It is the same as ARM, the RID translation is only needed to support
the ATS case.

> Domain_id is not global today. So if there are multiple vIOMMU instance
> exposed to guest, the vDomain_id would have conflict between the
> invalidation commands submitted via different vIOMMU instances.

You'd need to expose multiple physical instances as multiple virtual
instances.

> we may make its allocation global, I'm also curious when and how should
> we build up the vDomain_id->pDomain_id relationship in kernel. Maybe a
> new iommufd uapi just like the vRID set? 

I'm not so sure it even works sensibly for VT-d. Don't you also have
to convert the type of invalidation as well? Eg the nested mode uses
the DID quite differently than the guest that is not nesting.

IIRC you end up converting vDIDs into pPASID/pDIDs and doing a PASID
invalidation.

The point of doing all this stuff is to avoid invalidation
multiplication, one guest invalidation should translate into one host
invalidation. When properly setup the guest invalidations should not
iterate over host objects..

Jason