Re: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU

[Jason Gunthorpe <jgg@ziepe.ca>]

On Tue, Sep 05, 2023 at 11:48:31AM +0530, Vasant Hegde wrote:

> Reading through the discussion so far again and the other series, my
> understanding is :
>   - set_dev_pasid() will check the compatibility and bind device/pasid only if
> its compatibility. In AMD case we will check against protection domain. Ex:
>  If we have two devices (devA and devB) in two different protection domain then:
>    set_dev_pasid(sva_domain, devA, pasidX) - SUCCESS
>    set_dev_pasid(sva_domain, devB, pasidX) - Compatibility check fail
>    Core will allocate new SVA domain (sva_domain_new)
>    set_dev_pasid(sva_domain_new, devB, pasidX) - SUCCESS

I don't expect a compatability check to ever fail on an AMD driver -
what condition do you imagine where re-allocating a SVA domain will
make it work with a device?

>   - We will track mmu notifier and other data required for invalidation in SVA
> protection domain.

Yes
 
>   - During invalidation, we will retrieve SVA protection domain using mmu
> notifier. Use device protection domain which was tracked in this SVA domain for
> invalidation.

The iommu_domain/protection_domain must NOT be 1:1 with a device. The
driver must maintain a list of devices attached to the domain, and the
per-device-attachment parameters like PASID/cache tags/etc.

This is very important.

> > It is not the same, you have this weird sva_pasid thing in here. PASID
> > is NOT part of the SVA layer.
> > 
> > The API expects UNMANAGED domains will support PASID attach as well,
> > that is a significant use case.
> 
> Can you elaborate the use cases you are referring here?
> 
> We do have use cases for PASID and PASID+PRI. But I am not aware of any use case
> for UNMANAGED domain.

The iommu API is evolving so there are only a few domain types:
BLOCKED, IDENTITY, PAGING, SVA, OPAQUE

PAGING is what we today call UNMANAGED/DMA/DMA_FQ

PAGING domains need to support PASID+PRI, in today's language that
means UNMANAGED domains.

We have many use cases for PRI support with generic PAGING domains.

SVA is a special case of a PAGING domain where there is no map/unmap/invalidate
API and the IO page table comes from a mm_struct.

Start by making UNMANAGED (aka PAGING) domains work with PASID + PRI
and then SVA is a very small incremental step.

Jason