Re: [PATCH v2 8/8] iommu/arm-smmu-v3: Support IOMMU_DOMAIN_NESTED

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Aug 30, 2024 at 04:09:04PM +0000, Mostafa Saleh wrote:
 
> On Tue, Aug 27, 2024 at 12:51:38PM -0300, Jason Gunthorpe wrote:
> > For SMMUv3 a IOMMU_DOMAIN_NESTED is composed of a S2 iommu_domain acting
> > as the parent and a user provided STE fragment that defines the CD table
> > and related data with addresses translated by the S2 iommu_domain.
> >
> > The kernel only permits userspace to control certain allowed bits of the
> > STE that are safe for user/guest control.
> >
> > IOTLB maintenance is a bit subtle here, the S1 implicitly includes the S2
> > translation, but there is no way of knowing which S1 entries refer to a
> > range of S2.
> >
> > For the IOTLB we follow ARM's guidance and issue a CMDQ_OP_TLBI_NH_ALL to
> > flush all ASIDs from the VMID after flushing the S2 on any change to the
> > S2.
> >
> > Similarly we have to flush the entire ATC if the S2 is changed.
> >
> 
> I am still reviewing this patch, but just some quick questions.
> 
> 1) How does userspace do IOTLB maintenance for S1 in that case?

We do all the TLBI/ATC/CD invalidations via the VIOMMU uapi:
https://lore.kernel.org/linux-iommu/cover.1724776335.git.nicolinc@nvidia.com/

In another word, nesting support requires the VIOMMU p1 series
at least.

> 2) Is there a reason the UAPI is designed this way?
> The way I imagined this, is that userspace will pass the pointer to the CD
> (+ format) not the STE (or part of it).
> Making user space messing with shareability and cacheability of S1 CD access
> feels odd. (Although CD configure page table access which is similar).

Given that STE.S1ContextPtr itself is an IPA/GPA, it feels to me
that the HW is designed in such a fashion of user space managing
the CD table and its entries?

CD cache will be flushed if CFGI_CD{_ALL} is trapped. This would
be the same if we pass CD info via the uAPI.

What's the concern for shareability?

Thanks
Nicolin