Re: [PATCH RFCv1 07/14] iommufd: Add viommu set/unset_dev_id ops

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, May 24, 2024 at 05:24:11AM +0000, Tian, Kevin wrote:
> > > > > > On Tue, May 14, 2024 at 06:59:07PM -0700, Nicolin Chen wrote:
> > > > > > > So, you want a proxy S1 domain for a device to attach, in case
> > > > > > > of a stage-2 only setup, because an S2 domain will no longer has
> > > > > > > a VMID, since it's shared among viommus. In the SMMU driver case,
> > > > > > > an arm_smmu_domain won't have an smmu pointer, so a device
> > can't
> > > > > > > attach to an S2 domain but always an nested S1 domain, right?
> > > > > >
> > > > > > That seems like a simple solution to the VMID lifetime, but it means
> > > > > > the kernel has to decode more types of vSTE.

> > And the narrative at the top was trying to describe the links:
> >   [ device ] => [ proxy identity S1 ] => [ viommu [ shared S2 ] ]
> > v.s.
> >   [ device ] => [ non-shareable S2 ]
> >
> > So the first case can take advantage of VIOMMU_INVALIDATE v.s.
> > the second case requires a DEV_INVALIDATE.
> 
> and one side-effect in the first case is to save one VMID for
> non-shareable S2 hence improves iotlb efficiency.

Hmm, how is that?

VMID is currently stored in an S2 domain, actually. The viommu
is a VMID holder to potentially decouple VMID from S2 domain,
because VMID is per SMMU instance while S2 domain is shareable:
   [ dev0 ] => [ S1 dom0 ] => [ viommu0 (VMID0) [ shared S2 ] ]
   [ dev1 ] => [ S1 dom1 ] => [ viommu1 (VMID1) [ shared S2 ] ]

By the way, we can also have (very likely our initial version):
   [ dev0 ] => [ S1 dom0 ] => [ viommu0 [ non-sharable S2 dom0 (VMID0) ] ]
   [ dev1 ] => [ S1 dom1 ] => [ viommu1 [ non-sharable S2 dom1 (VMID1) ] ]

Thanks
Nicolin