RE: [RFC PATCH v2 0/7] iommu/arm-smmu-v3: Use pinned KVM VMID for stage 2

[Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>]

> -----Original Message-----
> From: Jason Gunthorpe <jgg@ziepe.ca>
> Sent: Thursday, February 8, 2024 3:36 PM
> To: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> Cc: kvmarm@lists.linux.dev; iommu@lists.linux.dev; linux-arm-
> kernel@lists.infradead.org; linuxarm@huawei.com; kevin.tian@intel.com;
> alex.williamson@redhat.com; maz@kernel.org; oliver.upton@linux.dev;
> will@kernel.org; robin.murphy@arm.com; jean-philippe@linaro.org;
> jonathan.cameron@huawei.com
> Subject: Re: [RFC PATCH v2 0/7] iommu/arm-smmu-v3: Use pinned KVM VMID
> for stage 2
> 
> On Thu, Feb 08, 2024 at 03:18:30PM +0000, Shameer Kolothum wrote:
> > Hi,
> >
> > On an ARM64 system with a SMMUv3 implementation that fully supports
> > Broadcast TLB Maintenance(BTM) feature as part of the Distributed
> > Virtual Memory(DVM) protocol, the CPU TLB invalidate instructions are
> > received by SMMUv3. This is very useful when the SMMUv3 shares the
> > page tables with the CPU(eg: Guest SVA use case). For this to work,
> > the SMMUv3 must use the same VMID that is allocated by KVM to configure
> > the nested stage 2(S2) translations.
> 
> Ah so you are going all the way and looking to enable BTM within a
> vSVA environment too?

Yes, eager to get the vSVA support 😊 Its been ages and tired of rebasing for our
private git for the vSVA cases!.

Also for Host SVA, since we are using S1 now, I don’t think this matters that much.

(I do have some questions on the iommufd based vSVA , but I will come back to it,
once I cleanup my Qemu branch for that.)

> 
> > An earlier proposal sent out[1] a while back resulted in changing the
> > ARM64/KVM VMID allocator similar to the ASID allocator to make it
> > better suited for this.
> >
> > This RFC adds,
> >  -Support for pinned KVM VMID.
> >  -Support associating KVM pointer and iommufd ctx.
> >  -Changes to domain_alloc_user() to receive a kvm pointer.
> >  -Configure SMMUV3 S2 using KVM VMID
> >  -Finally enable BTM only if SMMUV3 supports S1 translation. This
> >   is to make sure that PAGING domains always use S1 and S2 is only
> >   used for nested domains with a valid KVM. The idea is to make sure
> >   when BTM is enabled in Guest, we use KVM VMID for S2.
> >
> > Not sure I miss any explicit TLB invalidations with any use case
> > that may configure a S2 with a private VMID that matches a KVM
> > one.
> >
> > This is based on Jason's ongoing SMMUv3 refactor series[2].
> 
> I'm glad to see this, thank you for finishing the BTM stuff!
> 
> If someone is using it I wonder if we need to get a more solid answer
> on the races with invalidation an ASID reassign.. I added some notes
> in comments in part 2 after I audited all of it.

I saw that one. The main point here is, if we guarantee that BTM is only used
for a nested S2 case and there will be a pinned KVM VMID for that, I am 
not sure we need to worry about any S2 related conflicts in TLBIs.

Thanks,
Shameer