Re: Cache Invalidation Solution for Nested IOMMU

[Nicolin Chen <nicolinc@nvidia.com>]

On Mon, Apr 03, 2023 at 08:00:12AM +0000, Tian, Kevin wrote:
> External email: Use caution opening links or attachments
> 
> 
> > From: Nicolin Chen <nicolinc@nvidia.com>
> > Sent: Monday, April 3, 2023 8:34 AM
> >
> > The new set_rid/unset_rid ioctls and the mmap interface would be
> > essential for VCMDQ support that we'd like to achieve at the end
> > of this journey. So, personally I'd like to see it can be used at
> > this stage, by the generic SMMUv3 (and potentially VT-d) too.
> >
> 
> We talked earlier that there could be multiple VCMDQ's when the
> guest is assigned multiple devices behind different SMMU's. How
> does the mmap interface per iommufd work in that scenario?

Trying to documenting that each IOMMUFD object can possibly have
a shared page, the mmap interface takes the index of an IOMMUFD
object ID. So, either a pt_id(S1) or a dev_id should be able to
identify which physical SMMU, I think.

> and looks this is different from the requirement of having a
> software short path in kernel to reduce the invalidation overhead
> for emulated vIOMMUs. In this case the invalidation queue is
> in guest memory then instead we want a registration cmd here.

Yes for the first part. There are certain difficulties of doing
a short path, such as host kernel replacing the host queue that
the actual HW ran on, with a guest TLBI queue. So, my draft is
more about batching.

For the last part, what's that "registration cmd" to do? In my
draft, the hypervisor dispatch all invalidation commands to the
guest TLBI queue (or call it user queue), which is transparent
to the guest OS.

Thanks
Nicolin