Re: Cache Invalidation Solution for Nested IOMMU

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Apr 04, 2023 at 04:50:14PM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Jason Gunthorpe [mailto:jgg@nvidia.com]
> > Sent: 04 April 2023 17:20
> > To: Nicolin Chen <nicolinc@nvidia.com>
> > Cc: Robin Murphy <robin.murphy@arm.com>; kevin.tian@intel.com;
> > yi.l.liu@intel.com; eric.auger@redhat.com; baolu.lu@linux.intel.com;
> > Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> > jean-philippe@linaro.org; iommu@lists.linux.dev
> > Subject: Re: Cache Invalidation Solution for Nested IOMMU
> > 
> > On Mon, Apr 03, 2023 at 05:02:09PM -0700, Nicolin Chen wrote:
> > 
> > > My preference is to have a mmap'd page, so the interface can
> > > be reused later by VCMDQ too. Performance-wise, it should be
> > > good enough, since it does batching, IMHO.
> > 
> > You can't reuse mmaping the queue page with vcmdq, so it doesn't seem
> > meaningful to me.
> > 
> > There should be no mmap on the SW path. If you need a half step
> > between an ioctl as a batch and a full vhost-like queue scheme then
> > using iouring with pre-registered memory would be appropriate.
> > 
> > I feel like this is a topic Shameerali should share some insight with
> > since the Huawei implementation will rely on this SW path.
> 
> So far the tests we have done are mainly using IOCTL method without
> batching and I don't have any numbers to compare against the batched
> method yet.

The question is more are you happy with the performance of single
ioctl or does this need optimization. It sits in a hot path of the mm,
so it broadly impacts certain workloads once SVA is enabled.

Jason