Re: Cache Invalidation Solution for Nested IOMMU

[Nicolin Chen <nicolinc@nvidia.com>]

On Wed, Apr 05, 2023 at 08:37:56AM -0300, Jason Gunthorpe wrote:
> On Tue, Apr 04, 2023 at 10:45:56PM -0700, Nicolin Chen wrote:
> > On Tue, Apr 04, 2023 at 01:20:01PM -0300, Jason Gunthorpe wrote:
> > > On Mon, Apr 03, 2023 at 05:02:09PM -0700, Nicolin Chen wrote:
> > > 
> > > > My preference is to have a mmap'd page, so the interface can
> > > > be reused later by VCMDQ too. Performance-wise, it should be
> > > > good enough, since it does batching, IMHO.
> > > 
> > > You can't reuse mmaping the queue page with vcmdq, so it doesn't seem
> > > meaningful to me.
> > > 
> > > There should be no mmap on the SW path. If you need a half step
> > > between an ioctl as a batch and a full vhost-like queue scheme then
> > > using iouring with pre-registered memory would be appropriate.
> > 
> > I've changed to a non-mmap approach that the host kernel reads
> > the guest queue directly and inserts all invalidation commands
> > into the host queue.
> > 
> > The qsz could be as large as 128 x 64K pages. So, there has to
> > be a big array of pages getting pinned in the handler.
> > 
> > (The handler still needs a pathway to report errors. I will add
> >  tomorrow.)
> > 
> > Does the implementation below look fine in general?
> 
> In general get_user_pages() is really slow compared to copy_from_user

Will change to using copy_from_user().

Thanks!
Nic