Re: [PATCH v8 07/15] iommupt: Add map_pages op

From: Jason Gunthorpe <jgg@nvidia.com>
To: Alexey Kardashevskiy <aik@amd.com>
Cc: Alexandre Ghiti <alex@ghiti.fr>, Anup Patel <anup@brainfault.org>,
	Albert Ou <aou@eecs.berkeley.edu>,
	Jonathan Corbet <corbet@lwn.net>,
	iommu@lists.linux.dev, Joerg Roedel <joro@8bytes.org>,
	Justin Stitt <justinstitt@google.com>,
	linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org,
	linux-riscv@lists.infradead.org, llvm@lists.linux.dev,
	Bill Wendling <morbo@google.com>,
	Nathan Chancellor <nathan@kernel.org>,
	Nick Desaulniers <nick.desaulniers+lkml@gmail.com>,
	Miguel Ojeda <ojeda@kernel.org>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Walmsley <pjw@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Shuah Khan <shuah@kernel.org>,
	Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
	Will Deacon <will@kernel.org>,
	Alejandro Jimenez <alejandro.j.jimenez@oracle.com>,
	James Gowans <jgowans@amazon.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Michael Roth <michael.roth@amd.com>,
	Pasha Tatashin <pasha.tatashin@soleen.com>,
	patches@lists.linux.dev, Samiullah Khawaja <skhawaja@google.com>,
	Vasant Hegde <vasant.hegde@amd.com>
Subject: Re: [PATCH v8 07/15] iommupt: Add map_pages op
Date: Fri, 27 Feb 2026 09:48:05 -0400	[thread overview]
Message-ID: <20260227134805.GJ5933@nvidia.com> (raw)
In-Reply-To: <c6ac32f2-2531-4bd0-abaf-ff76e9cc864e@amd.com>

On Fri, Feb 27, 2026 at 12:39:28PM +1100, Alexey Kardashevskiy wrote:
> 
> 
> On 27/2/26 02:04, Jason Gunthorpe wrote:
> > On Thu, Feb 26, 2026 at 10:11:56AM +1100, Alexey Kardashevskiy wrote:
> > > > The flow would be some thing like..
> > > >    1) Create an IOAS
> > > >    2) Create a HWPT. If there is some known upper bound on RMP/etc page
> > > >       size then limit the HWPT page size to the upper bound
> > > >    3) Map stuff into the ioas
> > > >    4) Build the RMP/etc and map ranges of page granularity
> > > >    5) Call iommufd to adjust the page size within ranges
> > > 
> > > I am about to try this approach now. 5) means splitting bigger pages
> > > to smaller and I remember you working on that hitless IO PDEs
> > > smashing, do you have something to play with? I could not spot
> > > anything on github but do not want to reinvent. Thanks,
> > 
> > I thought this thread had concluded you needed to use the HW engines
> 
> The HW engine has to be used for smashing while DMAing to 2M page
> being smashed. It is not needed when the insecure->trusted switch
> happens and IOMMU now needs to match already configured RMP.

Oh? I'm surprised shared->private is different that private->shared..

Regardless, I think if you go this path you have to stick to 4k IOPTEs
and avoid the HW engine. Maybe that is good enough to start.

> > for this and if so then KVM should maintain the IOMMU S2 where it can
> > synchronize things and access the HW engines?
> 
> I want to explore the idea of using the gmemfd->iommufd notification
> mechanism for smashing too (as these smashes are always the result
> of page state changes and this requires a notification on its own as
> we figured out) and plumb that HW engine to the IOMMU side,
> somewhere in the AMD IOMMU driver. Hard to imagine KVM learning
> about IOMMU.

Equally hard to imagine IOMMU changing the RMP.. Since you explained
the HW engine changes both I don't know what you will do.

Maybe guestmemfd needs to own the RMP updates and it can somehow
invoke the HW engine and co-ordinate all the parts. This sounds very
hard as well, so IDK.

Jason