Re: [PATCH v8 07/15] iommupt: Add map_pages op

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Jan 21, 2026 at 12:08:19PM +1100, Alexey Kardashevskiy wrote:
> > I mean that the HW requires multiple SW controlled tables to all be
> > sizes must be matched. Instead the HW should read all the tables and
> > compute the appropriate smallest size automatically.
> 
> Not sure I follow. IOMMU table matches the QEMU page table, it is
> two tables already and IOMMU cannot just blindly use 2M PTEs if the
> guest is backed with 4K pages.

That is just because AMD HW can't handle it.

For example if you look at the CPU when the guest S1 page table has a
1G PTE and the KVM S2 has a 2M PTE the CPU doesn't explode, it walks
the S1, walks the S2 and loads a 2M PTE into the TLB.

This issue with the RMP is no different, if you get a 2M IOPTE then
the HW should check the RMP and load in a 4K IOPTE to the IOTLB if
that is what the RMP requires.

That the HW doesn't do that means you have all these difficult
problems.

> > I don't think you need hitless here, if the guest is doing
> > encrpyed/decrypted conversions then it can be expected to not do DMA
> > at the same time, or at least it is OK if DMA during this period
> > fails.
> 
> The guest converts only a handful of 4Ks (say, the guest userspace
> wants to read certificates from guest-os->host-os->fw) and only that
> converted part is not expected for DMA but the rest of 2MB page is
> DMA-able.

Yes, that's very true!

> > So long as the VMM gets a chance to fix the iommu before the guest
> > understands the RMP change is completed it would be OK.
> 
> The IOMMU HW needs to understand the change too. After I smash IO
> PDE, there is a small window before smashing an RMP entry when
> incoming trafic may hit not-converted part of a 2MB page and RMP
> check in the IOMMU will fail. That mentioned above HW+FW engine can
> stall DMA for a few ms while it is smashing things.

> oh but I can :) It is a FW call which takes a pointer to an 2MB
> IOPDE, a new table of 4K PTEs filled with the old PDE's pfn plus
> offsets and then the FW exchanges the old IOPDE with a new table and
> smashes the corresponding RMP, and it suspends the DMA while doing
> so.

That's a completely grotesque solution! 

It violates all of our software layers. The IOMMU and RMP are not
controled by the same software entity and you propose to have a FW
call that edits *both* together somehow? How is that even going to
work safely?

Can't you do things in a sequence?

Change the iommu from 2M to 4K, flush, then change the RMP from 2M to
4K?

> If I get it right, for other platforms, the entire IOMMU table is
> going to live in a secure space so there will be similar FW calls so
> it is not that different.

At least ARM the iommu S2 table is in secure memory and the secure FW
keeps it 1:1 with the KVM S2 table. So edits to the KVM automatically
make matching edits to the IOMMU. Only one software layer is
responsible for things.

That is *very* different from saying that kvm or iommu has to go and
reach into the other subsystem and edit their in-memory structures.

Currently kvm has no idea about the iommu.

So if you want to make use of that you have to solve this fundamental
issue that we can't issue the FW call without some security
synchronization and locking between KVM and iommu.

Jason