Re: [PATCH rfcv3 15/21] intel_iommu: Bind/unbind guest page table to host

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Jun 16, 2025 at 08:14:27PM -0700, Nicolin Chen wrote:
> On Mon, Jun 16, 2025 at 08:15:11AM +0000, Duan, Zhenzhong wrote:
> > >IIUIC, the guest kernel cmdline can switch the mode between the
> > >stage1 (nesting) and stage2 (legacy/emulated VT-d), right?
> > 
> > Right. E.g., kexec from "intel_iommu=on,sm_on" to "intel_iommu=on,sm_off",
> > Then first kernel will run in scalable mode and use stage1(nesting) and
> > second kernel will run in legacy mode and use stage2.
> 
> In scalable mode, guest kernel has a stage1 (nested) domain and
> host kernel has a stage2 (nesting parent) domain. In this case,
> the VFIO container IOAS could be the system AS corresponding to
> the kernel-managed stage2 domain.
> 
> In legacy mode, guest kernel has a stage2 (normal) domain while
> host kernel has a stage2 (shadow) domain? In this case, the VFIO
> container IOAS should be the iommu AS corresponding to the kernel
> guest-level stage2 domain (or should it be shadow)?

What you want is to disable HW support for legacy mode in qemu so the
kernel rejects sm_off operation.

The HW spec is really goofy, we get an ecap_slts but it only applies
to a PASID table entry (scalable mode). So the HW has to support
second stage for legacy always but can turn it off for PASID?

IMHO the intention was to allow the VMM to not support shadowing, but
it seems the execution was mangled.

I suggest fixing the Linux driver to refuse to run in sm_on mode if
the HW supports scalable mode and ecap_slts = false. That may not be
100% spec compliant but it seems like a reasonable approach.

> The ARM model that Shameer is proposing only allows a nested SMMU
> when such a legacy mode is off. This simplifies a lot of things.
> But the difficulty of the VT-d model is that it has to rely on a
> guest bootcmd during runtime..

ARM is cleaner because it doesn't have these drivers issues. qemu can
reliably say not to use the S2 and all the existing guest kernels will
obey that.

AMD has the same issues, BTW, arguably even worse as I didn't notice
any way to specify if the v1 page table is supported :\

Jason