Re: [PATCH RFCv1 05/10] hw/arm/virt: Add VIRT_NESTED_SMMU

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Jul 11, 2024 at 08:48:10AM -0700, Andrea Bolognani wrote:

> IIUC having devices associated to the "correct" SMMU is only a
> requirement for optimal performance, not pure functionality, right?
> In other words, getting the association wrong will make things slower
> but they will keep working.

As part of declaring the iommu as a device libvirt should also specify
the iommu capabilities and features.

There are capabilities that require perfect 1:1 pIOMMU to vIOMMU
matching, or it can't work.

For instance if the pIOMMU has a direct path for invalidations then
the pIOMMU, pPCI, vIOMMU, and vPCI all must be perfectly matched
because the vIOMMU will deliver invalidations directly to a single
pIOMMU with no possibility to steer them to the correct place in
hypervisor SW.

> Is a 1:1 match between pSMMU and vSMMU mandatory? On a system with 4
> SMMUs, could I create a VMs with
> 
>   vSMMU0 <-> pSMMU0
>   vSMMU1 <-> pSMMU1
> 
> and another one with
> 
>   vSMMU0 <-> pSMMU2
>   vSMMU1 <-> pSMMU3

This is fine
 
> assuming of course that devices are assigned to the correct VM?

But devices on pSMMU0 cannot be assigned to the second VM in above
situations.

> How is the association between vSMMU and pSMMU created anyway? Is
> that something that the user can control, or is it done automatically
> somehow?

From a kernel side VIOMMU will be created in IOMMUFD and it will get
as input a reference to a VFIO device. The VIOMMU will be linked to
that VFIO device's pIOMMU.

Also creating a dummy vIOMMU without any devices and thus no way to
reach the pIOMMU is definately "tricky", I'm not sure exactly what is
possible there.

Jason