Re: [PATCH RFCv2 09/13] iommufd: Add IOMMU_OPTION_SW_MSI_START/SIZE ioctls

[Jason Gunthorpe <jgg@nvidia.com>]

On Wed, Jan 29, 2025 at 02:44:12PM +0100, Eric Auger wrote:
> Hi,
> 
> 
> On 1/11/25 4:32 AM, Nicolin Chen wrote:
> > For systems that require MSI pages to be mapped into the IOMMU translation
> > the IOMMU driver provides an IOMMU_RESV_SW_MSI range, which is the default
> > recommended IOVA window to place these mappings. However, there is nothing
> > special about this address. And to support the RMR trick in VMM for nested
> well at least it shall not overlap VMM's RAM. So it was not random either.
> > translation, the VMM needs to know what sw_msi window the kernel is using.
> > As there is no particular reason to force VMM to adopt the kernel default,
> > provide a simple IOMMU_OPTION_SW_MSI_START/SIZE ioctl that the VMM can use
> > to directly specify the sw_msi window that it wants to use, which replaces
> > and disables the default IOMMU_RESV_SW_MSI from the driver to avoid having
> > to build an API to discover the default IOMMU_RESV_SW_MSI.
> IIUC the MSI window will then be different when using legacy VFIO
> assignment and iommufd backend.

? They use the same, iommufd can have userspace override it. Then it
will ignore the reserved region.

> MSI reserved regions are exposed in
> /sys/kernel/iommu_groups/<n>/reserved_regions
> 0x0000000008000000 0x00000000080fffff msi
 
> Is that configurability reflected accordingly?

?

Nothing using iommufd should parse that sysfs file.
 
> How do you make sure it does not collide with other resv regions? I
> don't see any check here.

Yes this does need to be checked, it does look missing. It still needs
to create a reserved region in the ioas when attaching to keep the
areas safe and it has to intersect with the incoming reserved
regions from the driver.

> > + * @IOMMU_OPTION_SW_MSI_START:
> > + *    Change the base address of the IOMMU mapping region for MSI doorbell(s).
> > + *    It must be set this before attaching a device to an IOAS/HWPT, otherwise
> > + *    this option will be not effective on that IOAS/HWPT. User can choose to
> > + *    let kernel pick a base address, by simply ignoring this option or setting
> > + *    a value 0 to IOMMU_OPTION_SW_MSI_SIZE. Global option, object_id must be 0

> I think we should document it cannot be put at a random place either.

It can be put at any place a map can be placed.

That also needs to be checked when creating a domain, it can't be
outside the geometry.

Jason