Re: RMRR device on non-Intel platform

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Apr 21, 2023 at 09:21:12PM +0800, Baolu Lu wrote:
> On 2023/4/21 20:04, Jason Gunthorpe wrote:
> > @@ -2210,6 +2213,22 @@ static int __iommu_device_set_domain(struct iommu_group *group,
> >   {
> >   	int ret;
> > +	/*
> > +	 * If the driver has requested IOMMU_RESV_DIRECT then we cannot allow
> > +	 * the blocking domain to be attached as it does not contain the
> > +	 * required 1:1 mapping. This test effectively exclusive the device from
> > +	 * being used with iommu_group_claim_dma_owner() which will block vfio
> > +	 * and iommufd as well.
> > +	 */
> > +	if (dev->iommu->requires_direct &&
> > +	    (new_domain->type == IOMMU_DOMAIN_BLOCKED ||
> > +	     new_domain == group->blocking_domain)) {
> > +		dev_warn(
> > +			dev,
> > +			"Firmware has requested this device have a 1:1 IOMMU mapping, rejecting configuring the device without a 1:1 mapping. Contact your platform vendor.");
> > +		return -EINVAL;
> > +	}
> > +
> >   	if (dev->iommu->attach_deferred) {
> >   		if (new_domain == group->default_domain)
> >   			return 0;
> 
> How about enforcing this in iommu_group_claim_dma_owner() 

It is more general here, since this applies to any attempt to attach a
blocking domain, eg if we future miscode something else it will still
be protected. It is subtle enough, and we all missed this for a long
time already I prefer we be robust.

> and change the iommu drivers to use "atomic replacement" instead of
> blocking translation transition when switching to a new domain?

That seems unlikely to happen on a broad scale..

> Assuming that the kernel drivers should always use the default
> domain, or handle the IOMMU_RESV_DIRECT by themselves if they decide
> to use its own unmanaged domain for kernel DMA.

Long term we want to get to the point where all kernel drivers call
a claim_dma_owner before they mess with the domains. This is our
locking protocol to say that it is actually safe to do it.

If we reach a point where a kernel driver wants to make its own domain
and needs to work with FW that pushes IOMMU_RESV_DIRECT for its device
then we should add a claim_dma_owner variant for trusted users that
avoids using the blocking domain.

Jason