Re: [RFC PATCHv2 0/3] Support for nesting IOMMUs in VFIO

[Will Deacon <will.deacon-5wv7dgnIgG8@public.gmane.org>]

On Thu, Jul 10, 2014 at 05:14:05PM +0100, Alex Williamson wrote:
> On Thu, 2014-07-10 at 16:45 +0100, Will Deacon wrote:
> > On Thu, Jul 10, 2014 at 04:38:18PM +0100, Alex Williamson wrote:
> > > On Thu, 2014-07-10 at 12:34 +0100, Will Deacon wrote:
> > > > Do you think that returning something like -EOPNOTSUPP from an attach is
> > > > sufficient for userspace to figure out what's gone wrong?
> > > 
> > > It seems like this would all be a little easier if we had some sort of
> > > bus_type iterator, then when we want to see if nesting is supported we'd
> > > iterate the bus_types, test iommu_present(), iommu_domain_alloc(),
> > > iommu_domain_set_attr(), iommu_domain_free().  We'd only allow a nesting
> > > domain to be set if the IOMMU driver for at least one bus_type supports
> > > it.  Then I think enforcing it would be a little more obvious to
> > > userspace.  There is a bus_kset, but we'd need an iterator or at least
> > > to enable find_bus() and have type1 walk a list of bus names known to
> > > possibly support nesting (uglier, but functional).  Thanks,
> > 
> > That would work fine if we only had one IOMMU instance per bus. The problem
> > is, the ARM SMMU driver (at least) can handle multiple SMMU instances on a
> > single bus, only some of which may be capable of nesting. Until we've done
> > an attach, it's impossible to know what the capabilities are (since the
> > attach is what binds the domain to a physical IOMMU).
> > 
> > So you'd actually need to iterate the bus_type, test iommu_present(),
> > iommu_domain_alloc(), iommu_domain_attach(), iommu_domain_free(). The
> > attach, of course, then requires devices which means you ultimately end up
> > rolling VFIO_SET_IOMMU and VFIO_SET_CONTAINER into a single operation. I
> > don't think that helps the group abstraction much!
> 
> I don't really see how that diminishes the value.  It still means that
> it's only possible to set the IOMMU type of a container to nesting if
> it's possible to support a nesting IOMMU.  The user would get an error
> trying to attach any group to the container that can't be supported as
> nesting, whether that's behind the wrong bus_type or behind the wrong
> IOMMU in the correct bus_type.  The iommu-sysfs extensions that recently
> went into the iommu next branch would help to expose this kind of
> information to the user.
> 
> It seems like an improvement versus statically advertising support for
> something that actually cannot be enabled.  Thanks,

Ok, I suppose it's an improvement. Given that we don't have a bus_type
iterator, would you object if I only do this for the pci_bus_type for now
(i.e. the only bus that's VFIO currently supports).

Will