Re: [PATCH] VFIO driver: Non-privileged user level PCI drivers

[Tom Lyon <pugs@lyon-about.com>]

On Tuesday 01 June 2010 03:46:51 am Michael S. Tsirkin wrote:
> On Tue, Jun 01, 2010 at 01:28:48PM +0300, Avi Kivity wrote:
> > On 06/01/2010 12:55 PM, Michael S. Tsirkin wrote:
> >>
> >>>   It can't program the iommu.
> >>> What
> >>> the patch proposes is that userspace tells vfio about the needed
> >>> mappings, and vfio programs the iommu.
> >>>      
> >> There seems to be some misunderstanding.  The userspace interface
> >> proposed forces a separate domain per device and forces userspace to
> >> repeat iommu programming for each device.  We are better off sharing a
> >> domain between devices and programming the iommu once.
> >>    
> >
> >   iommufd = open(/dev/iommu);
> >   ioctl(iommufd, IOMMUFD_ASSIGN_RANGE, ...)
> >   ioctl(vfiofd, VFIO_SET_IOMMU, iommufd)
> >
> > ?
> 
> Yes.
> 
> > If so, I agree.
> 
> Good.

I'm not really opposed to multiple devices per domain, but let me point out how I
ended up here.  First, the driver has two ways of mapping pages, one based on the
iommu api and one based on the dma_map_sg api.  With the latter, the system
already allocates a domain per device and there's no way to control it. This was
presumably done to help isolation between drivers.  If there are multiple drivers
in the user level, do we not want the same isoation to apply to them?  
Also, domains are not a very scarce resource - my little core i5 has 256, 
and the intel architecture goes to 64K.
And then there's the fact that it is possible to have multiple disjoint iommus on a system,
so it may not even be possible to bring 2 devices under one domain. 

Given all that, I am inclined to leave it alone until someone has a real problem.
Note that not sharing iommu domains doesn't mean you can't share device memory,
just that you have to do multiple mappings