Re: [Qemu-devel] [RFC PATCH] vfio: VFIO PCI driver for Qemu

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 2012-07-26 at 19:40 +0300, Avi Kivity wrote:
> On 07/26/2012 07:33 PM, Alex Williamson wrote:
> >> 
> >> In the common case, on x86 (but I'm repeating myself), the iommu group
> >> includes just one device, yes?  Could we make pci-stub an alias for the
> >> corresponding vfio steps?
> > 
> > PCI bridges masking devices is not as uncommon as you'd like, that's
> > exactly why Andreas is using VFIO instead of KVM assignment.  
> 
> Well, we are using it in production for quite a while with few such reports.

In the enterprise space, sure.  In the hobbiest/power user space, I
suspect users are too often finding that it doesn't work and move on to
something else.  Maybe we'll know if it's working better if we get more
complaints about random oddball devices not working because people are
actually able to get far enough to try it.

> > Not to
> > mention that VFIO takes a much more strict stance on multifunction ACS
> > requirements, typically resulting in all function of a multifunction
> > device being inseparable.  So no, I don't think multi-device groups will
> > be unusual at all, even on x86.  Playing games with pci-stub sounds like
> > a nightmare.  Personally I think we have the opportunity to make libvirt
> > and tools like virt-manager a lot better with VFIO.  They no longer need
> > to do PCI bridge testing or ACS checking for VFIO and they can better
> > inform the user about what devices need to be removed from the host to
> > provide safe assignment.
> > 
> > 
> > If we have both vfio and kvm assignment in the same tree there's no
> > reason we couldn't intermix them within a VM.  Unfortunately we have to
> > beware of KVM assignment's poor ownership model, but that's true whether
> > the device is attached to vfio-pci or some other driver.  Maybe we
> > should prevent that, but I see that happening by deprecating KVM
> > assignment and eventually disabling and removing it.  Thanks,
> 
> That's the plan.  By making the command lines compatible, we allow
> upgrading the kernel and qemu, but keeping libvirt or another stack, and
> more importantly their config files, unchanged.

That's why I think libvirt should do it, there's no reason libvirt can't
detect that vfio is available and default to it.  Nothing in the xml
file needs to change.  We can add an option to specify what backend to
use for people that care.  If someone is launching qemu from a script
they hit the same setup issues we've been discussing for vfio-pci, so
there's not much to preserve there.

> Perhaps we could do it part-way by making pci-assign do the magic needed
> to switch from pci-stub to an iommu group and forwarding the device to
> vfio-pci.  But it would probably be root-only.

That would definitely be root only and perhaps even only works in some
cases, assuming we decide we can only do it for single-device groups.
Then we get partial breakage and users left wondering why the VM with a
NIC assigned works, but the VM with a TV card fails.  As a user, I hate
those kinds of problems.  Thanks,

Alex