Re: [Qemu-devel] PCI Passthrough KVM with vfio-pci giving EINVAL on PCI Probe

[Alex Williamson <alex.williamson@redhat.com>]

On Wed, 24 Feb 2016 20:04:08 +0000
Nicholas Andre <n@axfp.org> wrote:

> I am attempting to attach PCIe devices in passthrough to my KVM domain. I
> have two devices in one IOMMU group (#28):
> 
> 03:00.0 PCI bridge: Texas Instruments XIO2001 PCI Express-to-PCI Bridge
> 04:00.0 Multimedia audio controller: Texas Instruments TMS320C6414
> TMS320C6415 TMS320C6416
> 
> and lspci -n values:
> 
> 03:00.0 0604: 104c:8240
> 04:00.0 0401: 104c:a106
> 
> Both of which are attached to my KVM domain with managed configuration.
> 
> I am running Ubuntu 14.04. I have:
> 
>    - Installed KVM, set up domains, ensured KVM and vfio-pci modules are
>    loaded
>    - Set intel_iommu=on on the kernel command line and verified it is
>    enabled with dmesg
>    - kvm-ok prints acceleration is all set, VT-d is supported on the chipset
>    - Verified that the 03:00.0 device does not have a driver currently
>    listed under ls -l /sys/bus/pci/devices/0000\:03\:00.0/ though it does
>    have a firmware listed.
> 
> When attempting to start the VM I get:
> 
> virsh # start rivendell
> error: Failed to start domain rivendell
> error: Failed to bind PCI device '0000:03:00.0' to vfio-pci: No such device
> 
> and under syslog I get what I believe to be the PCIe EINVAL code:
> 
> Feb 23 19:46:46 tupac kernel: [  214.159625] vfio-pci: probe of
> 0000:03:00.0 failed with error -22
> 
> Trying to do this manually: I do this to bind the devices to vfio-pci:
> 
> # echo "104c a106" > /sys/bus/pci/drivers/vfio-pci/new_id
> # echo "104c 8240" > /sys/bus/pci/drivers/vfio-pci/new_id
> 
> Then:
> 
> root@tupac:~# echo "0000:03:00.0" > /sys/bus/pci/drivers/vfio-pci/bind
> bash: echo: write error: No such device
> 
> However, the multimedia controller appears AOK:
> 
> root@tupac:~# ls -l /sys/bus/pci/drivers/vfio-pci/
> total 0
> lrwxrwxrwx 1 root root    0 Feb 24 13:07 0000:04:00.0 ->
> ../../../../devices/pci0000:00/0000:00:02.2/0000:03:00.0/0000:04:00.0
> --w------- 1 root root 4096 Feb 24 13:05 bind
> lrwxrwxrwx 1 root root    0 Feb 24 13:07 module -> ../../../../module/vfio_pci
> --w------- 1 root root 4096 Feb 24 13:15 new_id
> --w------- 1 root root 4096 Feb 23 19:47 remove_id
> --w------- 1 root root 4096 Feb 23 19:43 uevent
> --w------- 1 root root 4096 Feb 24 13:07 unbind
> 
> 03:00.0 (which is a PCIe to PCI adapter) is not showing up under vfio-pci's
> devices. They're in the same IOMMU group so I don't think I can attach them
> individually.
> 
> What is causing this? Is this some issue with the presence of the regular
> old PCI bus in the system behind the adapter?

vfio-pci does not support bridges nor is there any reason to bind the
bridge to vfio-pci.  The IOMMU group requirement is satisfied by the
bridge not being bound to any drivers.  We used to have some bridge
drivers whitelisted to allow the group to be viable with them attached,
newer kernels skip testing the driver of bridge devices.  Is there some
specific reason you want to attach this bridge to the VM?  Thanks,

Alex