Re: kernel BUG at drivers/iommu/intel-iommu.c:732! , kernel 4.14.39

[Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>]

On Tue, 22 May 2018 13:30:20 +0200
plexo rama <plexorama-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

> Hi all,
> 
> 
> I receive the following kernel dump when trying to load QLogic qede in-tree
> driver for a dual-port
> 
> *QLogic* Corp. FastLinQ QL41000 Series 10/25/40/50GbE Controller (rev 02).
> One of the ports (45:00.0) is bound to the vfio-pci driver for
> pci-passthrough to a KVM/QEMU guest. Id like to use the PF of the second
> port on the host. I've unbound the pci device (45:00.1) from vfio-pci, but when
> I modprobe qede, I receive this:

AIUI, this device does not provide a PCIe ACS capability, therefore the
kernel assumes the functions of the device are not isolated and places
them in the same IOMMU group.  The vendor has also not indicated that
the PCIe functions are DMA isolated nor provided quirks to indicate any
sort of isolation.  vfio-pci requires DMA isolation between users,
whether they be separate userspace instances or between kernel and
users.  Via the process above, assuming 45:00.0 is in use by a user
instance when it occurs, you've compromised the isolation.  Generally
the response to this is a BUG in the vfio code, but in the case of this
driver it attempts to map DMA in the probe routine and therefore
encounters a different BUG where one device within the group has a
mapping through the IOMMU API and the other is attempting to map
through the DMA API, where the domains within intel-iommu are
incompatible.

In short, the device you're using doesn't support the DMA isolation
necessary for the use case you're trying to impose on it and the kernel
protects itself by preventing it, harshly.  Perhaps ask QLogic if
there's actually ACS equivalent isolation provided between PCIe
functions of the device and we can implement quirks for it.  Otherwise
you'll need a separate card to use for the host.  Thanks,

Alex