Re: KVM pci-assign - iommu width is not sufficient for mapped address

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 2016-01-07 at 15:48 +0530, Shyam wrote:
> Hi All,
> 
> We are using Linux Kernel 3.18.19 for running KVM VM's with
> pci-assign'ed SRIOV VF interfaces.
> 
> We understand that VFIO is the new recommended way, but unfortunately
> it reduces performance significantly on our IO workloads (upto the
> order of 40-50%) when compared to pci-passthrough. We run trusted
> VM's
> & expose services to the external world. Since we control the VM's,
> IOMMU security with VFIO is not exactly mandatory, but performance is
> much more important that we get with pci-assign.
> 
> We observe a strange behaviour that has already been discussed in
> this
> forum which is upon a VM spawn it causes the following fault
> resulting
> in qemu-kvm crashing
> 
> Jan  7 09:41:57 q6-s1 kernel: [90037.228477] intel_iommu_map: iommu
> width (48) is not sufficient for the mapped address
> (fffffffffe001000)
> Jan  7 09:41:57 q6-s1 kernel: [90037.308229]
> kvm_iommu_map_address:iommu failed to map pfn=95000
> 
> We observe that this problem happens only if guest linux running 3.5
> kernel is spun up & this problem doesnt happen when running guest
> linux with 3.6 kernel (i.e. all guest with kernels like 3.2 etc up
> till 3.5 causes the above crash whereas any guest kernel >=3.6 doesnt
> cause this issue).
> 
> So something changed between kernel 3.5 to 3.6 in the guest that
> doesnt expose this problem. We have two questions:
> 1 - we understand that VFIO suffered a similar problem & it was fixed
> with https://github.com/qemu/qemu/commit/d3a2fd9b29e43e202315d5e99399
> b99622469c4a.
> Alex Williamson suggested that KVM driver needs an equivalent version
> of the fix. Can anybody suggest hints on where this fix should be
> made?
> 2 - Any insights on what changes in linux kernel between 3.5 to 3.6
> on
> the guest that avoids this problem?
> 
> Any helps/input greatly appreciated. Thanks!

Legacy KVM device assignment is deprecated, so I'd suggest your efforts
are better spent reporting and trying to fix any performance difference
you're seeing between pci-assign and vfio-pci.  I have a really hard
time believing there's anywhere close to a 40-50% difference.  What's
the device?  What's the workload?  At some point you're likely to find
that pci-assign is no longer even present.  Thanks,

Alex