From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: KVM pci-assign - iommu width is not sufficient for mapped address Date: Thu, 07 Jan 2016 07:10:12 -0700 Message-ID: <1452175812.29599.132.camel@redhat.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE To: Shyam , kvm@vger.kernel.org Return-path: Received: from mx1.redhat.com ([209.132.183.28]:47615 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752062AbcAGOKN (ORCPT ); Thu, 7 Jan 2016 09:10:13 -0500 In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Thu, 2016-01-07 at 15:48 +0530, Shyam wrote: > Hi All, >=20 > We are using Linux Kernel 3.18.19 for running KVM VM's with > pci-assign'ed SRIOV VF interfaces. >=20 > We understand that VFIO is the new recommended way, but unfortunately > it reduces performance significantly on our IO workloads (upto the > order of 40-50%) when compared to pci-passthrough. We run trusted > VM's > & expose services to the external world. Since we control the VM's, > IOMMU security with VFIO is not exactly mandatory, but performance is > much more important that we get with pci-assign. >=20 > We observe a strange behaviour that has already been discussed in > this > forum which is upon a VM spawn it causes the following fault > resulting > in qemu-kvm crashing >=20 > Jan=C2=A0=C2=A07 09:41:57 q6-s1 kernel: [90037.228477] intel_iommu_ma= p: iommu > width (48) is not sufficient for the mapped address > (fffffffffe001000) > Jan=C2=A0=C2=A07 09:41:57 q6-s1 kernel: [90037.308229] > kvm_iommu_map_address:iommu failed to map pfn=3D95000 >=20 > We observe that this problem happens only if guest linux running 3.5 > kernel is spun up & this problem doesnt happen when running guest > linux with 3.6 kernel (i.e. all guest with kernels like 3.2 etc up > till 3.5 causes the above crash whereas any guest kernel >=3D3.6 does= nt > cause this issue). >=20 > So something changed between kernel 3.5 to 3.6 in the guest that > doesnt expose this problem. We have two questions: > 1 - we understand that VFIO suffered a similar problem & it was fixed > with https://github.com/qemu/qemu/commit/d3a2fd9b29e43e202315d5e99399 > b99622469c4a. > Alex Williamson suggested that KVM driver needs an equivalent version > of the fix. Can anybody suggest hints on where this fix should be > made? > 2 - Any insights on what changes in linux kernel between 3.5 to 3.6 > on > the guest that avoids this problem? >=20 > Any helps/input greatly appreciated. Thanks! Legacy KVM device assignment is deprecated, so I'd suggest your efforts are better spent reporting and trying to fix any performance difference you're seeing between pci-assign and vfio-pci. =C2=A0I have a really ha= rd time believing there's anywhere close to a 40-50% difference. =C2=A0Wha= t's the device? =C2=A0What's the workload? =C2=A0At some point you're likel= y to find that pci-assign is no longer even present. =C2=A0Thanks, Alex