From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: Performance test result between virtio_pci MSI-X disable and enable Date: Wed, 1 Dec 2010 16:03:58 +0200 Message-ID: <20101201140357.GD12912@redhat.com> References: <4CEB6FCC.7000407@redhat.com> <201011301724.16896.sheng.yang@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: "Yang, Sheng" , Avi Kivity , Gleb Natapov , "aliguori@us.ibm.com" , "rusty@rustcorp.com.au" , "kvm@vger.kernel.org" To: lidong chen Return-path: Received: from mx1.redhat.com ([209.132.183.28]:46679 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752745Ab0LAOEi (ORCPT ); Wed, 1 Dec 2010 09:04:38 -0500 Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Wed, Dec 01, 2010 at 04:41:38PM +0800, lidong chen wrote: > I used sr-iov, give each vm 2 vf. > after apply the patch, and i found performence is the same. >=20 > the reason is in function msix_mmio_write, mostly addr is not in mmio= range. >=20 > static int msix_mmio_write(struct kvm_io_device *this, gpa_t addr, in= t len, > const void *val) > { > struct kvm_assigned_dev_kernel *adev =3D > container_of(this, struct kvm_assigned_dev_kernel, > msix_mmio_dev); > int idx, r =3D 0; > unsigned long new_val =3D *(unsigned long *)val; >=20 > mutex_lock(&adev->kvm->lock); > if (!msix_mmio_in_range(adev, addr, len)) { > // return here. > r =3D -EOPNOTSUPP; > goto out; > } >=20 > i printk the value: > addr start end len > F004C00C F0044000 F0044030 4 >=20 > 00:06.0 Ethernet controller: Intel Corporation Unknown device 10ed (r= ev 01) > Subsystem: Intel Corporation Unknown device 000c > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Latency: 0 > Region 0: Memory at f0040000 (32-bit, non-prefetchable) [size=3D16K] > Region 3: Memory at f0044000 (32-bit, non-prefetchable) [size=3D16K] > Capabilities: [40] MSI-X: Enable+ Mask- TabSize=3D3 > Vector table: BAR=3D3 offset=3D00000000 > PBA: BAR=3D3 offset=3D00002000 >=20 > 00:07.0 Ethernet controller: Intel Corporation Unknown device 10ed (r= ev 01) > Subsystem: Intel Corporation Unknown device 000c > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Latency: 0 > Region 0: Memory at f0048000 (32-bit, non-prefetchable) [size=3D16K] > Region 3: Memory at f004c000 (32-bit, non-prefetchable) [size=3D16K] > Capabilities: [40] MSI-X: Enable+ Mask- TabSize=3D3 > Vector table: BAR=3D3 offset=3D00000000 > PBA: BAR=3D3 offset=3D00002000 >=20 >=20 >=20 > +static bool msix_mmio_in_range(struct kvm_assigned_dev_kernel *adev, > + gpa_t addr, int len) > +{ > + gpa_t start, end; > + > + BUG_ON(adev->msix_mmio_base =3D=3D 0); > + start =3D adev->msix_mmio_base; > + end =3D adev->msix_mmio_base + PCI_MSIX_ENTRY_SIZE * > + adev->msix_max_entries_nr; > + if (addr >=3D start && addr + len <=3D end) > + return true; > + > + return false; > +} Hmm, this check looks wrong to me: there's no guarantee that guest uses the first N entries in the table. E.g. it could use a single entry, but only the last one. >=20 >=20 > 2010/11/30 Yang, Sheng : > > On Tuesday 30 November 2010 17:10:11 lidong chen wrote: > >> sr-iov also meet this problem, MSIX mask waste a lot of cpu resour= ce. > >> > >> I test kvm with sriov, which the vf driver could not disable msix. > >> so the host os waste a lot of cpu. =A0cpu rate of host os is 90%. > >> > >> then I test xen with sriov, there ara also a lot of vm exits cause= d by > >> MSIX mask. > >> but the cpu rate of xen and domain0 is less than kvm. cpu rate of = xen > >> and domain0 is 60%. > >> > >> without sr-iov, the cpu rate of xen and domain0 is higher than kvm= =2E > >> > >> so i think the problem is kvm waste more cpu resource to deal with= MSIX > >> mask. and we can see how xen deal with MSIX mask. > >> > >> if this problem sloved, maybe with MSIX enabled, the performace is= better. > > > > Please refer to my posted patches for this issue. > > > > http://www.spinics.net/lists/kvm/msg44992.html > > > > -- > > regards > > Yang, Sheng > > > >> > >> 2010/11/23 Avi Kivity : > >> > On 11/23/2010 09:27 AM, lidong chen wrote: > >> >> can you tell me something about this problem. > >> >> thanks. > >> > > >> > Which problem? > >> > > >> > -- > >> > I have a truly marvellous patch that fixes the bug which this > >> > signature is too narrow to contain. > >