From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Theurer Subject: Re: performance of virtual functions compared to virtio Date: Mon, 25 Apr 2011 15:49:20 -0500 Message-ID: <1303764560.24606.49.camel@atheurer-ubuntu10> References: <4DAF8EF0.8010203@gmail.com> <1303353349.3110.181.camel@x201> <4DB5B1C4.4000602@gmail.com> <1303755193.3431.50.camel@x201> <4DB5C65C.20306@gmail.com> <1303759773.3431.64.camel@x201> <4DB5D053.1070401@gmail.com> Reply-To: habanero@linux.vnet.ibm.com Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: Alex Williamson , KVM mailing list To: David Ahern Return-path: Received: from e37.co.us.ibm.com ([32.97.110.158]:41530 "EHLO e37.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932223Ab1DYUtw (ORCPT ); Mon, 25 Apr 2011 16:49:52 -0400 Received: from d03relay05.boulder.ibm.com (d03relay05.boulder.ibm.com [9.17.195.107]) by e37.co.us.ibm.com (8.14.4/8.13.1) with ESMTP id p3PKkwqN022198 for ; Mon, 25 Apr 2011 14:46:58 -0600 Received: from d03av01.boulder.ibm.com (d03av01.boulder.ibm.com [9.17.195.167]) by d03relay05.boulder.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id p3PKnOxZ257198 for ; Mon, 25 Apr 2011 14:49:28 -0600 Received: from d03av01.boulder.ibm.com (loopback [127.0.0.1]) by d03av01.boulder.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id p3PKnN8J023859 for ; Mon, 25 Apr 2011 14:49:23 -0600 In-Reply-To: <4DB5D053.1070401@gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On Mon, 2011-04-25 at 13:49 -0600, David Ahern wrote: > > On 04/25/11 13:29, Alex Williamson wrote: > > So we're effectively getting host-host latency/throughput for the VF, > > it's just that in the 82576 implementation of SR-IOV, the VF takes a > > latency hit that puts it pretty close to virtio. Unfortunate. I think > > For host-to-VM using VFs is worse than virtio which is counterintuitive. > > > you'll find that passing the PF to the guests should be pretty close to > > that 185us latency. I would assume (hope) the higher end NICs reduce > > About that 185usec: do you know where the bottleneck is? It seems as if > the packet is held in some queue waiting for an event/timeout before it > is transmitted. you might want to check the VF driver. I know versions of the ixgbevf driver have a throttled interrupt option which will increase latency with some settings. I don't remember if the igbvf driver has the same feature. If it does, you will want to turn this option off for best latency. > > David > > > > this, but it seems to be a hardware limitation, so it's hard to predict. > > Thanks, > > > > Alex -Andrew