From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: performance of virtual functions compared to virtio Date: Mon, 25 Apr 2011 14:27:34 -0600 Message-ID: <1303763254.3431.79.camel@x201> References: <4DAF8EF0.8010203@gmail.com> <1303353349.3110.181.camel@x201> <4DB5B1C4.4000602@gmail.com> <1303755193.3431.50.camel@x201> <4DB5C65C.20306@gmail.com> <1303759773.3431.64.camel@x201> <4DB5D053.1070401@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: KVM mailing list To: David Ahern Return-path: Received: from mx1.redhat.com ([209.132.183.28]:6840 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932542Ab1DYU1h (ORCPT ); Mon, 25 Apr 2011 16:27:37 -0400 In-Reply-To: <4DB5D053.1070401@gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On Mon, 2011-04-25 at 13:49 -0600, David Ahern wrote: > > On 04/25/11 13:29, Alex Williamson wrote: > > So we're effectively getting host-host latency/throughput for the VF, > > it's just that in the 82576 implementation of SR-IOV, the VF takes a > > latency hit that puts it pretty close to virtio. Unfortunate. I think > > For host-to-VM using VFs is worse than virtio which is counterintuitive. On the same host, just think about the data path of one versus the other. On the guest side, there's virtio vs a physical NIC. virtio is designed to be virtualization friendly, so hopefully has less context switches in setting up and processing transactions. Once the packet leaves the assigned physical NIC, it has to come back up the entire host I/O stack, while the virtio device is connected to an internal bridge and bypasses all but the upper level network routing. > > you'll find that passing the PF to the guests should be pretty close to > > that 185us latency. I would assume (hope) the higher end NICs reduce > > About that 185usec: do you know where the bottleneck is? It seems as if > the packet is held in some queue waiting for an event/timeout before it > is transmitted. I don't know specifically, I don't do much network performance tuning. Interrupt coalescing could be a factor, along with various offload settings, and of course latency of the physical NIC hardware and interconnects. Thanks, Alex