From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Ahern Subject: Re: performance of virtual functions compared to virtio Date: Mon, 25 Apr 2011 14:40:33 -0600 Message-ID: <4DB5DC41.7060308@gmail.com> References: <4DAF8EF0.8010203@gmail.com> <1303353349.3110.181.camel@x201> <4DB5B1C4.4000602@gmail.com> <1303755193.3431.50.camel@x201> <4DB5C65C.20306@gmail.com> <1303759773.3431.64.camel@x201> <4DB5D053.1070401@gmail.com> <1303763254.3431.79.camel@x201> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: KVM mailing list To: Alex Williamson Return-path: Received: from mail-pv0-f174.google.com ([74.125.83.174]:64337 "EHLO mail-pv0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932340Ab1DYUkg (ORCPT ); Mon, 25 Apr 2011 16:40:36 -0400 Received: by pvg12 with SMTP id 12so1390534pvg.19 for ; Mon, 25 Apr 2011 13:40:36 -0700 (PDT) In-Reply-To: <1303763254.3431.79.camel@x201> Sender: kvm-owner@vger.kernel.org List-ID: On 04/25/11 14:27, Alex Williamson wrote: > On Mon, 2011-04-25 at 13:49 -0600, David Ahern wrote: >> >> On 04/25/11 13:29, Alex Williamson wrote: >>> So we're effectively getting host-host latency/throughput for the VF, >>> it's just that in the 82576 implementation of SR-IOV, the VF takes a >>> latency hit that puts it pretty close to virtio. Unfortunate. I think >> >> For host-to-VM using VFs is worse than virtio which is counterintuitive. > > On the same host, just think about the data path of one versus the > other. On the guest side, there's virtio vs a physical NIC. virtio is > designed to be virtualization friendly, so hopefully has less context > switches in setting up and processing transactions. Once the packet > leaves the assigned physical NIC, it has to come back up the entire host > I/O stack, while the virtio device is connected to an internal bridge > and bypasses all but the upper level network routing. I get the virtio path, but you lost me on the physical NIC. I thought the point of VFs is to bypass the host from having to touch the packet, so the processing path with a VM using a VF would be the same as a non-VM. David > >>> you'll find that passing the PF to the guests should be pretty close to >>> that 185us latency. I would assume (hope) the higher end NICs reduce >> >> About that 185usec: do you know where the bottleneck is? It seems as if >> the packet is held in some queue waiting for an event/timeout before it >> is transmitted. > > I don't know specifically, I don't do much network performance tuning. > Interrupt coalescing could be a factor, along with various offload > settings, and of course latency of the physical NIC hardware and > interconnects. Thanks, > > Alex >