From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Ahern Subject: Re: performance of virtual functions compared to virtio Date: Mon, 25 Apr 2011 15:14:28 -0600 Message-ID: <4DB5E434.1040708@gmail.com> References: <4DAF8EF0.8010203@gmail.com> <1303353349.3110.181.camel@x201> <4DB5B1C4.4000602@gmail.com> <1303755193.3431.50.camel@x201> <4DB5C65C.20306@gmail.com> <1303759773.3431.64.camel@x201> <4DB5D053.1070401@gmail.com> <1303763254.3431.79.camel@x201> <4DB5DC41.7060308@gmail.com> <1303765341.3431.94.camel@x201> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: KVM mailing list To: Alex Williamson Return-path: Received: from mail-pv0-f174.google.com ([74.125.83.174]:37491 "EHLO mail-pv0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750931Ab1DYVOb (ORCPT ); Mon, 25 Apr 2011 17:14:31 -0400 Received: by pvg12 with SMTP id 12so10081pvg.19 for ; Mon, 25 Apr 2011 14:14:31 -0700 (PDT) In-Reply-To: <1303765341.3431.94.camel@x201> Sender: kvm-owner@vger.kernel.org List-ID: On 04/25/11 15:02, Alex Williamson wrote: > On Mon, 2011-04-25 at 14:40 -0600, David Ahern wrote: >> >> On 04/25/11 14:27, Alex Williamson wrote: >>> On Mon, 2011-04-25 at 13:49 -0600, David Ahern wrote: >>>> >>>> On 04/25/11 13:29, Alex Williamson wrote: >>>>> So we're effectively getting host-host latency/throughput for the VF, >>>>> it's just that in the 82576 implementation of SR-IOV, the VF takes a >>>>> latency hit that puts it pretty close to virtio. Unfortunate. I think >>>> >>>> For host-to-VM using VFs is worse than virtio which is counterintuitive. >>> >>> On the same host, just think about the data path of one versus the >>> other. On the guest side, there's virtio vs a physical NIC. virtio is >>> designed to be virtualization friendly, so hopefully has less context >>> switches in setting up and processing transactions. Once the packet >>> leaves the assigned physical NIC, it has to come back up the entire host >>> I/O stack, while the virtio device is connected to an internal bridge >>> and bypasses all but the upper level network routing. >> >> I get the virtio path, but you lost me on the physical NIC. I thought >> the point of VFs is to bypass the host from having to touch the packet, >> so the processing path with a VM using a VF would be the same as a non-VM. > > In the VF case, the host is only involved in processing the packet on > it's end of the connection, but the packet still has to go all the way > out to the physical device and all the way back. Handled on one end by > the VM and the other end by the host. > > An analogy might be sending a letter to an office coworker in a > neighboring cube. You could just pass the letter over the wall (virtio) > or you could go put it in the mailbox, signal the mail carrier, who > comes and moves it to your neighbor's mailbox, who then gets signaled > that they have a letter (device assignment). > > Since the networks stacks are completely separate from one another, > there's very little difference in data path whether you're talking to > the host, a remote system, or a remote VM, which is reflected in your > performance data. Hope that helps, Got you. I was thinking host-VM as VM on separate host; I didn't make that clear. Thanks for clarifying - I like the letter example. David > > Alex >