From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Ahern Subject: Re: performance of virtual functions compared to virtio Date: Mon, 25 Apr 2011 11:46:03 -0600 Message-ID: <4DB5B35B.5070703@gmail.com> References: <4DAF8EF0.8010203@gmail.com> <1303353349.3110.181.camel@x201> <4DAFE5BE.1070506@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Alex Williamson , KVM mailing list To: Avi Kivity Return-path: Received: from mail-pw0-f46.google.com ([209.85.160.46]:41281 "EHLO mail-pw0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758695Ab1DYRqI (ORCPT ); Mon, 25 Apr 2011 13:46:08 -0400 Received: by pwi15 with SMTP id 15so1310447pwi.19 for ; Mon, 25 Apr 2011 10:46:08 -0700 (PDT) In-Reply-To: <4DAFE5BE.1070506@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 04/21/11 02:07, Avi Kivity wrote: > On 04/21/2011 05:35 AM, Alex Williamson wrote: >> Device assignment via a VF provides the lowest latency and most >> bandwidth for *getting data off the host system*, though virtio/vhost is >> getting better. If all you care about is VM-VM on the same host or >> VM-host, then virtio is only limited by memory bandwidth/latency and >> host processor cycles. Your processor has 25GB/s of memory bandwidth. >> On the other hand, the VF has to send data all the way out to the wire >> and all the way back up through the NIC to get to the other VM/host. >> You're using a 1Gb/s NIC. Your results actually seem to indicate you're >> getting better than wire rate, so maybe you're only passing through an >> internal switch on the NIC, in any case, VFs are not optimal for >> communication within the same physical system. They are optimal for off >> host communication. Thanks, >> > > Note I think in both cases we can make significant improvements: > - for VFs, steer device interrupts to the cpus which run the vcpus that > will receive the interrupts eventually (ISTR some work about this, but > not sure) I don't understand your point here. I thought interrupts for the VF were only delivered to the guest, not the host. David > - for virtio, use a DMA engine to copy data (I think there exists code > in upstream which does this, but has this been enabled/tuned?) >