From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark McLoughlin Subject: Re: Network throughput limits for local VM <-> VM communication Date: Tue, 09 Jun 2009 14:39:40 +0100 Message-ID: <1244554780.11883.21.camel@blaa> References: <0199E0D51A61344794750DC57738F58E67D2398F9E@GVW1118EXC.americas.hpqcorp.net> Reply-To: Mark McLoughlin Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: "kvm@vger.kernel.org" To: "Fischer, Anna" Return-path: Received: from mx2.redhat.com ([66.187.237.31]:53158 "EHLO mx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751536AbZFINjl (ORCPT ); Tue, 9 Jun 2009 09:39:41 -0400 In-Reply-To: <0199E0D51A61344794750DC57738F58E67D2398F9E@GVW1118EXC.americas.hpqcorp.net> Sender: kvm-owner@vger.kernel.org List-ID: On Tue, 2009-06-09 at 11:06 +0000, Fischer, Anna wrote: > I am testing network throughput between two guests residing on the > same physical machine. I use a bridge to pass packets between those > guests and the virtio NIC model. I am wondering why the throughput > only goes up to about 970Mbps. Should we not be able to achieve much > higher throughput if the packets do not actually go out on the > physical wire? What are the limitations for throughput performance > under KVM/virtio? I can see that by default the interfaces (the tap > devices) have TX queue length set to 500, and I wonder if increasing > this would make any difference? Also, are there other things I would > need to consider to achieve higher throughput numbers for local guest > <-> guest communication? The CPU is not maxed out at all, and shows as > being idle for most of the time while the throughput does not increase > any more. > > I run KVM under standard Fedora Core 10 with a Linux kernel 2.6.27. The first thing to check is that GSO is enabled - you can check with "ethtool -k eth0" in the guests. Are you starting qemu from the command line or e.g. using libvirt? The libvirt version in F-10 didn't know how to enable IFF_VNET_HDR on the tapfd before passing it to qemu. Really, I'd suggest updating to F-11 before digging further - you'll have qemu-kvm-0.10.5, linux-2.6.29.4 and libvirt-0.6.2. Cheers, Mark.