Re: Network throughput limits for local VM <-> VM communication

[Mark McLoughlin <markmc@redhat.com>]

On Tue, 2009-06-09 at 11:06 +0000, Fischer, Anna wrote:

> I am testing network throughput between two guests residing on the
> same physical machine. I use a bridge to pass packets between those
> guests and the virtio NIC model. I am wondering why the throughput
> only goes up to about 970Mbps. Should we not be able to achieve much
> higher throughput if the packets do not actually go out on the
> physical wire? What are the limitations for throughput performance
> under KVM/virtio? I can see that by default the interfaces (the tap
> devices) have TX queue length set to 500, and I wonder if increasing
> this would make any difference? Also, are there other things I would
> need to consider to achieve higher throughput numbers for local guest
> <-> guest communication? The CPU is not maxed out at all, and shows as
> being idle for most of the time while the throughput does not increase
> any more.
> 
> I run KVM under standard Fedora Core 10 with a Linux kernel 2.6.27.

The first thing to check is that GSO is enabled - you can check with
"ethtool -k eth0" in the guests.

Are you starting qemu from the command line or e.g. using libvirt? The
libvirt version in F-10 didn't know how to enable IFF_VNET_HDR on the
tapfd before passing it to qemu.

Really, I'd suggest updating to F-11 before digging further - you'll
have qemu-kvm-0.10.5, linux-2.6.29.4 and libvirt-0.6.2.

Cheers,
Mark.