From mboxrd@z Thu Jan 1 00:00:00 1970 From: Daniel Bareiro Subject: Re: Bandwith limitation with KVM VMs Date: Tue, 4 Aug 2009 06:48:25 -0300 Message-ID: <20090804094825.GM23503@defiant.freesoftware.org> References: <20090803163205.GJ23503@defiant.freesoftware.org> <4A7715CC.7030901@gmail.com> <20090804011742.GL23503@defiant.freesoftware.org> <4A77A48A.1060606@gmail.com> Reply-To: dbareiro@gmx.net Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Qgof7w7UksPF5inF" To: KVM General Return-path: Received: from mail.gmx.net ([213.165.64.20]:56191 "HELO mail.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S932624AbZHDJsb (ORCPT ); Tue, 4 Aug 2009 05:48:31 -0400 Received: from defiant (defiant.freesoftware.org [10.1.0.65]) by hermes.freesoftware.org (Postfix) with ESMTP id C18E79D2E for ; Tue, 4 Aug 2009 06:48:21 -0300 (ART) Content-Disposition: inline In-Reply-To: <4A77A48A.1060606@gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: --Qgof7w7UksPF5inF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Gregory. On Monday, 03 August 2009 23:01:30 -0400, Gregory Haskins wrote: > >> There is no set artificial limit afaict, though there are a large > >> number of factors that can affect performance. Of course, > >> everything has an ultimate ceiling (KVM included) but I have found > >> this limit in KVM to be orders of magnitude faster than 10Mbps. > >> Properly tuned, you should easily be able to saturate a GE link at > >> line rate, or even 4Gbps-5Gpbs of a 10GE link. =20 > >> However, since you are only hitting 10Mb/s now, there is ton of > >> headroom left even on upstream KVM so you might find it to be > >> satisfactory as is, once you address the current bottleneck in your > >> setup. > >> Things to check: What linkspeed does the host see to the next hop? > >> How much bandwidth does the host see to the same end-point? What > >> is your overall topology, especially for the VM (are you using -net > >> tap, etc). What MTU are you using. Etc. =20 > > It draws attention that when executing 'cfgmaker' from MRTG server > > against the IP of the VM, it returns max speed of 1250 kBytes/s, > > that is to say 10 Mbps: > >=20 > > sparky:~# /usr/bin/cfgmaker --global 'WorkDir: /tmp' --global \=20 > > 'Options[_]: bits,growright' xxxxxxxxxxxxxx@10.1.0.42 > > [...] > > MaxBytes[10.1.0.42_2]: 1250000 > >=20 > >=20 > > But nevertheless from within of the VM I see the following thing: > >=20 > > aps2:~# ethtool eth0 > > Settings for eth0: > > Supported ports: [ TP MII ] > > Supported link modes: 10baseT/Half 10baseT/Full > > 100baseT/Half 100baseT/Full > > Supports auto-negotiation: Yes > > Advertised link modes: 10baseT/Half 10baseT/Full > > 100baseT/Half 100baseT/Full > > Advertised auto-negotiation: Yes > > Speed: 100Mb/s > > Duplex: Full > > Port: MII > > PHYAD: 32 > > Transceiver: internal > > Auto-negotiation: on > > Supports Wake-on: pumbg > > Wake-on: d > > Current message level: 0x00000007 (7) > > Link detected: yes > >=20 > >=20 > > do you think that can give some indication? =20 > Hmm.. I am not familiar with MRTG/cfgmaker, but from your ethtool output > I suspect you are not using virtio. How do you launch the guest? The VMs are booted with a syntax similar to the following one: root@ss02:~# ps ax|grep aps2|grep -v grep 28711 ? Sl 8171:06 kvm -hda /dev/vm/aps2-raiz -hdb \ /dev/vm/aps2-space -hdc /dev/vm/aps2-index -hdd /dev/vm/aps2-cache -m \ 4096 -smp 4 -net nic,vlan=3D0,macaddr=3D00:16:3E:00:00:27 -net tap \ -daemonize -vnc :5 -k es -localtime -monitor \ telnet:localhost:4005,server,nowait -serial \ telnet:localhost:4045,server,nowait \ According to I was reading, it would be necessary to use with -net the option model=3Dvirtio, so I would not be using virtio with the VMs. In order to provide a little more information, I will say to you that this VM is running on host with Ubuntu Hardy Heron server amd64 with kernel 2.6.24 and kvm-62, both installed from the Ubuntu repositories, but my idea is to upgrade in the short term to kernel 2.6.30 with KVM-88 compiled by myself basing me on a suggestion that did Avi to me by problems that I am having of memory usage [1]. The host machine has two physical interfaces with the following configuration: auto br0 iface br0 inet static address 10.1.0.47 netmask 255.255.0.0 network 10.1.0.0 broadcast 10.1.255.255 bridge_ports eth1 bridge_stp off bridge_maxwait 5 auto eth0 iface eth0 inet static address 10.3.0.47 netmask 255.255.0.0 network 10.3.0.0 broadcast 10.3.255.255 gateway 10.3.0.5 Executing ethtool from the host machine, I get the following data: root@ss02:~# ethtool eth0 Settings for eth0: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 100Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: g Link detected: yes root@ss02:~# ethtool eth1 Settings for eth1: Supported ports: [ TP ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full Advertised auto-negotiation: Yes Speed: 100Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1 Transceiver: internal Auto-negotiation: on Supports Wake-on: g Wake-on: g Link detected: yes Apparently even though the interface for bridge supports 10/100/1000, the one of the VM is 10/100. > Have you actually run something like netperf or an rsync to see what > type of bandwidth you actually get? Perhaps this is just the mrtg > tool getting confused about the actual interface capabilities. It draws attention to me that I have MRTG registering the bandwidth of several routers and for those cases it register over 10 Mbps without problem. These are the values that I obtain in a snapshot with iptraf: Iface Total IP NonIP BadIP Activity lo 120 120 0 0 3,00 kbits/sec eth0 2654 2654 0 0 56,40 kbits/sec eth1 346250 346250 0 0 18459,20 kbits/sec tap0 24 24 0 0 0,40 kbits/sec tap2 128 128 0 0 1,20 kbits/sec=20 tap3 8366 8366 0 0 685,40 kbits/sec tap1 24 24 0 0 0,40 kbits/sec tap5 19531 19531 0 0 2742,80 kbits/sec tap4 328258 328258 0 0 16127,80 kbits/sec Also, according to MRTG, at this precise moment the VM interface is not saturated in 10 Mbps. There is some command whose execution can be used to associate a tap device with a VM? Like additional note, I am observing that the sum of the activity on each tap device exceeds the measurement for eth1. This can be possible? Thanks for your reply. Regards, Daniel [1] http://thread.gmane.org/gmane.comp.emulators.kvm.devel/37704 --=20 Fingerprint: BFB3 08D6 B4D1 31B2 72B9 29CE 6696 BF1B 14E6 1D37 Powered by Debian GNU/Linux Squeeze - Linux user #188.598 --Qgof7w7UksPF5inF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: Digital signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEARECAAYFAkp4A+kACgkQZpa/GxTmHTeH4wCfeJAQfrQzrxVcOKi9tWC3UJ1T LVEAn1iLHmmuh43Z1GIzneh/Ryg9afno =FtHg -----END PGP SIGNATURE----- --Qgof7w7UksPF5inF--