* [Qemu-devel] qemu-kvm guest network stalls
@ 2010-06-26 20:36 William King
2010-06-28 8:08 ` [Qemu-devel] " William King
0 siblings, 1 reply; 4+ messages in thread
From: William King @ 2010-06-26 20:36 UTC (permalink / raw)
To: qemu-devel
I have an ubuntu lucid host and an ubuntu lucid guest. Under heavy
network load the guest network stalls and is only able to come back
after running "sudo /etc/init.d/networking restart" on the guest.
Here are some (hopefully) helpful details.
------------------------------------Host---------------------------
# uname -a
Linux virtserv 2.6.32-22-server #36-Ubuntu SMP Thu Jun 3 20:38:33 UTC
2010 x86_64 GNU/Linux
# ifconfig -v
br0 Link encap:Ethernet HWaddr 00:e0:4d:20:1f:86
inet addr:192.168.100.145 Bcast:192.168.100.255
Mask:255.255.255.0
inet6 addr: fe80::2e0:4dff:fe20:1f86/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1254011 errors:0 dropped:0 overruns:0 frame:0
TX packets:1070096 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:439139185 (439.1 MB) TX bytes:836663432 (836.6 MB)
eth0 Link encap:Ethernet HWaddr 00:e0:4d:20:1f:86
inet6 addr: fe80::2e0:4dff:fe20:1f86/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:124712767 errors:0 dropped:0 overruns:0 frame:0
TX packets:91086029 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:162562714983 (162.5 GB) TX bytes:36005498601 (36.0 GB)
Interrupt:29 Base address:0x8000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:83577 errors:0 dropped:0 overruns:0 frame:0
TX packets:83577 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:15722576 (15.7 MB) TX bytes:15722576 (15.7 MB)
virbr0 Link encap:Ethernet HWaddr 5a:f3:dc:99:84:ee
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
inet6 addr: fe80::58f3:dcff:fe99:84ee/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:468 (468.0 B)
vnet0 Link encap:Ethernet HWaddr ca:8b:d8:66:e6:f5
inet6 addr: fe80::c88b:d8ff:fe66:e6f5/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:81138964 errors:0 dropped:0 overruns:0 frame:0
TX packets:111911797 errors:0 dropped:0 overruns:100 carrier:0
collisions:0 txqueuelen:500
RX bytes:31177137669 (31.1 GB) TX bytes:147190153358 (147.1 GB)
# dpkg -l |grep qemu
ii kvm
1:84+dfsg-0ubuntu16+0.12.4+noroms+0ubuntu3~ppa1 dummy transitional
pacakge from kvm to qemu-kvm
ii qemu-common
0.12.4+noroms-0ubuntu3~ppa1 qemu common
functionality (bios, documentation, etc)
ii qemu-kvm
0.12.4+noroms-0ubuntu3~ppa1 Full virtualization on
i386 and amd64 hardware
I installed the 0.12.4 qemu-kvm package that was built for lucid and
uploaded here:
https://edge.launchpad.net/~ubuntu-virt/+archive/ppa/+packages
# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_fd 0
bridge_maxwait 0
------------------------------Guest-------------------------------
uname -a
Linux torrents 2.6.32-22-server #36-Ubuntu SMP Thu Jun 3 20:38:33 UTC
2010 x86_64 GNU/Linux
# ifconfig -v
eth0 Link encap:Ethernet HWaddr 52:54:00:1e:6e:b4
inet addr:192.168.100.203 Bcast:192.168.100.255
Mask:255.255.255.0
inet6 addr: fe80::5054:ff:fe1e:6eb4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:39410626 errors:0 dropped:0 overruns:0 frame:0
TX packets:28376412 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:52260862769 (52.2 GB) TX bytes:10005355703 (10.0 GB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:14096 errors:0 dropped:0 overruns:0 frame:0
TX packets:14096 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:67362075 (67.3 MB) TX bytes:67362075 (67.3 MB)
# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
Notes:
The TX and RX amounts are different from guest to host because I have
restarted the guest 'recently'.
One interesting thing I found was that the command 'arp -an' returns
nothing on the guest while it is stalled.
On 0.12.3 I was getting network stalls every 1-3 minutes with out fail.
After upgrading to 0.12.4 I still get the stalls mostly every 1-5
minutes but some times it lasts over an hour between stalls.
-William King
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Qemu-devel] Re: qemu-kvm guest network stalls
2010-06-26 20:36 [Qemu-devel] qemu-kvm guest network stalls William King
@ 2010-06-28 8:08 ` William King
2010-06-28 8:28 ` Stefan Hajnoczi
0 siblings, 1 reply; 4+ messages in thread
From: William King @ 2010-06-28 8:08 UTC (permalink / raw)
To: qemu-devel
Is this the wrong list to get help tracking down why the network stalls
under heavy load?
I have been on irc talking with multiple people, some believe the
problem is located in "xming> so it's 100% linux-bridge/ebtables", but I
have no way to confirm this.
I have been able to reproduce the network stall with two completely
different pieces of hardware. One is an AMD dual core, the other an
Intel i7, each have different network hardware, etc. Same issue.
On 06/26/2010 01:36 PM, William King wrote:
> I have an ubuntu lucid host and an ubuntu lucid guest. Under heavy
> network load the guest network stalls and is only able to come back
> after running "sudo /etc/init.d/networking restart" on the guest.
>
> Here are some (hopefully) helpful details.
> ------------------------------------Host---------------------------
>
> # uname -a
> Linux virtserv 2.6.32-22-server #36-Ubuntu SMP Thu Jun 3 20:38:33 UTC
> 2010 x86_64 GNU/Linux
>
> # ifconfig -v
> br0 Link encap:Ethernet HWaddr 00:e0:4d:20:1f:86
> inet addr:192.168.100.145 Bcast:192.168.100.255
> Mask:255.255.255.0
> inet6 addr: fe80::2e0:4dff:fe20:1f86/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1254011 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1070096 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:439139185 (439.1 MB) TX bytes:836663432 (836.6 MB)
>
> eth0 Link encap:Ethernet HWaddr 00:e0:4d:20:1f:86
> inet6 addr: fe80::2e0:4dff:fe20:1f86/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:124712767 errors:0 dropped:0 overruns:0 frame:0
> TX packets:91086029 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:162562714983 (162.5 GB) TX bytes:36005498601 (36.0 GB)
> Interrupt:29 Base address:0x8000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:83577 errors:0 dropped:0 overruns:0 frame:0
> TX packets:83577 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:15722576 (15.7 MB) TX bytes:15722576 (15.7 MB)
>
> virbr0 Link encap:Ethernet HWaddr 5a:f3:dc:99:84:ee
> inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
> inet6 addr: fe80::58f3:dcff:fe99:84ee/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:468 (468.0 B)
>
> vnet0 Link encap:Ethernet HWaddr ca:8b:d8:66:e6:f5
> inet6 addr: fe80::c88b:d8ff:fe66:e6f5/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:81138964 errors:0 dropped:0 overruns:0 frame:0
> TX packets:111911797 errors:0 dropped:0 overruns:100 carrier:0
> collisions:0 txqueuelen:500
> RX bytes:31177137669 (31.1 GB) TX bytes:147190153358 (147.1 GB)
>
> # dpkg -l |grep qemu
> ii kvm
> 1:84+dfsg-0ubuntu16+0.12.4+noroms+0ubuntu3~ppa1 dummy transitional
> pacakge from kvm to qemu-kvm
> ii qemu-common
> 0.12.4+noroms-0ubuntu3~ppa1 qemu common
> functionality (bios, documentation, etc)
> ii qemu-kvm
> 0.12.4+noroms-0ubuntu3~ppa1 Full virtualization on
> i386 and amd64 hardware
>
>
> I installed the 0.12.4 qemu-kvm package that was built for lucid and
> uploaded here:
> https://edge.launchpad.net/~ubuntu-virt/+archive/ppa/+packages
>
> # cat /etc/network/interfaces
> # This file describes the network interfaces available on your system
> # and how to activate them. For more information, see interfaces(5).
>
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto eth0
> iface eth0 inet dhcp
>
> auto br0
> iface br0 inet dhcp
> bridge_ports eth0
> bridge_stp off
> bridge_fd 0
> bridge_maxwait 0
>
>
>
> ------------------------------Guest-------------------------------
> uname -a
> Linux torrents 2.6.32-22-server #36-Ubuntu SMP Thu Jun 3 20:38:33 UTC
> 2010 x86_64 GNU/Linux
>
> # ifconfig -v
> eth0 Link encap:Ethernet HWaddr 52:54:00:1e:6e:b4
> inet addr:192.168.100.203 Bcast:192.168.100.255
> Mask:255.255.255.0
> inet6 addr: fe80::5054:ff:fe1e:6eb4/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:39410626 errors:0 dropped:0 overruns:0 frame:0
> TX packets:28376412 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:52260862769 (52.2 GB) TX bytes:10005355703 (10.0 GB)
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:14096 errors:0 dropped:0 overruns:0 frame:0
> TX packets:14096 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:67362075 (67.3 MB) TX bytes:67362075 (67.3 MB)
>
> # cat /etc/network/interfaces
> # This file describes the network interfaces available on your system
> # and how to activate them. For more information, see interfaces(5).
>
> # The loopback network interface
> auto lo
> iface lo inet loopback
>
> # The primary network interface
> auto eth0
> iface eth0 inet dhcp
>
>
>
>
>
> Notes:
> The TX and RX amounts are different from guest to host because I have
> restarted the guest 'recently'.
> One interesting thing I found was that the command 'arp -an' returns
> nothing on the guest while it is stalled.
>
> On 0.12.3 I was getting network stalls every 1-3 minutes with out fail.
> After upgrading to 0.12.4 I still get the stalls mostly every 1-5
> minutes but some times it lasts over an hour between stalls.
>
> -William King
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] Re: qemu-kvm guest network stalls
2010-06-28 8:08 ` [Qemu-devel] " William King
@ 2010-06-28 8:28 ` Stefan Hajnoczi
2010-06-28 8:45 ` William King
0 siblings, 1 reply; 4+ messages in thread
From: Stefan Hajnoczi @ 2010-06-28 8:28 UTC (permalink / raw)
To: William King; +Cc: qemu-devel
On Mon, Jun 28, 2010 at 9:08 AM, William King <quentusrex@gmail.com> wrote:
> Is this the wrong list to get help tracking down why the network stalls
> under heavy load?
This list is the right one. There is at least one known network stall bug:
http://sourceforge.net/tracker/?func=detail&atid=893831&aid=2506814&group_id=180599
http://www.mail-archive.com/kvm@vger.kernel.org/msg06774.html
I sounds like you have the same issue?
Stefan
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] Re: qemu-kvm guest network stalls
2010-06-28 8:28 ` Stefan Hajnoczi
@ 2010-06-28 8:45 ` William King
0 siblings, 0 replies; 4+ messages in thread
From: William King @ 2010-06-28 8:45 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: qemu-devel
Thanks for the reply Stefan.
That looks to be exactly the same bug I'm dealing with here. The one
difference is that I can reproduce it easily on test hardware. So I have
two separate boxes that can go down(and run test code) without
interfering with anything in production.
My test setup has a local torrent server, and 5 other machines on the
network seeding a very large(10GB) random data file that I created. When
the guest tries to download this file from the other servers this
aggravates the issue. To the point that I could cause it to fail within
minutes, repeatedly. I created this test after someone on irc said that
the issue could have something to do with udp traffic rather than tcp
traffic.
I am running latest ubuntu 10.04 on the hosts and the guests, and I can
wipe(and reinstall) the hosts quickly and easily.
On 06/28/2010 01:28 AM, Stefan Hajnoczi wrote:
> On Mon, Jun 28, 2010 at 9:08 AM, William King <quentusrex@gmail.com> wrote:
>> Is this the wrong list to get help tracking down why the network stalls
>> under heavy load?
>
> This list is the right one. There is at least one known network stall bug:
>
> http://sourceforge.net/tracker/?func=detail&atid=893831&aid=2506814&group_id=180599
> http://www.mail-archive.com/kvm@vger.kernel.org/msg06774.html
>
> I sounds like you have the same issue?
>
> Stefan
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2010-06-28 8:45 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-06-26 20:36 [Qemu-devel] qemu-kvm guest network stalls William King
2010-06-28 8:08 ` [Qemu-devel] " William King
2010-06-28 8:28 ` Stefan Hajnoczi
2010-06-28 8:45 ` William King
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).