kvm-36 losing (tap-based) network connectivity mid-session

kvm-36 losing (tap-based) network connectivity mid-session

[Charles Duffy]

kvm-36 is repeatedly timing out partway through a long network transfor 
and failing to resume. Both host and guest are 64-bit. This has happened 
with a number of prior releases as well; it does not appear to be a 
recently-introduced issue.

My command line looks like the following:

$ qemu-system-x86_64 -kernel kernel-x86_64-2.6.20-gentoo-r8 -initrd 
initramfs.igz -hda hda -hdb hdb -append "panic=30 console=ttyS0 
action=install" -nographic -net nic -net 
'tap,ifname=tap0,script=./tap0-start'

/dev/kvm is quite certainly being used, as KVM immediately segfaults on 
startup without it (or in the presence of the -no-kvm option).

dmesg on the host shows a great number of "rtc: lost some interrupts at 
1024Hz" messages.

The end of a typical session looks much like the following:

====
79000K .......... .......... .......... .......... ..........  4% 5.43M 
   4:33
79050K .......... .......... .......... .......... ..........  4% 6.10M 
   4:03
79100K .......... .......... .......... .......... ..........  4% 8.14M 
   3:02
79150K ......                                                  4% 5.96M 
   4:09

02:45:23 (2.06 MB/s) - Read error at byte 81055847/1639127012 
(Connection timed out). Retrying.

--02:45:23-- 
http://10.0.0.30/gentoo-installer/deployed_server-stage4-amd64-2007.0.tar.bz2
   (try: 2) => `/mnt/gentoo/deployed_server-stage4-amd64-2007.0.tar.bz2'
Connecting to 10.0.0.30:80... Hangcheck: hangcheck value past margin!
failed: Connection timed out.
Retrying.

--02:45:53-- 
http://10.0.0.30/gentoo-installer/deployed_server-stage4-amd64-2007.0.tar.bz2
   (try: 3) => `/mnt/gentoo/deployed_server-stage4-amd64-2007.0.tar.bz2'
Connecting to 10.0.0.30:80... failed: Connection timed out.
Retrying.

--02:46:23-- 
http://10.0.0.30/gentoo-installer/deployed_server-stage4-amd64-2007.0.tar.bz2
   (try: 4) => `/mnt/gentoo/deployed_server-stage4-amd64-2007.0.tar.bz2'
====

Network access works, and then suddenly doesn't. I am more reliably able 
to reproduce this issue using -net tap than -net user.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: kvm-36 losing (tap-based) network connectivitymid-session

[Dor Laor]

>kvm-36 is repeatedly timing out partway through a long network transfor
>and failing to resume. Both host and guest are 64-bit. This has
happened
>with a number of prior releases as well; it does not appear to be a
>recently-introduced issue.
>
>My command line looks like the following:
>
>$ qemu-system-x86_64 -kernel kernel-x86_64-2.6.20-gentoo-r8 -initrd
>initramfs.igz -hda hda -hdb hdb -append "panic=30 console=ttyS0
>action=install" -nographic -net nic -net
>'tap,ifname=tap0,script=./tap0-start'
>
>/dev/kvm is quite certainly being used, as KVM immediately segfaults on
>startup without it (or in the presence of the -no-kvm option).
>
>dmesg on the host shows a great number of "rtc: lost some interrupts at
>1024Hz" messages.
>
>The end of a typical session looks much like the following:
>
>====
>79000K .......... .......... .......... .......... ..........  4% 5.43M
>   4:33
>79050K .......... .......... .......... .......... ..........  4% 6.10M
>   4:03
>79100K .......... .......... .......... .......... ..........  4% 8.14M
>   3:02
>79150K ......                                                  4% 5.96M
>   4:09
>
>02:45:23 (2.06 MB/s) - Read error at byte 81055847/1639127012
>(Connection timed out). Retrying.
>
>--02:45:23--
>http://10.0.0.30/gentoo-installer/deployed_server-stage4-amd64-
>2007.0.tar.bz2
>   (try: 2) =>
`/mnt/gentoo/deployed_server-stage4-amd64-2007.0.tar.bz2'
>Connecting to 10.0.0.30:80... Hangcheck: hangcheck value past margin!
>failed: Connection timed out.
>Retrying.
>
>--02:45:53--
>http://10.0.0.30/gentoo-installer/deployed_server-stage4-amd64-
>2007.0.tar.bz2
>   (try: 3) =>
`/mnt/gentoo/deployed_server-stage4-amd64-2007.0.tar.bz2'
>Connecting to 10.0.0.30:80... failed: Connection timed out.
>Retrying.
>
>--02:46:23--
>http://10.0.0.30/gentoo-installer/deployed_server-stage4-amd64-
>2007.0.tar.bz2
>   (try: 4) =>
`/mnt/gentoo/deployed_server-stage4-amd64-2007.0.tar.bz2'
>====
>
>Network access works, and then suddenly doesn't. I am more reliably
able
>to reproduce this issue using -net tap than -net user.
>


Can you try it without rtc (either use -no-rtc, I'm not sure we have
this flag or
just rm /dev/rtc before you run the vm).
What's your host HZ configuration?
If the above doesn't work, can you repeat the test with -no-kvm?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: kvm-36 losing (tap-based) network connectivitymid-session

[Charles Duffy]

Dor Laor wrote:
> Can you try it without rtc (either use -no-rtc, I'm not sure we have
> this flag or just rm /dev/rtc before you run the vm).

Tried it (and -tdf) without effect.

> What's your host HZ configuration?

CONFIG_HZ=250

> If the above doesn't work, can you repeat the test with -no-kvm?

-no-kvm segfaults immediately on startup.

===
Starting program: /usr/bin/kvm -no-kvm -kernel 
kernel-x86_64-2.6.20-gentoo-r8 -hda grubdisk.img
[Thread debugging using libthread_db enabled]
[New Thread 47987424286272 (LWP 13655)]
Could not configure '/dev/rtc' to have a 1024 Hz timer. This is not a fatal
error, but for better emulation accuracy either use a 2.6 host Linux 
kernel or
type 'echo 1024 > /proc/sys/dev/rtc/max-user-freq' as root.

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 47987424286272 (LWP 13655)]
0x0000000000000000 in ?? ()
(gdb) bt

#0  0x0000000000000000 in ?? ()
#1  0x0000000000481e8b in cpu_x86_exec (env1=<value optimized out>) at 
/usr/src/debug/app-emulation/kvm-36/kvm-36/qemu/cpu-exec.c:750
#2  0x0000000000409a6d in main_loop () at 
/usr/src/debug/app-emulation/kvm-36/kvm-36/qemu/vl.c:6311
#3  0x000000000040de7a in main (argc=0, argv=0x7fffba1c59c8) at 
/usr/src/debug/app-emulation/kvm-36/kvm-36/qemu/vl.c:7774
#4  0x00002ba4f12ea5e6 in __libc_start_main () from /usr/local/lib/libc.so.6
#5  0x0000000000405689 in _start ()
===

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: kvm-36 losing (tap-based) network connectivitymid-session

[Avi Kivity]

Charles Duffy wrote:
> Dor Laor wrote:
>   
>> Can you try it without rtc (either use -no-rtc, I'm not sure we have
>> this flag or just rm /dev/rtc before you run the vm).
>>     
>
> Tried it (and -tdf) without effect.
>
>   
>> What's your host HZ configuration?
>>     
>
> CONFIG_HZ=250
>
>   
>

Can you check with the other emulated nic? (-nic model=...)


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: kvm-36 losing (tap-based) network connectivitymid-session

[Charles Duffy]

Avi Kivity wrote:
> Can you check with the other emulated nic? (-nic model=...)
Happens with both ne2k_pci and rtl8139; I haven't tried with others.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[RESOLVED] kvm-36 losing (tap-based) network connectivity mid-session

[Charles Duffy]

As of kvm-43, this is no longer happening. I haven't tested intermediate 
versions to locate the exact place where the issue ceased.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/