From mboxrd@z Thu Jan 1 00:00:00 1970 From: Avi Kivity Subject: Re: strange guest slowness after some time Date: Wed, 18 Mar 2009 08:29:36 +0200 Message-ID: <49C094D0.2070905@redhat.com> References: <49B29705.6000904@wpkg.org> <49B4E7A6.3090601@redhat.com> <49B4EDC9.9020504@wpkg.org> <49B4EE8F.7050001@redhat.com> <49BD007C.9030809@wpkg.org> <49BF7FB6.9080503@wpkg.org> <49BF8686.5050707@redhat.com> <49BF88BE.7040805@wpkg.org> <49BFC276.1080600@gmx.net> <49BFC743.20000@redhat.com> <49BFCC04.9080402@wpkg.org> <49BFD84F.8050509@gmx.net> <49BFD999.503@redhat.com> <49BFE146.2090308@wpkg.org> <49BFF20D.6010303@wpkg.org> <49BFF45A.5070307@gmx.net> <49BFF8D2.5080000@wpkg.org> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Felix Leimbach , kvm@vger.kernel.org, Rusty Russell , Anthony Liguori To: Tomasz Chmielewski Return-path: Received: from mx1.redhat.com ([66.187.233.31]:60693 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754156AbZCRG3F (ORCPT ); Wed, 18 Mar 2009 02:29:05 -0400 In-Reply-To: <49BFF8D2.5080000@wpkg.org> Sender: kvm-owner@vger.kernel.org List-ID: Tomasz Chmielewski wrote: > Felix Leimbach schrieb: > >>> OK, another bug found. >>> >>> Set your MTU to 100. >>> >>> On two hosts, do: >>> >>> HOST1_MTU1500# dd if=/dev/zero | ssh manager@HOST2 dd of=/dev/null >>> HOST2_MTU100# dd if=/dev/zero | ssh manager@HOST1 dd of=/dev/null >>> >>> HOST2 with MTU 100 will crash after 10-15 minutes (with packet count >>> still not overflown). >>> >> Intersting. What are the packet counter at crash time (roughly)? >> >> My - currently running - test is: >> >> Guest 1 (Linux): >> MTU 150 >> # cat /dev/zero | nc 7777 >> >> Guest 2 (Windows 2003 Server): >> MTU: 1500 >> # nc -l -p 7777 > NUL >> >> My packet are currently at 63 million without a problem - yet. > > I have it running with MTU 1500. And one of the guests (the one which > was crashing with MTU=100) froze. > > On a VNC console I can see: > > virtio_net virtio0: id 64 is not a head! > BUG: soft lockup - CPU#0 stuck for 61s! [ssh:2265] > > And "soft lockup" is being printed periodically. VNC and serial > console do not react to any key press. Guest do not react on ACPI > events (shutdown). > kvm/qemu process is using 100% CPU. > > See this screenshot: > > http://www1.wpkg.org/lockup.png > > > Guest that locks up is running Debian Lenny with 2.6.26 kernel. > Guest that does not lock up runs Mandriva 2009.0 with 2.6.27.x kernel. > (data being transferred both side to/from each of these hosts). Copying the virtio folks... something is wrong. You can obtain a stack trace of the locked up guest by doing (qemu) gdbserver 1234 $ gdb /path/to/guest/vmlinux (gdb) target remote localhost:1234 (gdb) backtrace I don't know host you obtain the guest vmlinux on debian; on Fedora it is contained in kernel-debuginfo. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.