From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bernd Schubert Subject: Re: [regression] virtio net locks up Date: Mon, 30 Jul 2012 17:33:40 +0000 (UTC) Message-ID: References: <4F0DB647.4080307@itwm.fraunhofer.de> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: QUOTED-PRINTABLE To: kvm@vger.kernel.org Return-path: Received: from plane.gmane.org ([80.91.229.3]:36275 "EHLO plane.gmane.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755181Ab2G3Rdx (ORCPT ); Mon, 30 Jul 2012 13:33:53 -0400 Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Svtqp-0002z1-Et for kvm@vger.kernel.org; Mon, 30 Jul 2012 19:33:51 +0200 Received: from squid1.itwm.fhg.de ([131.246.191.181]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 30 Jul 2012 19:33:51 +0200 Received: from bernd.schubert by squid1.itwm.fhg.de with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 30 Jul 2012 19:33:51 +0200 Sender: kvm-owner@vger.kernel.org List-ID: Hello Stefan, Stefan Hajnoczi gmail.com> writes: >=20 > On Wed, Jan 11, 2012 at 4:18 PM, Bernd Schubert > itwm.fraunhofer.de> wrote: > > On 01/11/2012 05:04 PM, Stefan Hajnoczi wrote: > >> Try pinging the host's IP address from inside the guest. =C2=A0Run= tcpdump > >> on the guest's tap interface from the host and observe whether or = not > >> you see any packets being sent from the guest. > > sorry for my terribly late reply. As usual I got distracted by too many= other things and then returned the hardware I was running the VMs on. My new = desktop system is better suitable to run kvm and I can easily reproduce it now = with 3.5 on host and guest side. So its not fixed in recent versions yet. > > > > Seems arp requests are still going out, but then don't go in: > > > > 17:16:21.202547 ARP, Reply 192.168.123.1 is-at 00:25:90:38:09:cd (o= ui > > Unknown), length 28 > > 17:16:21.538724 ARP, Request who-has squeeze1 tell squeeze3, length= 28 > > 17:16:21.539026 ARP, Reply squeeze1 is-at 52:54:00:12:34:11 (oui Un= known), > > length 28 > > 17:16:22.200912 ARP, Request who-has 192.168.123.1 tell squeeze3, l= ength 28 >=20 > Okay, so it seems networking from the tap device and beyond is fine. >=20 > >> rmmod virtio_net inside the guest and then modprobe virtio_net aga= in. > >> See if network connectivity is restored (remember to rerun DHCP or > >> whatever, if necessary). > > > > > > Yep, that makes it work again. But probably is not the real solutio= n ;) >=20 > It's just another piece of information which helps debug this :). At > least nothing has wedged itself into an unrecoverable state. >=20 > When you said the problem happens without vhost, did you explicitly > run vhost=3Doff? Or did you just omit "vhost=3Don"? It was definitely off and I can confirm that it also locks up with vhos= t=3Don and vhost=3Doff with 3.5. >=20 > This sounds like a guest kernel/driver issue. I recommend testing > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git in > the guest to see if this has already been fixed. >=20 > If you have the -dbg RPMs installed it may be possible to insert a > probe into the virtio_net kernel module and observe receive > interrupts. This does require the right kernel CONFIG_ but you might > already have it enabled: >=20 > $ sudo perf probe --add skb_recv_done > $ sudo perf record -e probe:skb_recv_done -a > ...send some packets to the guest... > ^C > $ sudo perf script >=20 > If you see no skb_recv_done events then the guest driver is not > receiving a notification when packets are received. >=20 > You can find more about how to use perf-probe(1) at > http://blog.vmsplice.net/2011/03/how-to-use-perf-probe.html. Ah nice, I would have used systemtap, but always wanted to check how to= do it with perf :)=20 So once the virtio NIC has locked up, I don't get any events from it an= ymore - until I remove/re-insert the virtio module (including ifup/ifdown). I w= ill try to find some time later on this week to look into it again. Any further ideas how to proceed (I haven't even checked yet how virtio= works at all...). Thanks, Bernd