From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kiszka Subject: Re: one out of four existing kvm guest's not starting after system upgrade Date: Wed, 22 Feb 2012 18:58:29 +0100 Message-ID: <4F452CC5.20001@web.de> References: <201202172149.36904.thomas@fjellstrom.ca> <4F3F67A4.5050309@web.de> <201202180257.48987.thomas@fjellstrom.ca> <201202191313.58710.thomas@fjellstrom.ca> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig32E95A7950197CC89233CFA7" Cc: KVM To: thomas@fjellstrom.ca Return-path: Received: from fmmailgate04.web.de ([217.72.192.242]:55942 "EHLO fmmailgate04.web.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753119Ab2BVR6d (ORCPT ); Wed, 22 Feb 2012 12:58:33 -0500 Received: from moweb001.kundenserver.de (moweb001.kundenserver.de [172.19.20.114]) by fmmailgate04.web.de (Postfix) with ESMTP id B460C728260F for ; Wed, 22 Feb 2012 18:58:31 +0100 (CET) In-Reply-To: <201202191313.58710.thomas@fjellstrom.ca> Sender: kvm-owner@vger.kernel.org List-ID: This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig32E95A7950197CC89233CFA7 Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: quoted-printable On 2012-02-19 21:13, Thomas Fjellstrom wrote: > On Sat Feb 18, 2012, Thomas Fjellstrom wrote: >> On Sat Feb 18, 2012, Jan Kiszka wrote: >>> On 2012-02-18 09:50, Thomas Fjellstrom wrote: >>>> On Sat Feb 18, 2012, Jan Kiszka wrote: >>>>> On 2012-02-18 05:49, Thomas Fjellstrom wrote: >>>>>> I just updated my kvm host, kernel upgraded from 2.6.38 up to 3.2,= >>>>>> and qemu+qemu-kvm updated (not sure from what to what). But after >>>>>> the upgrade, one of my guests will not start up. It gets stuck wit= h >>>>>> 60-80% cpu use, almost no memory is allocated by qemu/kvm, and no >>>>>> output of any kind is seen (in the host console, or a bunch of the= >>>>>> guest output options like curses display, stdio output, virsh >>>>>> console, vnc or sdl output). I normally use libvirt to manage the >>>>>> guests, but I've attempted to run qemu manually, and have the same= >>>>>> problems. >>>>>> >>>>>> What can cause this? >>>>>> >>>>>> Just tested booting back into the old kernel, the one guest still >>>>>> won't start, while the rest do. I'm thoroughly confused. >>>>> >>>>> You mean if you only update qemu-kvm, the problem persists, just wi= th >>>>> lower probability? In that case, we definitely need the version of >>>>> your current qemu-kvm installation. Also, it would be nice to attac= h >>>>> gdb to the stuck qemu-kvm process, issuing a "thread apply all >>>>> backtrace" in that state. >>>>> >>>>> Jan >>>> >>>> Sorry I wasn't clear. If I just update qemu-kvm (And qemu with it, a= nd >>>> not the kernel), it always just hangs on load. >>>> >>>> current version: >>>> QEMU emulator version 1.0 (qemu-kvm-1.0 Debian 1.0+dfsg-8), Copyrigh= t >>>> (c) 2003-2008 Fabrice Bellard >>>> >>>> gdb thread apply all bt: >>>> Thread 2 (Thread 0x7fe7b810c700 (LWP 14650)): >>>> #0 0x00007fe7c0a11957 in ioctl () from /lib/x86_64-linux-gnu/libc.s= o.6 >>>> #1 0x00007fe7c53155e9 in kvm_vcpu_ioctl (env=3D, >>>> type=3D) at >>>> /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/kvm= - >>>> all.c:1101 >>>> #2 0x00007fe7c5315731 in kvm_cpu_exec (env=3D0x7fe7c61cb350) at >>>> /build/buildd- >>>> qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/kvm-all.c:987 #3 >>>> 0x00007fe7c52ecf31 in qemu_kvm_cpu_thread_fn (arg=3D0x7fe7c61cb350) = at >>>> /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/cpu= s.c >>>> : 740 #4 0x00007fe7c0ccdb50 in start_thread () from /lib/x86_64-lin= ux- >>>> gnu/libpthread.so.0 >>>> #5 0x00007fe7c0a1890d in clone () from /lib/x86_64-linux-gnu/libc.s= o.6 >>>> #6 0x0000000000000000 in ?? () >>>> >>>> Thread 1 (Thread 0x7fe7c5101900 (LWP 14648)): >>>> #0 0x00007fe7c0a12403 in select () from >>>> /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007fe7c525b56c in >>>> main_loop_wait (nonblocking=3D) at >>>> /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/mai= n- >>>> loop.c:456 >>>> #2 0x00007fe7c51a372f in main_loop () at >>>> /build/buildd-qemu-kvm_1.0+dfsg-8- >>>> amd64-ppNMqm/qemu-kvm-1.0+dfsg/vl.c:1482 >>>> #3 main (argc=3D, argv=3D, envp=3D>>> out>) at >>>> /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/vl.= c:3 >>>> 5 23 >>>> >>>> Thanks :) >>> >>> OK, then we need a kernel view on this. Can you try >>> >>> http://www.linux-kvm.org/page/Tracing >>> >>> ? >>> >>> Thanks, >>> Jan >> >> It made a rather large trace file. I'm pretty sure no-one wants me to >> actually link (or attach) the full 1.5G file, but the last 5000 lines >> might be useful, so I've compressed and attached them. (but just in ca= se, >> I'm lzma'ing the entire 1.5G trace data file) >=20 > I'm pretty much stumped on this. So I decided to try re-creating the vm= =20 > through virt-manager. Its up and running now. The only to major differe= nces I=20 > can see in the old and new config is the machine (-M pc-0.12 vs -M pc-1= =2E0)=20 > parameter, and the uuid. The rest of the parameters I played with a lot= trying=20 > to get it to work by starting up the vm manually from the cli. I can't = really=20 > see how those two changes would do much of anything considering the oth= er=20 > three VM's still are configured to use -M pc-0.12, and they work fine. >=20 Sorry, on vacation with limited IT access, thus wasn't able to look that traces yet. Hope someone else can pick this up. Jan --------------enig32E95A7950197CC89233CFA7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk9FLMoACgkQitSsb3rl5xSyCgCfRrMMcq8EGaOO/CFF3hcnles+ HbkAnAyj1yh/N1PRGqWHEL83WRK6JD5F =Ft1D -----END PGP SIGNATURE----- --------------enig32E95A7950197CC89233CFA7--