Re: one out of four existing kvm guest's not starting after system upgrade

[Thomas Fjellstrom <thomas@fjellstrom.ca>]

On Sat Feb 18, 2012, Jan Kiszka wrote:
> On 2012-02-18 09:50, Thomas Fjellstrom wrote:
> > On Sat Feb 18, 2012, Jan Kiszka wrote:
> >> On 2012-02-18 05:49, Thomas Fjellstrom wrote:
> >>> I just updated my kvm host, kernel upgraded from 2.6.38 up to 3.2, and
> >>> qemu+qemu-kvm updated (not sure from what to what). But after the
> >>> upgrade, one of my guests will not start up. It gets stuck with 60-80%
> >>> cpu use, almost no memory is allocated by qemu/kvm, and no output of
> >>> any kind is seen (in the host console, or a bunch of the guest output
> >>> options like curses display, stdio output, virsh console, vnc or sdl
> >>> output). I normally use libvirt to manage the guests, but I've
> >>> attempted to run qemu manually, and have the same problems.
> >>> 
> >>> What can cause this?
> >>> 
> >>> Just tested booting back into the old kernel, the one guest still won't
> >>> start, while the rest do. I'm thoroughly confused.
> >> 
> >> You mean if you only update qemu-kvm, the problem persists, just with
> >> lower probability? In that case, we definitely need the version of your
> >> current qemu-kvm installation. Also, it would be nice to attach gdb to
> >> the stuck qemu-kvm process, issuing a "thread apply all backtrace" in
> >> that state.
> >> 
> >> Jan
> > 
> > Sorry I wasn't clear. If I just update qemu-kvm (And qemu with it, and
> > not the kernel), it always just hangs on load.
> > 
> > current version:
> > QEMU emulator version 1.0 (qemu-kvm-1.0 Debian 1.0+dfsg-8), Copyright (c)
> > 2003-2008 Fabrice Bellard
> > 
> > gdb thread apply all bt:
> > Thread 2 (Thread 0x7fe7b810c700 (LWP 14650)):
> > #0  0x00007fe7c0a11957 in ioctl () from /lib/x86_64-linux-gnu/libc.so.6
> > #1  0x00007fe7c53155e9 in kvm_vcpu_ioctl (env=<optimized out>,
> > type=<optimized out>) at
> > /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/kvm-
> > all.c:1101
> > #2  0x00007fe7c5315731 in kvm_cpu_exec (env=0x7fe7c61cb350) at
> > /build/buildd-
> > qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/kvm-all.c:987 #3 
> > 0x00007fe7c52ecf31 in qemu_kvm_cpu_thread_fn (arg=0x7fe7c61cb350) at
> > /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/cpus.c:
> > 740 #4  0x00007fe7c0ccdb50 in start_thread () from /lib/x86_64-linux-
> > gnu/libpthread.so.0
> > #5  0x00007fe7c0a1890d in clone () from /lib/x86_64-linux-gnu/libc.so.6
> > #6  0x0000000000000000 in ?? ()
> > 
> > Thread 1 (Thread 0x7fe7c5101900 (LWP 14648)):
> > #0  0x00007fe7c0a12403 in select () from /lib/x86_64-linux-gnu/libc.so.6
> > #1  0x00007fe7c525b56c in main_loop_wait (nonblocking=<optimized out>) at
> > /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/main-
> > loop.c:456
> > #2  0x00007fe7c51a372f in main_loop () at
> > /build/buildd-qemu-kvm_1.0+dfsg-8-
> > amd64-ppNMqm/qemu-kvm-1.0+dfsg/vl.c:1482
> > #3  main (argc=<optimized out>, argv=<optimized out>, envp=<optimized
> > out>) at
> > /build/buildd-qemu-kvm_1.0+dfsg-8-amd64-ppNMqm/qemu-kvm-1.0+dfsg/vl.c:35
> > 23
> > 
> > Thanks :)
> 
> OK, then we need a kernel view on this. Can you try
> 
> http://www.linux-kvm.org/page/Tracing
> 
> ?
> 
> Thanks,
> Jan

Before I get done with that, does it help that I see some dmesg warnings (with 
the 3.2 kernel) relating to some ioctls? 

[   26.694981] kvm: sending ioctl 80200204 to a partition!
[   26.695036] kvm: sending ioctl 5326 to a partition!

-- 
Thomas Fjellstrom
thomas@fjellstrom.ca