Re: Hangs

[Marcelo Tosatti <mtosatti@redhat.com>]

On Thu, Nov 20, 2008 at 09:10:57AM -0800, chris@versecorp.net wrote:
> On Wed, Nov 19, 2008 at 02:43:42PM -0800, chris@versecorp.net wrote:
> > Thanks for the responses,
> > 
> > I'm not sure if my problem is the same as Roland's, but it definitely sounds
> > plausible.  I had been running ntpdate in the host to synchronize time every hour (in a cron job), so it sounds as if we could be seeing the same issue.
> > 
> 
> Actually, with ntpdate taken out of crontab, I'm still seeing periodic
> hangs, so it's either a different problem or I'm hitting it in a
> different manner.
> 
> OK, I installed kvm-79 and kernel 2.6.27.6, and here's the the kvm-stat output 
> with 1 guest hung and 3 more operational:

<snip>

> If I shut down the 3 operational guests leaving just the hung guest, the
> kvm-stat output is all 0s:
> 
>  efer_relo      exits  fpu_reloa  halt_exit  halt_wake  host_stat  hypercall  insn_emul  insn_emul     invlpg   io_exits  irq_exits  irq_windo  largepage  mmio_exit  mmu_cache  mmu_flood  mmu_pde_z  mmu_pte_u  mmu_pte_w  mmu_recyc  mmu_shado  nmi_windo   pf_fixed   pf_guest  remote_tl  request_i  signal_ex  tlb_flush
>          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0

So the guest is not actually running here, which means its
QEMU that its hanging at.

> The hung guest in this case was run with this command:
> 
> sudo /usr/local/bin/qemu-system-x86_64             \
>      -daemonize                                    \
>      -no-kvm-irqchip                               \
>      -hda Imgs/ndev_root.img                       \
>      -m 1024                                       \
>      -cdrom ISOs/ubuntu-8.10-server-amd64.iso      \
>      -vnc :4                                       \
>      -net nic,macaddr=DE:AD:BE:EF:04:04,model=e1000 \
>      -net tap,ifname=tap4,script=/home/chris/kvm/qemu-ifup.sh \
>      >>& Logs/ndev_run.log
> 
> 
> I should also mention that when the guest is hung, I can still switch
> to the monitor with ctrl-alt 2. So, at least it's a little bit alive.

In coma perhaps.

> I've also noticed that the behavior with the hung guest is slightly
> different on kvm-79 than it was earlier. When the guest hangs, the kvm
> process in the host doesn't spin at 100% busy any longer - the guest is
> just unresponsive at both the network and VNC console.

> Also, I've noticed that if I reset the guest from the monitor, the
> guest will boot up again, and I can get through to it on the network,
> but strangely, the mouse and keyboard will still be hung at the
> VNC console (except that I can still switch back and forth to the
> monitor).
>
> Hope some of this helps, let me know if you need to me to provide any
> other troubleshooting info.

$ gdb -p pid-of-qemu

(gdb) info threads

Print the backtrace for every thread with:

(gdb) thread N
(gdb) bt