From mboxrd@z Thu Jan 1 00:00:00 1970 From: chris@versecorp.net Subject: Re: Hangs Date: Thu, 20 Nov 2008 09:10:57 -0800 Message-ID: <20081120171057.GA13638@versecorp.net> References: <200811192243.mAJMhgFU005350@versecorp.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii To: kvm@vger.kernel.org, mtosatti@redhat.com, rl@brabbel.net Return-path: Received: from www.versecorp.net ([74.208.43.213]:47342 "EHLO versecorp.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756150AbYKTRLt (ORCPT ); Thu, 20 Nov 2008 12:11:49 -0500 Content-Disposition: inline In-Reply-To: <200811192243.mAJMhgFU005350@versecorp.net> Sender: kvm-owner@vger.kernel.org List-ID: On Wed, Nov 19, 2008 at 02:43:42PM -0800, chris@versecorp.net wrote: > Thanks for the responses, > > I'm not sure if my problem is the same as Roland's, but it definitely sounds > plausible. I had been running ntpdate in the host to synchronize time every hour (in a cron job), so it sounds as if we could be seeing the same issue. > Actually, with ntpdate taken out of crontab, I'm still seeing periodic hangs, so it's either a different problem or I'm hitting it in a different manner. OK, I installed kvm-79 and kernel 2.6.27.6, and here's the the kvm-stat output with 1 guest hung and 3 more operational: efer_relo exits fpu_reloa halt_exit halt_wake host_stat hypercall insn_emul insn_emul invlpg io_exits irq_exits irq_windo largepage mmio_exit mmu_cache mmu_flood mmu_pde_z mmu_pte_u mmu_pte_w mmu_recyc mmu_shado nmi_windo pf_fixed pf_guest remote_tl request_i signal_ex tlb_flush 0 333 24 32 0 331 2 212 0 0 78 4 0 0 188 0 0 0 2 2 0 0 0 1 1 0 0 0 12 0 360 3 30 0 331 0 290 0 0 0 4 0 0 269 0 0 0 5 5 0 0 0 35 5 0 0 0 15 0 287 2 30 0 307 0 202 0 0 52 2 0 0 194 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 389 20 29 0 405 0 277 0 0 78 3 0 0 267 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 307 4 32 0 315 0 219 0 0 52 3 0 0 198 0 0 0 0 0 0 0 0 0 0 0 0 0 11 0 327 2 35 0 346 2 285 0 0 0 4 0 0 274 0 0 0 2 2 0 0 0 1 1 0 0 0 7 0 334 22 31 0 342 0 217 0 0 78 4 0 0 201 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 311 3 28 0 324 0 280 0 0 0 2 0 0 265 0 0 0 0 0 0 0 0 0 0 0 0 0 9 0 292 2 32 0 313 0 204 0 0 52 3 0 0 196 0 0 0 0 0 0 0 0 0 0 0 0 0 4 0 791 23 46 0 780 10 352 0 0 364 6 0 0 320 0 0 0 10 10 0 0 0 5 5 0 0 0 20 0 251 3 30 0 259 2 214 0 0 0 4 0 0 198 0 0 0 2 2 0 0 0 1 1 0 0 0 10 0 313 2 31 0 330 0 278 0 0 0 4 0 0 266 0 0 0 0 0 0 0 0 0 0 0 0 0 6 0 330 22 30 0 339 0 215 0 0 78 3 0 0 199 0 0 0 0 0 0 0 0 0 0 0 0 0 8 If I shut down the 3 operational guests leaving just the hung guest, the kvm-stat output is all 0s: efer_relo exits fpu_reloa halt_exit halt_wake host_stat hypercall insn_emul insn_emul invlpg io_exits irq_exits irq_windo largepage mmio_exit mmu_cache mmu_flood mmu_pde_z mmu_pte_u mmu_pte_w mmu_recyc mmu_shado nmi_windo pf_fixed pf_guest remote_tl request_i signal_ex tlb_flush 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 The hung guest in this case was run with this command: sudo /usr/local/bin/qemu-system-x86_64 \ -daemonize \ -no-kvm-irqchip \ -hda Imgs/ndev_root.img \ -m 1024 \ -cdrom ISOs/ubuntu-8.10-server-amd64.iso \ -vnc :4 \ -net nic,macaddr=DE:AD:BE:EF:04:04,model=e1000 \ -net tap,ifname=tap4,script=/home/chris/kvm/qemu-ifup.sh \ >>& Logs/ndev_run.log I should also mention that when the guest is hung, I can still switch to the monitor with ctrl-alt 2. So, at least it's a little bit alive. I've also noticed that the behavior with the hung guest is slightly different on kvm-79 than it was earlier. When the guest hangs, the kvm process in the host doesn't spin at 100% busy any longer - the guest is just unresponsive at both the network and VNC console. Also, I've noticed that if I reset the guest from the monitor, the guest will boot up again, and I can get through to it on the network, but strangely, the mouse and keyboard will still be hung at the VNC console (except that I can still switch back and forth to the monitor). Hope some of this helps, let me know if you need to me to provide any other troubleshooting info. Chris