Re: paravirt clock stil causing hangs in kvm-65

[Marcelo Tosatti <mtosatti@redhat.com>]

On Mon, Apr 07, 2008 at 06:34:57PM -0300, Marcelo Tosatti wrote:
> On Mon, Apr 07, 2008 at 01:53:36PM +0200, Nikola Ciprich wrote:
> > Hi,
> > 
> > I also tried paravirt clock again in latest git with kvm-65 patch 
> > applied, and problem with cpu-lockups persists:
> > 
> > [10813.654806] BUG: soft lockup - CPU#0 stuck for 61s! [swapper:0]
> > [10813.655789] CPU 0:
> > [10813.656624] Modules linked in: virtio_pci virtio_ring virtio_blk virtio 
> > piix dm_snapshot dm_zero dm_mirror dm_mod ide_disk
> >   ide_core sd_mod scsi_mod ext3 jbd ehci_hcd ohci_hcd uhci_hcd
> > [10813.658805] Pid: 0, comm: swapper Not tainted 2.6.25-rc7 #5
> > [10813.658805] RIP: 0010:[<ffffffff80222ab2>]  [<ffffffff80222ab2>] 
> > native_safe_halt+0x2/0x10
> > [10813.658805] RSP: 0018:ffffffff805adf50  EFLAGS: 00000296
> > [10813.658805] RAX: 000000019b08eeb0 RBX: ffffffff805f5000 RCX: 
> > 000000019b08eeb0
> > [10813.658805] RDX: 0000000000000006 RSI: 00000000356832b0 RDI: 
> > ffffffff805adf38
> > [10813.658805] RBP: 0000000000000da8 R08: 0000000000000000 R09: 
> > 0000000000000000
> > [10813.658805] R10: 0000000000000001 R11: 0000000000000002 R12: 
> > ffffffff802228ed
> > [10813.658805] R13: 00000000000132a0 R14: ffffffff80200bba R15: 
> > ffff81000100a280
> > [10813.658805] FS:  0000000000000000(0000) GS:ffffffff80576000(0000) 
> > knlGS:0000000000000000
> > [10813.658805] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> > [10813.658805] CR2: 00007fac0f852000 CR3: 0000000000201000 CR4: 
> > 00000000000006e0
> > [10813.658805] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> > 0000000000000000
> > [10813.658805] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> > 0000000000000400
> > [10813.658805]
> > [10813.658805] Call Trace:
> > [10813.658805]  [<ffffffff8020a55b>] ? default_idle+0x3b/0x70
> > [10813.658805]  [<ffffffff8020a520>] ? default_idle+0x0/0x70
> > [10813.658805]  [<ffffffff8020a60e>] ? cpu_idle+0x7e/0xe0
> > [10813.658805]  [<ffffffff80211630>] ? pda_init+0x30/0xb0
> > 
> > Can I somehow help to track this one down??
> 
> Hi Nikola,
> 
> I just reproduced this on a UP guest. Were you seeing the exact same
> stack trace in the guest with kvm-64 ?

I've been able to reproduce the problem. Symptoms are that when using
NOHZ vcpu0 LAPIC timer is ticking far less than the others (apparently vcpu0
is the only one ticking "correctly"):


nohz=on with kvmclock
[root@localhost ~]# cat /proc/timer_stats | grep apic
 13214,  8590 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 13214,  8589 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 13211,  8588 qemu-system-x86  apic_mmio_write (apic_timer_fn)
  389,  8587 qemu-system-x86  apic_mmio_write (apic_timer_fn)

nohz=off
 3253,  8672 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 2876,  8673 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 2543,  8674 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 2179,  8675 qemu-system-x86  apic_mmio_write (apic_timer_fn)

no-kvmclock
 1017,  8808 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 1577,  8809 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 1708,  8807 qemu-system-x86  apic_mmio_write (apic_timer_fn)
 1812,  8806 qemu-system-x86  apic_mmio_write (apic_timer_fn)


Glauber will start looking at this next week.


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone