Re: openbsd system_powerdown: "KVM internal error. Suberror: 1"

[Marcelo Tosatti <mtosatti@redhat.com>]

On Wed, Mar 16, 2011 at 11:09:11PM +0300, Michael Tokarev wrote:
> 16.03.2011 22:44, Marcelo Tosatti wrote:
> > On Fri, Mar 11, 2011 at 02:54:00PM +0300, Michael Tokarev wrote:
> >> Hello.
> >>
> >> I installed an openbsd 4.8 image today to play with,
> >> and noticed that when issuing "system_powerdown" in
> >> kvm monitor, in about 5 seconds, qemu-kvm spews this
> >> message in a tight loop:
> >>
> >>  KVM internal error. Suberror: 1
> >>  emulation failure
> >>  KVM internal error. Suberror: 1
> >>  emulation failure
> >>  ....
> >>
> >> ad infinitum, until interrupted.
> >>
> >> I verified the issue exists in 0.14 and 0.12 qemu-kvm,
> >> both 32 and 64bits.
> > 
> >> Freebsd does not trigger this behavour, it is running
> >> normally.
> >>
> >> kvm-0.12.5 behaves somewhat more sane in this case too,
> >> it prints some more information:
> >>
> >>  KVM internal error. Suberror: 1
> >>  rax 0000000000000030 rbx 0000000000000000 rcx 0000000000000000 rdx 000000000000b100
> >>  rsi 00000000d0201fc6 rdi 00000000d0ac1ad4 rsp 00000000d9651004 rbp 00000000d9759a38
> >>  r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
> >>  r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
> >>  rip 00000000d0202041 rflags 00000292
> []
> > Can you stop the guest and issue:
> > 
> > x/10i 0x00000000d0202041
> 
> (qemu) KVM internal error. Suberror: 1
> rax 0000000000000030 rbx 0000000000000000 rcx 00000000d0a3200c rdx 0000000000000000
> rsi 00000000d0201fc6 rdi 00000000d0ac1ad4 rsp 00000000d438d008 rbp 00000000d4495f08
> r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
> r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
> rip 00000000d0202041 rflags 00000282
> cs 0050 (00000000/ffffffff p 1 dpl 0 db 1 s 1 type b l 0 g 1 avl 0)
> ds 0010 (00000000/ffffffff p 1 dpl 0 db 1 s 1 type 3 l 0 g 1 avl 0)
> es 0010 (00000000/ffffffff p 1 dpl 0 db 1 s 1 type 3 l 0 g 1 avl 0)
> ss 0010 (00000000/ffffffff p 1 dpl 0 db 1 s 1 type 3 l 0 g 1 avl 0)
> fs 0058 (d0ac1aa0/000003db p 1 dpl 0 db 0 s 1 type 3 l 0 g 0 avl 0)
> gs 0010 (00000000/ffffffff p 1 dpl 0 db 1 s 1 type 3 l 0 g 1 avl 0)
> tr 0078 (d4494000/00000333 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
> ldt 0018 (d0a31580/00000087 p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
> gdt d42b1000/ffff
> idt d0a31620/7ff
> cr0 8001003b cr2 3c0029b0 cr3 71d8000 cr4 780 cr8 3 efer 0
> emulation failure, check dmesg for details
> (qemu) x/20i 0x00000000d0202036
> 0x00000000d0202036:  pop    %edi
> 0x00000000d0202037:  pop    %esi
> 0x00000000d0202038:  pop    %ebp
> 0x00000000d0202039:  pop    %ebx
> 0x00000000d020203a:  pop    %edx
> 0x00000000d020203b:  pop    %ecx
> 0x00000000d020203c:  pop    %eax
> 0x00000000d020203d:  sti
> 0x00000000d020203e:  add    $0x8,%esp
> 0x00000000d0202041:  iret
> 0x00000000d0202042:  mov    %esi,%esi
> 0x00000000d0202044:  mov    $0x70,%eax
> 0x00000000d0202049:  mov    %eax,0xd0990080
> 0x00000000d020204e:  sti
> 0x00000000d020204f:  push   $0x2
> 0x00000000d0202051:  call   0xd0570470
> 0x00000000d0202056:  add    $0x4,%esp
> 0x00000000d0202059:  jmp    *%esi
> 0x00000000d020205b:  nop
> 0x00000000d020205c:  mov    $0x40,%eax
> 
> The guest stops automatically after that message, so no need to
> stop it.  The address (rip) is the same as before, so it's repeatable.
> 
> I was using 0.12.5 for this, with 0.14 it's impossible to do anything
> after the "Emulation error" anymore, it spews these error messages too
> fast.
> 
> What interesting is that there's some race condition somewhere
> there: I tried 4 times, first the guest just suddenly rebooted
> right after system_powerdown (not shut down, it was a sudden
> reboot), next it went into the above situation, next (after
> restart) it rebooted again, also suddenly, next it come into
> emulation error.  It can do one or another, more or less randomly,
> but the above emulation error is quite a bit more common.

iret emulation is only partially implemented. Why is iret faulting
in the first place i don't know. Can you enable tracing with

echo kvm > /$debugfs/tracing/set_event 

And save the tail of the log, including events at $RIP?