KVM crash on unusual PM->RM transition

KVM crash on unusual PM->RM transition

[H. Peter Anvin]

The current test branch of Syslinux does PM->RM transition via a stub 
which can be located in high memory.  This is correct and valid (x86 
CPUs rely on this to boot): after clearing CR0.PE, CS.base retains its 
previous value until CS is reloaded by a far JMP (which in this case is 
the very next instruction).

Unfortunately, at least on a stock Fedora 10 system, KVM doesn't agree:

: anacreon 67 ; qemu-kvm -boot d -cdrom image.iso

unhandled vm exit: 0x80000021 vcpu_id 0
rax 0000000000000012 rbx 0000000007ade342 rcx 0000000000000000 rdx 
0000000007ade388
rsi 0000000000000000 rdi 0000000000800000 rsp 0000000007ade448 rbp 
0000000000007b94
r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 
0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 
0000000000000000
rip 0000000000000008 rflags 00023006
cs de36 (07ade36e/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
tr 0030 (fffbd000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
ldt 0000 (00000000/ffffffff p 0 dpl 0 db 0 s 0 type 0 l 0 g 0 avl 0)
gdt 7ade388/3f
idt 0/ffff
cr0 12 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
Abort

Kernel 2.6.27.21-170.2.56.fc10.x86_64, kvm-74-10.fc10.x86_64.

	-hpa

Re: KVM crash on unusual PM->RM transition

[H. Peter Anvin]

H. Peter Anvin wrote:
> The current test branch of Syslinux does PM->RM transition via a stub
> which can be located in high memory.  This is correct and valid (x86
> CPUs rely on this to boot): after clearing CR0.PE, CS.base retains its
> previous value until CS is reloaded by a far JMP (which in this case is
> the very next instruction).

Some trivial research later...

It looks like KVM will simply crash when it runs into a real-mode state
it can't approximate with V86 mode.  I guess I had the failed notion
that it would kick back such "impossible" states to Qemu.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: KVM crash on unusual PM->RM transition

[Avi Kivity]

H. Peter Anvin wrote:
> H. Peter Anvin wrote:
>   
>> The current test branch of Syslinux does PM->RM transition via a stub
>> which can be located in high memory.  This is correct and valid (x86
>> CPUs rely on this to boot): after clearing CR0.PE, CS.base retains its
>> previous value until CS is reloaded by a far JMP (which in this case is
>> the very next instruction).
>>     
>
> Some trivial research later...
>
> It looks like KVM will simply crash when it runs into a real-mode state
> it can't approximate with V86 mode.  I guess I had the failed notion
> that it would kick back such "impossible" states to Qemu.
>   

Exactly.  There's the emulate_invalid_guest_state module parameter which 
tells kvm to emulate during such state instead.  But this will often 
break as programs leave fs and gs in non-v86-mode compliant, requiring 
more of the emulator than it currently provides.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM crash on unusual PM->RM transition

[H. Peter Anvin]

Avi Kivity wrote:
>>
>> It looks like KVM will simply crash when it runs into a real-mode state
>> it can't approximate with V86 mode.  I guess I had the failed notion
>> that it would kick back such "impossible" states to Qemu.
> 
> Exactly.  There's the emulate_invalid_guest_state module parameter which
> tells kvm to emulate during such state instead.  But this will often
> break as programs leave fs and gs in non-v86-mode compliant, requiring
> more of the emulator than it currently provides.
> 

Any reason to not make use of Qemu in userspace, rather than relying on
the in-kernel interpreter for these?  The kernel interpreter is
obviously The Right Thing to avoid frequent ping-pong into the kernel,
but it seems to me that such a potential "long-term" situation might be
better handled by Qemu?

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: KVM crash on unusual PM->RM transition

[Avi Kivity]

H. Peter Anvin wrote:
> Any reason to not make use of Qemu in userspace, rather than relying on
> the in-kernel interpreter for these?  The kernel interpreter is
> obviously The Right Thing to avoid frequent ping-pong into the kernel,
> but it seems to me that such a potential "long-term" situation might be
> better handled by Qemu?
>   

It's a faq; there are multiple reasons:

- it would make the kernel/user interface complicated; right now the 
model is "kvm emulates the cpu and optionally lapic, ioapic, and pit'; 
that would change to "kvm sometimes emulates the cpu, but sometimes 
doesn't".  We'd need to add a description of when userspace can jump 
back into the kernel (AMD for example is perfectly happy in real mode).

- it would require a way for userspace to access the in-kernel 
lapic/ioapic/pit, and for these components to inject interrupts into 
userspace.

- it requires anyone using kvm to implement a complete x86 emulator.

- qemu doesn't do smp.

with emulate_invalid_guest_space we're actually pretty close to running 
big real mode; but no one's working on it so it's stagnated.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM crash on unusual PM->RM transition

[H. Peter Anvin]

Avi Kivity wrote:
> 
> - qemu doesn't do smp.
> 

It certainly claims to.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: KVM crash on unusual PM->RM transition

[Avi Kivity]

H. Peter Anvin wrote:
> Avi Kivity wrote:
>   
>> - qemu doesn't do smp.
>>
>>     
>
> It certainly claims to.
>
>   

Sorry, not specific enough: it doesn't do real SMP.  Atomic instructions 
are translated as multiple, non-atomic instructions, so you can't run 
tcg compiled code in parallel.  If your host is different than your 
target, you have memory ordering issues as well.

What qemu does is to multiplex multiple guest cpus on one host thread.  
It's nice but it isn't how kvm works.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM crash on unusual PM->RM transition

[H. Peter Anvin]

Avi Kivity wrote:
> 
> Sorry, not specific enough: it doesn't do real SMP.  Atomic instructions
> are translated as multiple, non-atomic instructions, so you can't run
> tcg compiled code in parallel.  If your host is different than your
> target, you have memory ordering issues as well.
> 
> What qemu does is to multiplex multiple guest cpus on one host thread. 
> It's nice but it isn't how kvm works.
> 

Erk, nevermind.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.