[BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Luca Tettamanti]

(sorry for the double post - I mistyped the address of the list)

Hi Uri, Avi,
I think I debugged the 

kvm: unhandled wrmsr: 0xc0000083

on guest reboot with recent KVM userspace.

The "root" cause is this commit from Uri:

commit 5e1accfc9e7a7d79244c862c04621f7ba23c6d38
Author: Uri Lublin <uril-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
Date:   Tue Jun 19 20:28:02 2007 +0300

    kvm: configure: for i386 machines make x86_64-softmmu the target
    
    To execute the ifdef X86_64 blocks on both sides

diff --git a/configure b/configure
index e4d1ec6..d9292fe 100755
--- a/configure
+++ b/configure
@@ -74,7 +74,7 @@ fi
 
 target_cpu() {
     if [[ $(uname -m) = i?86 ]]; then
-	echo i386
+	echo x86_64
     else
 	uname -m
     fi

which enabled the compilation of code depending on TARGET_X86_64.
Problems arise when the host is in 32 bit mode; Avi fixed part of the
issue with d9ff68d1 (masking the LM bit when the host is 32 bit).

The MSR issue is caused by load_regs (qemu/qemu-kvm.c); at line 304 (git
current) you can see:

#ifdef TARGET_X86_64
    set_msr_entry(&msrs[n++], MSR_CSTAR,             env->cstar);
    set_msr_entry(&msrs[n++], MSR_KERNELGSBASE,      env->kernelgsbase);
    set_msr_entry(&msrs[n++], MSR_FMASK,             env->fmask);
    set_msr_entry(&msrs[n++], MSR_LSTAR ,            env->lstar);
#endif

But the kernel side part (vmx.c) cannot handle those MSRs when using a
32 bit kernel (hence the "unhandled wrmsr").

As a side note: MSC_CSTAR (syscall target for compat mode) is supported
*only* on AMD processors (there's no syscall on Intel in 32 bit mode);
is it safe to use it unconditionally? (AFAICS vmx.c would do wrmsrl,
maybe it's not documented but supported?).

In order to fix this bug I hijacked "lm_capable_kernel" (introduced by
Avi) so that {load,save}_regs don't touch 64bit-only MSRs while the host
is in 32bit mode:

---
 qemu-kvm.c |   22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
index 4ba93d8..b8065db 100644
--- a/qemu/qemu-kvm.c
+++ b/qemu/qemu-kvm.c
@@ -9,6 +9,7 @@
 #endif
 
 int kvm_allowed = KVM_ALLOWED_DEFAULT;
+static int lm_capable_kernel;
 
 #ifdef USE_KVM
 
@@ -302,10 +303,12 @@ static void load_regs(CPUState *env)
 	set_msr_entry(&msrs[n++], MSR_STAR,              env->star);
     set_msr_entry(&msrs[n++], MSR_IA32_TSC, env->tsc);
 #ifdef TARGET_X86_64
-    set_msr_entry(&msrs[n++], MSR_CSTAR,             env->cstar);
-    set_msr_entry(&msrs[n++], MSR_KERNELGSBASE,      env->kernelgsbase);
-    set_msr_entry(&msrs[n++], MSR_FMASK,             env->fmask);
-    set_msr_entry(&msrs[n++], MSR_LSTAR  ,           env->lstar);
+    if (lm_capable_kernel) {
+        set_msr_entry(&msrs[n++], MSR_CSTAR,             env->cstar);
+        set_msr_entry(&msrs[n++], MSR_KERNELGSBASE,      env->kernelgsbase);
+        set_msr_entry(&msrs[n++], MSR_FMASK,             env->fmask);
+        set_msr_entry(&msrs[n++], MSR_LSTAR  ,           env->lstar);
+    }
 #endif
 
     rc = kvm_set_msrs(kvm_context, env->cpu_index, msrs, n);
@@ -439,10 +442,12 @@ static void save_regs(CPUState *env)
 	msrs[n++].index = MSR_STAR;
     msrs[n++].index = MSR_IA32_TSC;
 #ifdef TARGET_X86_64
-    msrs[n++].index = MSR_CSTAR;
-    msrs[n++].index = MSR_KERNELGSBASE;
-    msrs[n++].index = MSR_FMASK;
-    msrs[n++].index = MSR_LSTAR;
+    if (lm_capable_kernel) {
+        msrs[n++].index = MSR_CSTAR;
+        msrs[n++].index = MSR_KERNELGSBASE;
+        msrs[n++].index = MSR_FMASK;
+        msrs[n++].index = MSR_LSTAR;
+    }
 #endif
     rc = kvm_get_msrs(kvm_context, env->cpu_index, msrs, n);
     if (rc == -1) {
@@ -1001,7 +1006,6 @@ static void do_cpuid_ent(struct kvm_cpuid_entry *e, uint32_t function,
     if (function == 0x80000001) {
 	uint32_t h_eax, h_edx;
 	struct utsname utsname;
-	int lm_capable_kernel;
 
 	host_cpuid(function, &h_eax, NULL, NULL, &h_edx);
 	uname(&utsname);

get_msr_entry should be fine, cpu_save/cpu_load (used by savevm -
qemu/vl.c) may need a similar fix.

The patch stops the "unhandled wrmsr", but reboot is still not working
(guest is stuck using 100% of the CPU). The last working userspace is
KVM-28, and I tested it with recent kernel modules. Any idea on this
one?

Luca
-- 
Runtime error 6D at f000:a12f : user incompetente

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Luca Tettamanti wrote:
> which enabled the compilation of code depending on TARGET_X86_64.
> Problems arise when the host is in 32 bit mode; Avi fixed part of the
> issue with d9ff68d1 (masking the LM bit when the host is 32 bit).
>
> The MSR issue is caused by load_regs (qemu/qemu-kvm.c); at line 304 (git
> current) you can see:
>
> #ifdef TARGET_X86_64
>     set_msr_entry(&msrs[n++], MSR_CSTAR,             env->cstar);
>     set_msr_entry(&msrs[n++], MSR_KERNELGSBASE,      env->kernelgsbase);
>     set_msr_entry(&msrs[n++], MSR_FMASK,             env->fmask);
>     set_msr_entry(&msrs[n++], MSR_LSTAR ,            env->lstar);
> #endif
>
> But the kernel side part (vmx.c) cannot handle those MSRs when using a
> 32 bit kernel (hence the "unhandled wrmsr").
>
> As a side note: MSC_CSTAR (syscall target for compat mode) is supported
> *only* on AMD processors (there's no syscall on Intel in 32 bit mode);
> is it safe to use it unconditionally? (AFAICS vmx.c would do wrmsrl,
> maybe it's not documented but supported?).
>
> In order to fix this bug I hijacked "lm_capable_kernel" (introduced by
> Avi) so that {load,save}_regs don't touch 64bit-only MSRs while the host
> is in 32bit mode:
>
>   

Good catch -- patch applied, thanks.

> get_msr_entry should be fine, cpu_save/cpu_load (used by savevm -
> qemu/vl.c) may need a similar fix.
>
> The patch stops the "unhandled wrmsr", but reboot is still not working
> (guest is stuck using 100% of the CPU). The last working userspace is
> KVM-28, and I tested it with recent kernel modules. Any idea on this
> one?
>   

That's around the time kvm moved to its own main loop (for smp), so it's 
not surprising there's breakage there.  I tested erboot at the time, but 
not with all guests.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Jindrich Makovicka]

On Mon, 13 Aug 2007 12:09:46 +0300
Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> > get_msr_entry should be fine, cpu_save/cpu_load (used by savevm -
> > qemu/vl.c) may need a similar fix.
> >
> > The patch stops the "unhandled wrmsr", but reboot is still not
> > working (guest is stuck using 100% of the CPU). The last working
> > userspace is KVM-28, and I tested it with recent kernel modules.
> > Any idea on this one?
> >   
> 
> That's around the time kvm moved to its own main loop (for smp), so
> it's not surprising there's breakage there.  I tested erboot at the
> time, but not with all guests.
 
With newer KVMs (> 28), I also encounter Internet Explorer 6 (WinXP)
lockup with 100% CPU load on the Java download page
(http://jdl.sun.com/webapps/getjava/BrowserRedirect?locale=en&host=java.com:80).
I suspect that IE Java plugin is causing trouble. Desktop Java
applications work well though.

I am running on Core2.

-- 
Jindrich Makovicka


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Jindrich Makovicka wrote:
>
> On Mon, 13 Aug 2007 12:09:46 +0300
> Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
>   
>>> get_msr_entry should be fine, cpu_save/cpu_load (used by savevm -
>>> qemu/vl.c) may need a similar fix.
>>>
>>> The patch stops the "unhandled wrmsr", but reboot is still not
>>> working (guest is stuck using 100% of the CPU). The last working
>>> userspace is KVM-28, and I tested it with recent kernel modules.
>>> Any idea on this one?
>>>   
>>>       
>> That's around the time kvm moved to its own main loop (for smp), so
>> it's not surprising there's breakage there.  I tested erboot at the
>> time, but not with all guests.
>>     
>  
> With newer KVMs (> 28), I also encounter Internet Explorer 6 (WinXP)
> lockup with 100% CPU load on the Java download page
> (http://jdl.sun.com/webapps/getjava/BrowserRedirect?locale=en&host=java.com:80).
> I suspect that IE Java plugin is causing trouble. Desktop Java
> applications work well though.
>
>   

That page doesn't work for me -- it probably expects some HTTP POST
data.  Can you set down instructions to reproduce? (i.e. starting from
java.sun.com or similar)


-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Jindrich Makovicka]

On 8/17/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> That page doesn't work for me -- it probably expects some HTTP POST
> data.  Can you set down instructions to reproduce? (i.e. starting from
> java.sun.com or similar)

Sure.

1) go to http://java.com

2) click the big green "FREE JAVA DOWNLOAD"

Now, only the orange page header should appear, with CPU load at 100%
and IE not responding.

I had JRE 1.6 already installed so I don't know if it locks up without
a Java plugin too or not.

Regards,
-- 
Jindrich Makovicka

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Jindrich Makovicka wrote:
> On 8/17/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
>   
>> That page doesn't work for me -- it probably expects some HTTP POST
>> data.  Can you set down instructions to reproduce? (i.e. starting from
>> java.sun.com or similar)
>>     
>
> Sure.
>
> 1) go to http://java.com
>
> 2) click the big green "FREE JAVA DOWNLOAD"
>
> Now, only the orange page header should appear, with CPU load at 100%
> and IE not responding.
>
> I had JRE 1.6 already installed so I don't know if it locks up without
> a Java plugin too or not.
>
>   

Does this happen with the  Java plugin that's installed by default with 
IE6?  Or do I need to first download and install the plugin, then go 
back to that page again?


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Jindrich Makovicka]

On Sun, 19 Aug 2007 14:23:49 +0300
Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:

> Jindrich Makovicka wrote:
> > On 8/17/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> >   
> >> That page doesn't work for me -- it probably expects some HTTP POST
> >> data.  Can you set down instructions to reproduce? (i.e. starting
> >> from java.sun.com or similar)
> >>     
> >
> > Sure.
> >
> > 1) go to http://java.com
> >
> > 2) click the big green "FREE JAVA DOWNLOAD"
> >
> > Now, only the orange page header should appear, with CPU load at
> > 100% and IE not responding.
> >
> > I had JRE 1.6 already installed so I don't know if it locks up
> > without a Java plugin too or not.
> >
> >   
> 
> Does this happen with the  Java plugin that's installed by default
> with IE6?  Or do I need to first download and install the plugin,
> then go back to that page again?

You need Sun JRE 1.6 installed. It doesn't happen with clean WXP
install (I just rechecked both positive and negative cases).

I originally encountered this lockup with JRE's automatic update, but
the above is easier to reproduce.

-- 
Jindrich Makovicka

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Luca]

On 8/13/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
> Luca Tettamanti wrote:
> > The patch stops the "unhandled wrmsr", but reboot is still not working
> > (guest is stuck using 100% of the CPU). The last working userspace is
> > KVM-28, and I tested it with recent kernel modules. Any idea on this
> > one?
> >
>
> That's around the time kvm moved to its own main loop (for smp), so it's
> not surprising there's breakage there.  I tested erboot at the time, but
> not with all guests.

Here I can't reboot anything... the only thing that works is GRUB, but
I guess that it just goes through the BIOS.

I debugged a bit the problem, but I'm not sure about the solution.
This is what's happening:
- guest requests the reboot, the corresponding branch
kvm_main_loop_cpu() is executed.
- qemu_system_reset() is called; among the other stuff the APIC is
re-initialized.
- kvm_apic_init (called via reset notifier: qemu_system_reset ->
apic_reset -> apic_init_ipi ->
kvm_apic_init) set the ->init flag of the vcpu to 1
- in next loop  if (info->init) is executed. Forever ;-)

The only thing that clears ->init is kvm_update_after_sipi which in
turn is only called by apic_startup; (I'm following the execution with
gdb and that function is never called, btw). Anyway, as long as ->init
is set, guest code won't be executed, so qemu/kvm spins forever in the
main loop, executing update_regs_for_init().

Luca

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Luca wrote:
> On 8/13/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
>   
>> Luca Tettamanti wrote:
>>     
>>> The patch stops the "unhandled wrmsr", but reboot is still not working
>>> (guest is stuck using 100% of the CPU). The last working userspace is
>>> KVM-28, and I tested it with recent kernel modules. Any idea on this
>>> one?
>>>
>>>       
>> That's around the time kvm moved to its own main loop (for smp), so it's
>> not surprising there's breakage there.  I tested erboot at the time, but
>> not with all guests.
>>     
>
> Here I can't reboot anything... the only thing that works is GRUB, but
> I guess that it just goes through the BIOS.
>
> I debugged a bit the problem, but I'm not sure about the solution.
> This is what's happening:
> - guest requests the reboot, the corresponding branch
> kvm_main_loop_cpu() is executed.
> - qemu_system_reset() is called; among the other stuff the APIC is
> re-initialized.
> - kvm_apic_init (called via reset notifier: qemu_system_reset ->
> apic_reset -> apic_init_ipi ->
> kvm_apic_init) set the ->init flag of the vcpu to 1
> - in next loop  if (info->init) is executed. Forever ;-)
>
> The only thing that clears ->init is kvm_update_after_sipi which in
> turn is only called by apic_startup; (I'm following the execution with
> gdb and that function is never called, btw). Anyway, as long as ->init
> is set, guest code won't be executed, so qemu/kvm spins forever in the
> main loop, executing update_regs_for_init().
>   

Hmm. INIT processing should be different for the boot processor 
(->cpu_index == 0), in that it should resume execution after INIT, 
whereas non-boot processors enter a halt state waiting for a SIPI.  
Maybe that's the issue.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Luca Tettamanti]

Il Sun, Aug 19, 2007 at 10:36:06AM +0300, Avi Kivity ha scritto: 
> Luca wrote:
>> On 8/13/07, Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org> wrote:
>>   
>>> Luca Tettamanti wrote:
>>>     
>>>> The patch stops the "unhandled wrmsr", but reboot is still not working
>>>> (guest is stuck using 100% of the CPU). The last working userspace is
>>>> KVM-28, and I tested it with recent kernel modules. Any idea on this
>>>> one?
>>>>
>>>>       
>>> That's around the time kvm moved to its own main loop (for smp), so it's
>>> not surprising there's breakage there.  I tested erboot at the time, but
>>> not with all guests.
>>>     
>>
>> Here I can't reboot anything... the only thing that works is GRUB, but
>> I guess that it just goes through the BIOS.
>>
>> I debugged a bit the problem, but I'm not sure about the solution.
>> This is what's happening:
>> - guest requests the reboot, the corresponding branch
>> kvm_main_loop_cpu() is executed.
>> - qemu_system_reset() is called; among the other stuff the APIC is
>> re-initialized.
>> - kvm_apic_init (called via reset notifier: qemu_system_reset ->
>> apic_reset -> apic_init_ipi ->
>> kvm_apic_init) set the ->init flag of the vcpu to 1
>> - in next loop  if (info->init) is executed. Forever ;-)
>>
>> The only thing that clears ->init is kvm_update_after_sipi which in
>> turn is only called by apic_startup; (I'm following the execution with
>> gdb and that function is never called, btw). Anyway, as long as ->init
>> is set, guest code won't be executed, so qemu/kvm spins forever in the
>> main loop, executing update_regs_for_init().
>>   
>
> Hmm. INIT processing should be different for the boot processor 
> (->cpu_index == 0), in that it should resume execution after INIT, whereas 
> non-boot processors enter a halt state waiting for a SIPI.  Maybe that's 
> the issue.

Yes, the CPU is halted. I don't see any special treatment for BP vs. APs
in the code...
I tried the most obvious fix:

diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
index 709e714..1d83d26 100644
--- a/qemu/qemu-kvm.c
+++ b/qemu/qemu-kvm.c
@@ -702,6 +702,9 @@ static void update_regs_for_init(CPUState *env)
 {
     cpu_reset(env);
     load_regs(env);
+    /* BP */
+    if (env->cpu_index == 0)
+        vcpu_info[env->cpu_index].init = 0;
 }
 
 static void setup_kernel_sigmask(CPUState *env)

In order to reactivate the CPU after the reset. When the guest reboots I
get an unhandled vm exit:

unhandled vm exit:  0x80000021
rax 0000000000000000 rbx 0000000000000000 rcx 0000000000000000 rdx 0000000000000600
rsi 0000000000000000 rdi 0000000000000000 rsp 0000000000000000 rbp 0000000000000000
r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
rip 000000000000fff2 rflags 00000002
cs f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
tr 0080 (10850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
gdt 0/ffff
idt 0/ffff
cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0

which is a vm entry failure due to invalid guest state (RIP looks good
though :P)

Luca
-- 
Al termine di un pranzo di nozze mi hanno dato un
amaro alle erbe cosi' schifoso che perfino sull'etichetta
c'era un frate che vomitava.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Luca Tettamanti wrote:
> In order to reactivate the CPU after the reset. When the guest reboots I
> get an unhandled vm exit:
>
> unhandled vm exit:  0x80000021
> rax 0000000000000000 rbx 0000000000000000 rcx 0000000000000000 rdx 0000000000000600
> rsi 0000000000000000 rdi 0000000000000000 rsp 0000000000000000 rbp 0000000000000000
> r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
> r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
> rip 000000000000fff2 rflags 00000002
> cs f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> tr 0080 (10850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
> ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
> gdt 0/ffff
> idt 0/ffff
> cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
>
> which is a vm entry failure due to invalid guest state (RIP looks good
> though :P)
>   

Actually 0xfff2 is in the middle of an instruction.

I'm guessing an 'out' instruction triggered the reboot, and 
skip_emulated_instruction() added 2 to rip.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Luca Tettamanti]

Il Mon, Aug 20, 2007 at 10:58:57AM +0300, Avi Kivity ha scritto: 
> Luca Tettamanti wrote:
>> In order to reactivate the CPU after the reset. When the guest reboots I
>> get an unhandled vm exit:
>>
>> unhandled vm exit:  0x80000021
>> rax 0000000000000000 rbx 0000000000000000 rcx 0000000000000000 rdx 
>> 0000000000000600
>> rsi 0000000000000000 rdi 0000000000000000 rsp 0000000000000000 rbp 
>> 0000000000000000
>> r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 
>> 0000000000000000
>> r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 
>> 0000000000000000
>> rip 000000000000fff2 rflags 00000002
>> cs f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> tr 0080 (10850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
>> ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
>> gdt 0/ffff
>> idt 0/ffff
>> cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
>>
>> which is a vm entry failure due to invalid guest state (RIP looks good
>> though :P)
>>   
>
> Actually 0xfff2 is in the middle of an instruction.
>
> I'm guessing an 'out' instruction triggered the reboot, and 
> skip_emulated_instruction() added 2 to rip.

I think you're right; the reset is triggered by an outb to 0x64.

Now, with this patch:

diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
index 491c32c..722d838 100644
--- a/qemu/qemu-kvm.c
+++ b/qemu/qemu-kvm.c
@@ -706,8 +706,12 @@ static void update_regs_for_sipi(CPUState *env)
 
 static void update_regs_for_init(CPUState *env)
 {
-    cpu_reset(env);
-    load_regs(env);
+    if (env->cpu_index) {
+        cpu_reset(env);
+        load_regs(env);
+    } else {
+         vcpu_info[env->cpu_index].init = 0;
+    }
 }
 
 static void setup_kernel_sigmask(CPUState *env)

I can reboot using the BIOS (reboot=b) without the outb. I fail to see
why an extra reset causes the vm entry failure though.

Default reboot path (i.e. the outb) still fails:

exception 13 (0)
rax 0000000000000000 rbx 0000000000000000 rcx 000000000000ffff rdx 0000000000000700
rsi 0000000000000000 rdi 0000000000000000 rsp 0000000000000000 rbp 0000000000000000
r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
rip 000000000000ffff rflags 00033046
cs f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
tr 0080 (10850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
gdt 0/ffff
idt 0/ffff
cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 --> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

the #GP makes more sense than the vm entry failure if the the emulator
is jumping to fff2.

Luca
-- 
Il piu` bel momento dell'amore e` quando ci si illude che duri per 
sempre; il piu` brutto, quando ci si accorge che dura da troppo.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Luca Tettamanti wrote:
>> Actually 0xfff2 is in the middle of an instruction.
>>
>> I'm guessing an 'out' instruction triggered the reboot, and 
>> skip_emulated_instruction() added 2 to rip.
>>     
>
> I think you're right; the reset is triggered by an outb to 0x64.
>
> Now, with this patch:
>
> diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
> index 491c32c..722d838 100644
> --- a/qemu/qemu-kvm.c
> +++ b/qemu/qemu-kvm.c
> @@ -706,8 +706,12 @@ static void update_regs_for_sipi(CPUState *env)
>  
>  static void update_regs_for_init(CPUState *env)
>  {
> -    cpu_reset(env);
> -    load_regs(env);
> +    if (env->cpu_index) {
> +        cpu_reset(env);
> +        load_regs(env);
> +    } else {
> +         vcpu_info[env->cpu_index].init = 0;
> +    }
>  }
>   

Can you explain this patch?  Why is the boot cpu treated differently?

I think the only difference should be the halted flag.

>  
>  static void setup_kernel_sigmask(CPUState *env)
>
> I can reboot using the BIOS (reboot=b) without the outb. I fail to see
> why an extra reset causes the vm entry failure though.
>
> Default reboot path (i.e. the outb) still fails:
>
> exception 13 (0)
> rax 0000000000000000 rbx 0000000000000000 rcx 000000000000ffff rdx 0000000000000700
> rsi 0000000000000000 rdi 0000000000000000 rsp 0000000000000000 rbp 0000000000000000
> r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 0000000000000000
> r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000
> rip 000000000000ffff rflags 00033046
> cs f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
> tr 0080 (10850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
> ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
> gdt 0/ffff
> idt 0/ffff
> cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
> code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 --> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>
> the #GP makes more sense than the vm entry failure if the the emulator
> is jumping to fff2.
>   

Right.  Maybe the processor dropped out of vm86 mode and we're getting 
#gp on ds.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Luca Tettamanti]

Il Sun, Sep 09, 2007 at 03:51:20PM +0300, Avi Kivity ha scritto: 
> Luca Tettamanti wrote:
>>> Actually 0xfff2 is in the middle of an instruction.
>>>
>>> I'm guessing an 'out' instruction triggered the reboot, and 
>>> skip_emulated_instruction() added 2 to rip.
>>>     
>>
>> I think you're right; the reset is triggered by an outb to 0x64.
>>
>> Now, with this patch:
>>
>> diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
>> index 491c32c..722d838 100644
>> --- a/qemu/qemu-kvm.c
>> +++ b/qemu/qemu-kvm.c
>> @@ -706,8 +706,12 @@ static void update_regs_for_sipi(CPUState *env)
>>   static void update_regs_for_init(CPUState *env)
>>  {
>> -    cpu_reset(env);
>> -    load_regs(env);
>> +    if (env->cpu_index) {
>> +        cpu_reset(env);
>> +        load_regs(env);
>> +    } else {
>> +         vcpu_info[env->cpu_index].init = 0;
>> +    }
>>  }
>>   
>
> Can you explain this patch?  Why is the boot cpu treated differently?
> I think the only difference should be the halted flag.

The reset has already been done by qmeu_system_reset(), so it's
superfluous. Furthermore, the extra reset causes the vmentry failure. I
still don't understand which check is failing though...

>>   static void setup_kernel_sigmask(CPUState *env)
>>
>> I can reboot using the BIOS (reboot=b) without the outb. I fail to see
>> why an extra reset causes the vm entry failure though.
>>
>> Default reboot path (i.e. the outb) still fails:
>>
>> exception 13 (0)
>> rax 0000000000000000 rbx 0000000000000000 rcx 000000000000ffff rdx 
>> 0000000000000700
>> rsi 0000000000000000 rdi 0000000000000000 rsp 0000000000000000 rbp 
>> 0000000000000000
>> r8  0000000000000000 r9  0000000000000000 r10 0000000000000000 r11 
>> 0000000000000000
>> r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 
>> 0000000000000000
>> rip 000000000000ffff rflags 00033046
>> cs f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> ds 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> es 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)
>> tr 0080 (10850000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)
>> ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)
>> gdt 0/ffff
>> idt 0/ffff
>> cr0 60000010 cr2 0 cr3 0 cr4 0 cr8 0 efer 0
>> code: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 --> 00 
>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
>> 00 00 00 00
>>
>> the #GP makes more sense than the vm entry failure if the the emulator
>> is jumping to fff2.
>
> Right.  Maybe the processor dropped out of vm86 mode and we're getting #gp 
> on ds.

Ok, the culprit really is skip_emulated_instruction: skipping the
increment when EIP is 0xfff0 allows rebooting (yes, it's disgusting...)

So I think that there are two different issues:

1) Extra reset in update_regs_for_init causes vm entry failure due to
   invalid guest state

2) The emulator is doing something wrong since it used to handle the
   reset just fine

Luca
-- 
Una donna sposa un uomo sperando che cambi, e lui non cambiera`. Un
uomo sposa una donna sperando che non cambi, e lei cambiera`.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [BUG][PATCH?] kvm: unhandled wrmsr: 0xc0000083

[Avi Kivity]

Luca Tettamanti wrote:
> Il Sun, Sep 09, 2007 at 03:51:20PM +0300, Avi Kivity ha scritto: 
>   
>> Luca Tettamanti wrote:
>>     
>>>> Actually 0xfff2 is in the middle of an instruction.
>>>>
>>>> I'm guessing an 'out' instruction triggered the reboot, and 
>>>> skip_emulated_instruction() added 2 to rip.
>>>>     
>>>>         
>>> I think you're right; the reset is triggered by an outb to 0x64.
>>>
>>> Now, with this patch:
>>>
>>> diff --git a/qemu/qemu-kvm.c b/qemu/qemu-kvm.c
>>> index 491c32c..722d838 100644
>>> --- a/qemu/qemu-kvm.c
>>> +++ b/qemu/qemu-kvm.c
>>> @@ -706,8 +706,12 @@ static void update_regs_for_sipi(CPUState *env)
>>>   static void update_regs_for_init(CPUState *env)
>>>  {
>>> -    cpu_reset(env);
>>> -    load_regs(env);
>>> +    if (env->cpu_index) {
>>> +        cpu_reset(env);
>>> +        load_regs(env);
>>> +    } else {
>>> +         vcpu_info[env->cpu_index].init = 0;
>>> +    }
>>>  }
>>>   
>>>       
>> Can you explain this patch?  Why is the boot cpu treated differently?
>> I think the only difference should be the halted flag.
>>     
>
> The reset has already been done by qmeu_system_reset(), so it's
> superfluous. Furthermore, the extra reset causes the vmentry failure. 

I just committed a patch which prevented .init from being set to 1 on 
cpu_index == 0.

> I
> still don't understand which check is failing though...
>
>   

These are tough...

>>> the #GP makes more sense than the vm entry failure if the the emulator
>>> is jumping to fff2.
>>>       
>> Right.  Maybe the processor dropped out of vm86 mode and we're getting #gp 
>> on ds.
>>     
>
> Ok, the culprit really is skip_emulated_instruction: skipping the
> increment when EIP is 0xfff0 allows rebooting (yes, it's disgusting...)
>
> So I think that there are two different issues:
>
> 1) Extra reset in update_regs_for_init causes vm entry failure due to
>    invalid guest state
>
> 2) The emulator is doing something wrong since it used to handle the
>    reset just fine
>   

It may have been timing.  kvm continued to run for a bit, reaching a 
non-emulated instruction, and then the reset hit it in the face.  The 
reset is much quicker now.

We should probably both fix the kernel to handle reset-during-emulation 
correctly (one ugly way is to zero the instruction length if we're 
setting rip), and fix userspace to delay reset like it used to for 
compatibility with older kernels.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/