bugreport: guest crashes on reboot in KVM 65 on AMD

bugreport: guest crashes on reboot in KVM 65 on AMD

[Michal Ludvig]

Hi all,

in short: CentOS 5 guest (32bit) crashes when I run "reboot" in the VM 
with qemu message:
kvm_run: failed entry, reason 65535
kvm_run returned -8

In dmesg is this message:
kvm: 23769: cpu0 kvm_set_msr_common: MSR_IA32_MC0_STATUS 0x0, nop

Details:
KVM 65
Host is openSUSE 10.3 with my own 2.6.24, 64 bit
Guest is CentOS 5.1 with 2.6.18-53.1.14.el5, 32 bit
GCC 4.2.1 for module, GCC 3.3.3 for qemu
CPU "AMD Athlon(tm) 64 X2 Dual Core Processor 5600+"

Everything works fine, except that I can't reboot the guest. In VNC it 
says "md: stopping devices" and then Qemu exits with:
kvm_run: failed entry, reason 65535
kvm_run returned -8

IIRC reboot used to work with kvm-64

If you need more information let me know.

Michal

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: bugreport: guest crashes on reboot in KVM 65 on AMD

[Alexey Eremenko]

On Sun, Apr 13, 2008 at 1:49 PM, Michal Ludvig <mludvig@logix.net.nz> wrote:
> Hi all,
>
>  in short: CentOS 5 guest (32bit) crashes when I run "reboot" in the VM
>  with qemu message:
>  kvm_run: failed entry, reason 65535
>  kvm_run returned -8
>
>  In dmesg is this message:
>  kvm: 23769: cpu0 kvm_set_msr_common: MSR_IA32_MC0_STATUS 0x0, nop
>
>  Details:
>  KVM 65
>  Host is openSUSE 10.3 with my own 2.6.24, 64 bit
>  Guest is CentOS 5.1 with 2.6.18-53.1.14.el5, 32 bit
>  GCC 4.2.1 for module, GCC 3.3.3 for qemu
>  CPU "AMD Athlon(tm) 64 X2 Dual Core Processor 5600+"
>
>  Everything works fine, except that I can't reboot the guest. In VNC it
>  says "md: stopping devices" and then Qemu exits with:
>  kvm_run: failed entry, reason 65535
>  kvm_run returned -8
>
>  IIRC reboot used to work with kvm-64
>
>  If you need more information let me know.
>
>  Michal

Yea, this is known bug, that I reported to Avi recently. I hope it
will be fixed soon.

-- 
-Alexey Eromenko "Technologov"

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: bugreport: guest crashes on reboot in KVM 65 on AMD

[Avi Kivity]

Michal Ludvig wrote:
> Hi all,
>
> in short: CentOS 5 guest (32bit) crashes when I run "reboot" in the VM 
> with qemu message:
> kvm_run: failed entry, reason 65535
> kvm_run returned -8
>
> In dmesg is this message:
> kvm: 23769: cpu0 kvm_set_msr_common: MSR_IA32_MC0_STATUS 0x0, nop
>
> Details:
> KVM 65
> Host is openSUSE 10.3 with my own 2.6.24, 64 bit
> Guest is CentOS 5.1 with 2.6.18-53.1.14.el5, 32 bit
> GCC 4.2.1 for module, GCC 3.3.3 for qemu
> CPU "AMD Athlon(tm) 64 X2 Dual Core Processor 5600+"
>
> Everything works fine, except that I can't reboot the guest. In VNC it 
> says "md: stopping devices" and then Qemu exits with:
> kvm_run: failed entry, reason 65535
> kvm_run returned -8
>
> IIRC reboot used to work with kvm-64
>   

I think two bugs are involved:

- the guest invokes the triple-fault shutdown path, so it seems normal 
reboot doesn't work with iothread.  Haven't looked as to why yet.
- amd shutdown interception clears the vmcb, leaving asid == 0, which is 
illegal.  Adding force_new_asid() fixes this.  There are probably other 
issues with clearing the vmcb as well.

Joerg, can you find out if the "vmcb is undefined" applies to the entire 
vmcb or just the guest saved state?  Is there a way to differentiate 
between a guest initiated shutdown due to triple fault and the core 
melting down?

I'll commit the fix for the second problem, and look at the first later.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: bugreport: guest crashes on reboot in KVM 65 on AMD

[Joerg Roedel]

On Mon, Apr 14, 2008 at 12:43:30PM +0300, Avi Kivity wrote:
> Michal Ludvig wrote:
> >Hi all,
> >
> >in short: CentOS 5 guest (32bit) crashes when I run "reboot" in the VM with qemu message:
> >kvm_run: failed entry, reason 65535
> >kvm_run returned -8
> >
> >In dmesg is this message:
> >kvm: 23769: cpu0 kvm_set_msr_common: MSR_IA32_MC0_STATUS 0x0, nop
> >
> >Details:
> >KVM 65
> >Host is openSUSE 10.3 with my own 2.6.24, 64 bit
> >Guest is CentOS 5.1 with 2.6.18-53.1.14.el5, 32 bit
> >GCC 4.2.1 for module, GCC 3.3.3 for qemu
> >CPU "AMD Athlon(tm) 64 X2 Dual Core Processor 5600+"
> >
> >Everything works fine, except that I can't reboot the guest. In VNC it says "md: stopping devices" and then Qemu exits with:
> >kvm_run: failed entry, reason 65535
> >kvm_run returned -8
> >
> >IIRC reboot used to work with kvm-64
> >  
> 
> I think two bugs are involved:
> 
> - the guest invokes the triple-fault shutdown path, so it seems normal reboot doesn't work with iothread.  Haven't looked as to why yet.
> - amd shutdown interception clears the vmcb, leaving asid == 0, which is illegal.  Adding force_new_asid() fixes this.  There are probably 
> other issues with clearing the vmcb as well.
> 
> Joerg, can you find out if the "vmcb is undefined" applies to the entire vmcb or just the guest saved state?  Is there a way to 
> differentiate between a guest initiated shutdown due to triple fault and the core melting down?

Only the state saved in the VMCB is undefined after an intercepted
shutdown event. I am not sure if I understand your second question, what
do you mean with "core melting down" vs. guest initiated shutdown?

Joerg


-- 
           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: bugreport: guest crashes on reboot in KVM 65 on AMD

[Avi Kivity]

Joerg Roedel wrote:
> On Mon, Apr 14, 2008 at 12:43:30PM +0300, Avi Kivity wrote:
>   
>> Michal Ludvig wrote:
>>     
>>> Hi all,
>>>
>>> in short: CentOS 5 guest (32bit) crashes when I run "reboot" in the VM with qemu message:
>>> kvm_run: failed entry, reason 65535
>>> kvm_run returned -8
>>>
>>> In dmesg is this message:
>>> kvm: 23769: cpu0 kvm_set_msr_common: MSR_IA32_MC0_STATUS 0x0, nop
>>>
>>> Details:
>>> KVM 65
>>> Host is openSUSE 10.3 with my own 2.6.24, 64 bit
>>> Guest is CentOS 5.1 with 2.6.18-53.1.14.el5, 32 bit
>>> GCC 4.2.1 for module, GCC 3.3.3 for qemu
>>> CPU "AMD Athlon(tm) 64 X2 Dual Core Processor 5600+"
>>>
>>> Everything works fine, except that I can't reboot the guest. In VNC it says "md: stopping devices" and then Qemu exits with:
>>> kvm_run: failed entry, reason 65535
>>> kvm_run returned -8
>>>
>>> IIRC reboot used to work with kvm-64
>>>  
>>>       
>> I think two bugs are involved:
>>
>> - the guest invokes the triple-fault shutdown path, so it seems normal reboot doesn't work with iothread.  Haven't looked as to why yet.
>> - amd shutdown interception clears the vmcb, leaving asid == 0, which is illegal.  Adding force_new_asid() fixes this.  There are probably 
>> other issues with clearing the vmcb as well.
>>
>> Joerg, can you find out if the "vmcb is undefined" applies to the entire vmcb or just the guest saved state?  Is there a way to 
>> differentiate between a guest initiated shutdown due to triple fault and the core melting down?
>>     
>
> Only the state saved in the VMCB is undefined after an intercepted
> shutdown event. 

The vmcb stores two types of states, guest state and control 
information.  Are both undefined?

> I am not sure if I understand your second question, what
> do you mean with "core melting down" vs. guest initiated shutdown?
>   

IIRC shutdown can occur not only as a result of triple faults, due to 
other internal errors or inconsistent state.  Not sure about that though.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: bugreport: guest crashes on reboot in KVM 65 on AMD

[Joerg Roedel]

On Mon, Apr 14, 2008 at 01:29:50PM +0300, Avi Kivity wrote:
> Joerg Roedel wrote:
> >Only the state saved in the VMCB is undefined after an intercepted
> >shutdown event. 
> 
> The vmcb stores two types of states, guest state and control
> information.  Are both undefined?

Only the guest state is undefined.

> 
> >I am not sure if I understand your second question, what
> >do you mean with "core melting down" vs. guest initiated shutdown?
> >  
> 
> IIRC shutdown can occur not only as a result of triple faults, due to
> other internal errors or inconsistent state.  Not sure about that
> though.

No, there is no way to to differentiate between a shutdown caused by a
triple fault and other shutdown situations. If the shutdown event is
caused by a too hot processor it is not intercepted. This situation is
handled like a normal cpu shutdown.

-- 
           |           AMD Saxony Limited Liability Company & Co. KG
 Operating |         Wilschdorfer Landstr. 101, 01109 Dresden, Germany
 System    |                  Register Court Dresden: HRA 4896
 Research  |              General Partner authorized to represent:
 Center    |             AMD Saxony LLC (Wilmington, Delaware, US)
           | General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Re: bugreport: guest crashes on reboot in KVM 65 on AMD

[Avi Kivity]

Joerg Roedel wrote:
> On Mon, Apr 14, 2008 at 01:29:50PM +0300, Avi Kivity wrote:
>   
>> Joerg Roedel wrote:
>>     
>>> Only the state saved in the VMCB is undefined after an intercepted
>>> shutdown event. 
>>>       
>> The vmcb stores two types of states, guest state and control
>> information.  Are both undefined?
>>     
>
> Only the guest state is undefined.
>
>   

Okay, so we don't need to init_vmcb() there.  We can just reset the 
processor (actually, we should rely on userspace to reset the processor; 
IIRC shutdown does not reset the processor directly; instead the system 
board detects the shutdown and converts it to a reset, and we may want 
to emulate other types of system boards).

>>> I am not sure if I understand your second question, what
>>> do you mean with "core melting down" vs. guest initiated shutdown?
>>>  
>>>       
>> IIRC shutdown can occur not only as a result of triple faults, due to
>> other internal errors or inconsistent state.  Not sure about that
>> though.
>>     
>
> No, there is no way to to differentiate between a shutdown caused by a
> triple fault and other shutdown situations. If the shutdown event is
> caused by a too hot processor it is not intercepted. This situation is
> handled like a normal cpu shutdown.
>   

Well, that gives you some differentiation (enough for our needs).

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone