(xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

(xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Fanhenglong]

[-- Attachment #1.1: Type: text/plain, Size: 2048 bytes --]

Hi everyone,


I want to debug the procedure of windows os install with windbg,
windbg executes instruction(fxsave) after the blank vm is started and before guest iso start to install,


fxsave trigger the following code path:
vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
->ept_handle_violation
->hvm_hap_nested_page_fault
->handle_mmio_with_translation
->handle_mmio
->hvm_emulate_one
->x86_emulate

X86_emulate return X86EMUL_UNHANDLEABLE

The xl dmesg log;
(d5) Writing SMBIOS tables ...
(d5) Loading OVMF ...
(XEN) d5v0 Over-allocation for domain 5: 2097409 > 2097408
(XEN) memory.c:155:d5v0 Could not allocate order=0 extent: id=5 memflags=0 (0 of 1)
(d5) Loading ACPI ...
(d5) vm86 TSS at fc012d00
(d5) BIOS map:
(d5)  ffe00000-ffffffff: Main BIOS
(d5) E820 table:
(d5)  [00]: 00000000:00000000 - 00000000:000a0000: RAM
(d5)  HOLE: 00000000:000a0000 - 00000000:000f0000
(d5)  [01]: 00000000:000f0000 - 00000000:00100000: RESERVED
(d5)  [02]: 00000000:00100000 - 00000000:f0000000: RAM
(d5)  HOLE: 00000000:f0000000 - 00000000:fc000000
(d5)  [03]: 00000000:fc000000 - 00000001:00000000: RESERVED
(d5)  [04]: 00000001:00000000 - 00000002:0f6ed000: RAM
(d5) Invoking OVMF ...
(XEN) stdvga.c:147:d5v0 entering stdvga and caching modes
(XEN) stdvga.c:151:d5v0 leaving stdvga
(XEN) irq.c:276: Dom5 PCI link 0 changed 5 -> 11
(XEN) irq.c:276: Dom5 PCI link 1 changed 10 -> 11
(XEN) irq.c:276: Dom5 PCI link 2 changed 11 -> 10
(XEN) irq.c:276: Dom5 PCI link 3 changed 5 -> 10
(XEN) MMIO emulation failed: d5v0 64bit @ 0028:efe54dab -> 0f ae 07 fc ff 75 10 48 8b 4d 08 48 89 e2 48 83
(XEN) MMIO emulation failed: d5v0 64bit @ 0028:efe54dab -> 0f ae 07 fc ff 75 10 48 8b 4d 08 48 89 e2 48 83
(XEN) MMIO emulation failed: d5v0 64bit @ 0028:efe54dab -> 0f ae 07 fc ff 75 10 48 8b 4d 08 48 89 e2 48 83
(XEN) d5v0 Triple fault - invoking HVM shutdown action 1



*
Now I can't use windbg tool,
Can anyone give me some advice to solve the problem?




Thanks





















[-- Attachment #1.2: Type: text/html, Size: 10136 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Razvan Cojocaru]

On 06/24/2015 12:14 PM, Fanhenglong wrote:
> I want to debug the procedure of windows os install with windbg,
> 
> windbg executes instruction(fxsave) after the blank vm is started and
> before guest iso start to install,
> 
> fxsave trigger the following code path:
> vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
> ->ept_handle_violation
> ->hvm_hap_nested_page_fault
> ->handle_mmio_with_translation
> ->handle_mmio
> ->hvm_emulate_one
> ->x86_emulate
> 
> *X86_emulate return X86EMUL_UNHANDLEABLE*

How are you using Xen in this case? Are you by any chance using the
vm_event system in a way that sends back an "emulate" vm_event response
from userspace?

You might want to look at x86_emulate() in
xen/arch/x86/x86_emulate/x86_emulate.c and see if (and how) fxsave is
being handled.


HTH,
Razvan

Re: (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Andrew Cooper]

On 24/06/15 10:25, Razvan Cojocaru wrote:
> On 06/24/2015 12:14 PM, Fanhenglong wrote:
>> I want to debug the procedure of windows os install with windbg,
>>
>> windbg executes instruction(fxsave) after the blank vm is started and
>> before guest iso start to install,
>>
>> fxsave trigger the following code path:
>> vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
>> ->ept_handle_violation
>> ->hvm_hap_nested_page_fault
>> ->handle_mmio_with_translation
>> ->handle_mmio
>> ->hvm_emulate_one
>> ->x86_emulate
>>
>> *X86_emulate return X86EMUL_UNHANDLEABLE*
> How are you using Xen in this case? Are you by any chance using the
> vm_event system in a way that sends back an "emulate" vm_event response
> from userspace?
>
> You might want to look at x86_emulate() in
> xen/arch/x86/x86_emulate/x86_emulate.c and see if (and how) fxsave is
> being handled.

The fxsave instruction has no emulation implementation.

"0f ae 07" is "fxsave (%rdi)" which means that either introspection is
active, or %rdi is a pointer into an MMIO region.

~Andrew

Re: (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Razvan Cojocaru]

On 06/24/2015 12:31 PM, Andrew Cooper wrote:
> On 24/06/15 10:25, Razvan Cojocaru wrote:
>> On 06/24/2015 12:14 PM, Fanhenglong wrote:
>>> I want to debug the procedure of windows os install with windbg,
>>>
>>> windbg executes instruction(fxsave) after the blank vm is started and
>>> before guest iso start to install,
>>>
>>> fxsave trigger the following code path:
>>> vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
>>> ->ept_handle_violation
>>> ->hvm_hap_nested_page_fault
>>> ->handle_mmio_with_translation
>>> ->handle_mmio
>>> ->hvm_emulate_one
>>> ->x86_emulate
>>>
>>> *X86_emulate return X86EMUL_UNHANDLEABLE*
>> How are you using Xen in this case? Are you by any chance using the
>> vm_event system in a way that sends back an "emulate" vm_event response
>> from userspace?
>>
>> You might want to look at x86_emulate() in
>> xen/arch/x86/x86_emulate/x86_emulate.c and see if (and how) fxsave is
>> being handled.
> 
> The fxsave instruction has no emulation implementation.
> 
> "0f ae 07" is "fxsave (%rdi)" which means that either introspection is
> active, or %rdi is a pointer into an MMIO region.

I see, these are the cases we wanted to treat with the old patch (I
thick it was called "xen: Handle resumed instruction based on previous
mem_event reply" - the early versions, with RFC) that sometimes bypassed
the emulator in the introspection case. Without that, there's always
going to be a potential current or future instruction not emulated, and
then something like this happens.


Cheers,
Razvan

Re: (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Wei Liu]

On Wed, Jun 24, 2015 at 10:31:57AM +0100, Andrew Cooper wrote:
> On 24/06/15 10:25, Razvan Cojocaru wrote:
> > On 06/24/2015 12:14 PM, Fanhenglong wrote:
> >> I want to debug the procedure of windows os install with windbg,
> >>
> >> windbg executes instruction(fxsave) after the blank vm is started and
> >> before guest iso start to install,
> >>
> >> fxsave trigger the following code path:
> >> vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
> >> ->ept_handle_violation
> >> ->hvm_hap_nested_page_fault
> >> ->handle_mmio_with_translation
> >> ->handle_mmio
> >> ->hvm_emulate_one
> >> ->x86_emulate
> >>
> >> *X86_emulate return X86EMUL_UNHANDLEABLE*
> > How are you using Xen in this case? Are you by any chance using the
> > vm_event system in a way that sends back an "emulate" vm_event response
> > from userspace?
> >
> > You might want to look at x86_emulate() in
> > xen/arch/x86/x86_emulate/x86_emulate.c and see if (and how) fxsave is
> > being handled.
> 
> The fxsave instruction has no emulation implementation.
> 
> "0f ae 07" is "fxsave (%rdi)" which means that either introspection is
> active, or %rdi is a pointer into an MMIO region.
> 

So I think this is not a regression? (I'm now trying to identify
possible blockers for the release)

Wei.

> ~Andrew
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

答复: (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Fanhenglong]

Windows 8.0 iso can't install in uefi mode,
I have to use windbg to get more debug information, 
So using xen in this case


发件人: Razvan Cojocaru [mailto:rcojocaru@bitdefender.com] 
发送时间: 2015年6月24日 17:26
收件人: Fanhenglong; xen-devel@lists.xen.org
抄送: Liuqiming (John); Yanqiangjun; Huangpeng (Peter); Hanweidong (Randy)
主题: Re: [Xen-devel] (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

On 06/24/2015 12:14 PM, Fanhenglong wrote:
> I want to debug the procedure of windows os install with windbg,
> 
> windbg executes instruction(fxsave) after the blank vm is started and 
> before guest iso start to install,
> 
> fxsave trigger the following code path:
> vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
> ->ept_handle_violation
> ->hvm_hap_nested_page_fault
> ->handle_mmio_with_translation
> ->handle_mmio
> ->hvm_emulate_one
> ->x86_emulate
> 
> *X86_emulate return X86EMUL_UNHANDLEABLE*

How are you using Xen in this case? Are you by any chance using the vm_event system in a way that sends back an "emulate" vm_event response from userspace?

You might want to look at x86_emulate() in xen/arch/x86/x86_emulate/x86_emulate.c and see if (and how) fxsave is being handled.


HTH,
Razvan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: (xen 4.6 unstable) triple fault when execute fxsave during the procedure of guest iso install

[Jan Beulich]

>>> On 24.06.15 at 11:14, <fanhenglong@huawei.com> wrote:
> I want to debug the procedure of windows os install with windbg,
> windbg executes instruction(fxsave) after the blank vm is started and before 
> guest iso start to install,
> 
> 
> fxsave trigger the following code path:
> vmx_vmexit_handler(EXIT_REASON_EPT_VIOLATION)
> ->ept_handle_violation
> ->hvm_hap_nested_page_fault
> ->handle_mmio_with_translation
> ->handle_mmio
> ->hvm_emulate_one
> ->x86_emulate
> 
> X86_emulate return X86EMUL_UNHANDLEABLE
> 
> The xl dmesg log;
> (d5) Writing SMBIOS tables ...
> (d5) Loading OVMF ...
> (XEN) d5v0 Over-allocation for domain 5: 2097409 > 2097408
> (XEN) memory.c:155:d5v0 Could not allocate order=0 extent: id=5 memflags=0 
> (0 of 1)
> (d5) Loading ACPI ...
> (d5) vm86 TSS at fc012d00
> (d5) BIOS map:
> (d5)  ffe00000-ffffffff: Main BIOS
> (d5) E820 table:
> (d5)  [00]: 00000000:00000000 - 00000000:000a0000: RAM
> (d5)  HOLE: 00000000:000a0000 - 00000000:000f0000
> (d5)  [01]: 00000000:000f0000 - 00000000:00100000: RESERVED
> (d5)  [02]: 00000000:00100000 - 00000000:f0000000: RAM
> (d5)  HOLE: 00000000:f0000000 - 00000000:fc000000
> (d5)  [03]: 00000000:fc000000 - 00000001:00000000: RESERVED
> (d5)  [04]: 00000001:00000000 - 00000002:0f6ed000: RAM
> (d5) Invoking OVMF ...
> (XEN) stdvga.c:147:d5v0 entering stdvga and caching modes
> (XEN) stdvga.c:151:d5v0 leaving stdvga
> (XEN) irq.c:276: Dom5 PCI link 0 changed 5 -> 11
> (XEN) irq.c:276: Dom5 PCI link 1 changed 10 -> 11
> (XEN) irq.c:276: Dom5 PCI link 2 changed 11 -> 10
> (XEN) irq.c:276: Dom5 PCI link 3 changed 5 -> 10
> (XEN) MMIO emulation failed: d5v0 64bit @ 0028:efe54dab -> 0f ae 07 fc ff 75 10 48 8b 4d 08 48 89 e2 48 83
> (XEN) MMIO emulation failed: d5v0 64bit @ 0028:efe54dab -> 0f ae 07 fc ff 75 10 48 8b 4d 08 48 89 e2 48 83
> (XEN) MMIO emulation failed: d5v0 64bit @ 0028:efe54dab -> 0f ae 07 fc ff 75 10 48 8b 4d 08 48 89 e2 48 83
> (XEN) d5v0 Triple fault - invoking HVM shutdown action 1

Considering the address (below 4Gb) I'd view it equally possible
that it's OVMF that is running into this (and Windows may not
have got control at all by that time). But as others have said -
unless you're using VM events, it first of all would need to be
understood why fxsave would be issued on MMIO space, which
as a very minimum requires register state to be made visible.

Jan