Re: SSE instruction emulation issues

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 15/07/15 15:13, Paul Durrant wrote:
>> -----Original Message-----
>> From: Fabio Fantoni [mailto:fabio.fantoni@m2r.biz]
>> Sent: 15 July 2015 14:56
>> To: Jan Beulich
>> Cc: Andrew Cooper; Paul Durrant; Zhi Wang; xen-devel
>> Subject: Re: SSE instruction emulation issues
>>
>> Il 15/07/2015 13:35, Jan Beulich ha scritto:
>>>>>> On 15.07.15 at 13:13, <fabio.fantoni@m2r.biz> wrote:
>>>> Il 10/07/2015 14:16, Jan Beulich ha scritto:
>>>>>>>> On 10.07.15 at 14:00, <andrew.cooper3@citrix.com> wrote:
>>>>>> On 09/07/15 20:32, Zhi Wang wrote:
>>>>>>>       We found that MOVD instruction are used by some windows driver
>>>>>>> during developing XenGT, and also we found this one:
>>>>>>>
>>>>>>> (XEN) MMIO emulation failed: d7v1 64bit @ 0010:fffff8000294e273 ->
>> 66
>>>>>>> 0f e7 00 48 83 c0 10 45 3
>>>>>>> b cb 73 f0 45 85 c9
>>>>>> Disassembly:
>>>>>>      0:    66 0f e7 00              movntdq %xmm0,(%rax)
>>>>>>      4:    48 83 c0 10              add    $0x10,%rax
>>>>>>      8:    45 3b cb                 cmp    %r11d,%r9d
>>>>>>      b:    73 f0                    jae    0xfffffffffffffffd
>>>>>>      d:    45 85 c9                 test   %r9d,%r9d
>>>>>>
>>>>>> The x86 instruction emulator does appear to have a decode for this
>>>>>> instruction.  This failure suggests that the implementation is buggy.
>>>>>>
>>>>>> To start with diagnosing, add a test case to
>>>>>> tools/tests/x86_emulator/test_x86_emulator.c
>>>>> Considering that we already test MOVDQU, the emulation of which
>>>>> shares code with MOVNTDQ (which only differs in aspects not of
>>>>> interest to the emulator) I'm not sure this will turn up anything
>>>>> interesting. Perhaps an even easier step would be to simply run
>>>>> the emulator test on the machine where the issue is seen? We're
>>>>> playing some prefix byte tricks there... Otoh failure to execute
>>>>> the constructed instruction would bring down the hypervisor.
>>>> I also have a problem with mmio as I already reported many times but I
>>> And to be honest, I don't see the value in re-stating this every once
>>> in a while without providing any new information.
>>>
>>>> don't know if it is the same as the one reported by the intel developer
>>>> about xengt, I have it in linux hvm domUs with qxl.
>>> Looks different - their's was about MOVD (which we clearly don't
>>> support right now) while yours looks to be about MOVAPS.
>>>
>>>> Today with the latest xen update from git staging (with the addiction of
>>>> the xengt patch that add support of emulating SSE2 instruction MOVD) I
>>>> had a different domU's Xorg backtrace containing also a "error: Cannot
>>>> access memory at address":
>>> Sadly a gdb backtrace is nothing I can see use extract useful information
>>> from. Iirc Paul had already asked you to instrument the involved code
>>> paths (considering that the x86 insn emulator supports MOVAPS as used
>>> by the failing code) to figure out where in the whole involved stack the
>>> failure actually originates.
>>>
>>> Jan
>>>
>> Thanks for your reply, as I wrote the other times I don't know a better
>> debug method about particular things like this (x86 instructions
>> emulation) and I'm asking what I should do.
> Ok. I suggest you start with the handle_mmio() function in Xen (xen/arch/x86/hvm/io.c). This is where the code you're interested in is entered. Indeed the reason you see:
>
> (XEN) MMIO emulation failed: d7v1 64bit @ 0010:fffff8000294e273 -> 66
>
> is because handle_mmio() has called hvm_dump_emulation_state() because its call to hvm_emulate_one() has returned HVMEMUL_UNHANDLEABLE. The question is why?

For this, I can help somewhat.  66 is a prefix (specifically, operand
size override).  The lack of any further bytes printed by
hvm_dump_emulation_state() indicates that Xen failed to fetch them, when
obeying instruction fetch semantics while walking the pagetables.

As the instruction pointer is not on a page boundary, the most likely
reason for this in a plain VM is that some other vcpu has modifying the
paging structure under the feet of the emulator.

First of all, start debugging with a single vcpu domain.  Then you want
to find the conditions under which ops->insn_fetch() fails in
insn_fetch_bytes() in x86_emulate.c

~Andrew