Re: [Xen-users] xc_hvm_inject_trap() failing for int3 traps under Xen 4.2.2

[Antony Saba <Antony.Saba@mandiant.com>]

On 06/07/2013 09:51 AM, Steven Maresca wrote:
> On Thu, Jun 6, 2013 at 8:16 PM, AP <apxeng@gmail.com> wrote:
>> (Adding xen-devel)
>>
>> On Tue, Jun 4, 2013 at 2:34 PM, Antony Saba <Antony.Saba@mandiant.com> wrote:
>>>
>>> Hello,
>>>
>>> Can anyone verify if reinjecting int3 mem_events work for them under Xen
>>> 4.2.2?
>>>
>>> I'm trying to move some systems which are making use of int3 mem_events
>>> from Xen 4.1.x to Xen 4.2.2, but seem to having an issue with
>>> xc_hvm_inject_trap().
>>
>> Can you try with the following patch?
>>
>> --- a/tools/tests/xen-access/xen-access.c
>> +++ b/tools/tests/xen-access/xen-access.c
>> @@ -664,7 +664,7 @@ int main(int argc, char *argv[])
>>                  /* Reinject */
>>                  rc = xc_hvm_inject_trap(
>>                      xch, domain_id, req.vcpu_id, 3,
>> -                    HVMOP_TRAP_sw_exc, -1, 0, 0);
>> +                    HVMOP_TRAP_sw_exc, -1, 1, 0);
>>                  if (rc < 0)
>>                  {
>>                      ERROR("Error %d injecting int3\n", rc);
>>
>> BTW, I don't think you need to specify the instruction length for int3
>> as the IP should have been moved forward. But it might give us a clue
>> as to what is going on.

The same behavior occurs, rc < 0 and errno set.

>>
>>> I've got a simple program that doesn't do anything but "_asm int 3" in
>>> it's main.  Using the example in "tools/text/xen-access/", this is what
>>> the output of "xen-access 14 int3" looks like.  The domain is frozen
>>> after xen-access exits.
>>>   xenaccess init
>>>   max_pages = 40100
>>>   starting int3 14
>>>   Got event from Xen
>>>   Got event from Xen
>>>   INT3: rip=0000000000401000, gfn=1418f (vcpu 0)
>>>   xc: error: Error -1 injecting int3: Internal error
>>>   xenaccess shutting down on signal -1
>>>   xenaccess shut down on signal -1
>>>   xenaccess exit code -1
>>
>> If you set access required, then this is the expected behavior
>> otherwise the domain should continue running.

The domain is frozen whether or not I've passed -m to the xen-access binary.

>>
>>> This shows up in xl dmesg (nothing else shows up with debug=y):
>>> (XEN) d14v0: bogus time -341046118 (offsets -3367129229295/0)
>>>
>>> The same behavior occurs with both 32-bit and 64-bit HVM guests.
>>>
>>> I'm using Xen 4.2.2 built from the dist tarball.
>>>
>>> dom0 is Ubuntu 12.04.2 using kernel 3.2.0-45-generic #70-Ubuntu SMP Wed
>>> May 29 20:12:06 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> My CPU is an Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
>>>
>>> I've also tried on 2nd machine with the same Ubuntu/kernel versions, but
>>> with the following CPU: Intel(R) Core(TM) i7-2720QM CPU @ 2.20GHz
>>>
>>> On 4.1.2, the result of xc_hvm_inject_trap() is always non-zero and
>>> errno is set to ENOENT, but the int3 is reinjected anyway and works as

> 
> Tony,
> 
> I can confirm INT3 re-injection does work on 4.2.x and 4.3, but the
> problem you observed is certainly present.
> 
> As suggested, it was necessary when invoking xc_hvm_inject_trap to
> specify the 1-byte instruction length for 0xCC (without which the VM
> was intentionally crashed by Xen).

Based on this, I went a ahead and made the change in our own
application, which now appears to reinject correctly, despite returning
the error "No such file or directory".  There is a whole lot more than
int3 events being handled in that application, so it's definitely doing
something that the xen-access example is not.

> 
> In this case, there's no need to inspect the actual instruction
> referenced by the IP because it seems the trap is only fired for the
> one-byte variant (0xCD03 of course works properly, but no event is
> emitted).
> 
> Mirroring your experience with 4.1.2, for my testing on 4.2+ the
> return of xc_hvm_inject_trap is also always non-zero even for
> successful re-injection..whether that's intended is another question.
> 
Ignoring the error in xen-accesss.c, the domain still remains frozen
after the call to resume_page(), etc.

> Steve
> 
> NOTE: I would definitely consider it a bug that the xen-access.c
> example crashes guests when attempting to use the INT3
> mode...non-critical for most users, but nevertheless.
> 
If this does end up getting filed as a bug, I saw the same issues under
4.3-RC3.

Thanks.

-Tony

-- 
Antony Saba, antony.saba@mandiant.com