Re: Xen-4.5 HVMOP ABI issues

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 28/11/14 15:18, Jan Beulich wrote:
>>>> On 28.11.14 at 14:55, <andrew.cooper3@citrix.com> wrote:
>> The problem is with continuations which reuse the upper bits of the
>> input register, not with this HVMOP_op_mask specifically; the
>> HVMOP_op_mask simply adds to an existing problem.  This is something
>> which needs considering urgently because, as you identify above, we have
>> already suffered an accidental ABI breakage with the mem-op widening.
> Since we can't retroactively fix the mem-op widening, I still don't see
> what's urgent here: As long as we don't change any of these masks,
> nothing bad is going to happen. Of course one thing to consider with
> this aspect in mind is whether to change the hvm-op or gnttab-op
> masks again _before_ 4.5 goes out, based on whether we feel they're
> wide enough for the (un)foreseeable future.

By urgent, I mean exactly this, while we have the ability to tweak the
masks.

>
>> 32bit HVM guests reliably form a continuation on every single iteration
>> of a continuable hypercall (e.g. decrease reservation, which is the base
>> of my XSA-111 PoC), making it trivial to construct a migrateable HVM
>> domain which exposes the issue.
> Hmm, I can't offhand see why that would be, but what you write
> reads like you determined the reason?

I have never identified why, but it is reliable.  c/s 79de2d31f1 proves
that some preemption condition is true for 32bit HVM guests by the time
the hypercall handler is called.  It is unreliable with 64bit HVM
guests, suggesting that the compat translation layer may be the root
cause of the issue.

> I ask because an unavoidable
> use of continuations is certainly nice for making sure those code paths
> get tested, but would be quite desirable to get eliminated at least for
> non-debug builds.

While the preemption code is necessary to avoid spending ludicrous
quantities of time synchronously in Xen, it is currently having the
worst possible effect it could have on the system, by causing 32bit HVM
guests to undergo a vmentry/vmexit and double compat translation for
each iteration.

~Andrew