Re: Xen-4.5 HVMOP ABI issues

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 28/11/14 13:07, Jan Beulich wrote:
>>>> On 28.11.14 at 12:36, <andrew.cooper3@citrix.com> wrote:
>> However, it occurs to me that HVMOP_op_mask absolutely must live in the
>> public header files, as it is guest visible, and forms part of the ABI.
>>
>> Consider a userspace task which falls into a continuation, is preempted
>> and paused by the guest kernel, the entire VM migrated, and the task
>> unpaused later.  If HVMOP_op_mask has changed in that time, the
>> continuation will become erroneous.
>>
>> In particular, all hypercall continuation strategies we use *must* be
>> forward compatible when moving to newer versions of Xen, because Xen
>> cannot possibly guarantee that a continuation which starts will finish
>> on the same hypervisor.
> Hmm, I see the issue, but surfacing such implementation details
> would not only be unfortunate, but eventually prevent us from
> making future changes. Just recall the mem-op case where we had
> to widen the continuation encoding mask at some point. Hence while
> I can't immediately see a way to avoid the situation you describe
> (and it doesn't even take a user space task in a preemptible kernel),
> I think we should allow ourselves a little more time to find possible
> solutions other than locking down these masks (really they don't
> need to be exposed in the public headers, but we would need
> them to not change if no other solution can be found).

It is certainly unfortunate, but I don't see that we have any choice. 
The implementation details of continuations have already slipped into
the ABI.

>
> One thing to note is that the _introduction_ of such a mask (as
> has happened for hvm-op between 4.4 and 4.5) is not a problem
> as long as the respective bits all being zero has no special
> meaning. What I considered for mem-op a while ago though (and
> then forgot again) was to refuse non-zero start_extent values
> for any operations not making use of that mechanism. The same
> would of course be applicable to gnttab-op and hvm-op.
>
> What might additionally make this not that urgent an issue for 4.5
> is that only XSM_DM_PRIV guarded operations are affected, and
> I suppose a stubdom gets re-created on the target host (rather
> than migrated) when its client migrates.

The problem is with continuations which reuse the upper bits of the
input register, not with this HVMOP_op_mask specifically; the
HVMOP_op_mask simply adds to an existing problem.  This is something
which needs considering urgently because, as you identify above, we have
already suffered an accidental ABI breakage with the mem-op widening.

32bit HVM guests reliably form a continuation on every single iteration
of a continuable hypercall (e.g. decrease reservation, which is the base
of my XSA-111 PoC), making it trivial to construct a migrateable HVM
domain which exposes the issue.

~Andrew