Re: [rfc] [patch] more 'long' in the hypervisor interface

[Steve Ofsthun <sofsthun@virtualiron.com>]

Chris Wright wrote:
> * Hollis Blanchard (hollisb@us.ibm.com) wrote:
> 
>>We discussed a bit on IRC (developers are welcome to join OFTC #xen),
>>but to recap for the list...
>>
>>PPC will have
>>	typedef uint64_t xen_ulong_t;
>>That means that the fields in memory.h will keep the same
>>size/alignment, whether compiled 32- or 64-bit. This is the way the
>>interface should have been designed in the first place, but we're locked
>>into the current ABI on x86. However, since PPC has no current users, we
>>can define the ABI correctly from the start.
> 
> 
> I see.  I think it would be nice to work on the ABI such that it makes
> sense for the future 32/64 mixed modes.  So I guess I actually agree
> with your legacy typedef name ;-)

X86 32/64 mixed modes really have 2 independent compatibility issues.  One
is the calling conventions used to pass parameters through the hypercall
interface.  The second is the format of the data structures passed through
the calling conventions to the underlying hypervisor.

Today, we run 32/64 mixed mode HVM guests on a 64 bit hypervisor.  The
hypercall interface was modified to handle both 32-bit and 64-bit calling
conventions.  The underlying hypervisor however only supports 64-bit
structure formats.  A 64-bit guest can continue to use the standard headers
for passing data to hypercalls.  A 32-bit guest must redefine every structure
in the public interfaces to properly pass data to the hypervisor.

We would like to see the 32-bit and 64-bit structure definitions evolve
to a single size invariant version of the interface structures for both
32-bit and 64-bit guests.

> One issue is that 32-bit userspace effectively has direct access to
> 64-bit hypercall interface.  This can be handled in the 64-bit kernel by
> doing compat translation, by having 32-bit compat hypercall interface
> and jumping to right spot on hypercall page, or by having fixed size
> structure.  It's not clear to me the value of effectively exposing the
> ABI all the way to userspace.

I'm not sure I understand your use of the term 'userspace' here.  Do you
mean guest kernel mode, or actual unprivileged user code?

> What is the current plan for 32-bit kernel on 64-bit hv?  In this case
> a 32-bit compat hypercall page might be useful, or having fixed size
> structure.

For X86 there are probably two plans.  For paravirtual guests, there is a
strong desire to formalize the existing ABI.  This will force the 32-bit
and 64-bit ABIs to remain significantly different.  Since the underlying
hypervisors don't allow 32/64 mixed mode guests, there is little reason
to reconcile the two ABIs.  If the ABIs were identical today, you still
couldn't run mixed mode guests.

For HVM guests, the ABI is less established.  I'm not sure anyone but us
(Virtual Iron), is doing much with hypercalls from HVM guests.  We are
currently running paravirtualized drivers in HVM guests.  As the code
matures, we will be posting these patches.

We have had to deal with issues separate from the mechanical ABI issues.
For example, grant table transfers (used by the standard netfront/netback)
don't play well with QEMU's one time direct map of the entire HVM guest
address space.  In addition, the xen support needed by PV drivers is
specific to later 2.6 kernels.  Getting this code to work on older linux
kernels requires some additional work.

> My concern is that we'll never make a clean break if we slowly cobble up
> the interface with more hacks.  Maybe a forward looking compat interface
> would be a good breaking point.

I agree with you on this.  The longer this goes unaddressed, the more work it
will be to fix.

Steve
-- 
Steve Ofsthun - Virtual Iron Software, Inc.