Re: Essay on an important Xen decision (long)

[Harry Butterworth <harry@hebutterworth.freeserve.co.uk>]

[-- Attachment #1: Type: text/plain, Size: 4977 bytes --]

On Tue, 2006-01-10 at 11:26 -0800, Magenheimer, Dan (HP Labs Fort
Collins) wrote:
> A fundamental architectural decision has to be made for
> Xen regarding handling of physical/machine memory; at a high
> level, the question is:
> 
> 	Should Xen drivers be made more flexible to accommodate
> 	different approaches to managing physical memory, or
> 	should other architectures be required to conform to
> 	the Xen/x86 model?

I believe the right approach is to decouple the driver implementation
from the memory management architecture by defining a high level API to
build the drivers on.  The API should be expressed in terms of the
operations that the drivers need to perform rather than in terms of the
underlying primitives that are actually used to perform those
operations.

Such an API would allow decisions about memory management to be made
independent of the drivers and would allow the memory management
architecture to be changed relatively easily at a later date since the
resulting damage would be contained within the core library that
implemented the driver infrastructure API.

I think this is the right approach because:

o - Decoupling the drivers from the memory management architecture
reduces the cost of future memory management architecture changes and
keeps our options open, so is a lower risk approach than choosing a
memory management architecture now and trying to stick with it.

o - A good high level driver infrastructure API will clean up the
drivers considerably.

o - Containing the code which performs low-level memory manipulations
within a core driver infrastructure library written by an expert will
result in higher overall quality across all the drivers.

o - As a driver author, given a high level driver infrastructure API
which decouples me from the memory management architecture, the choice
of P==M, P2M or VP is no longer my concern.

I have made a first attempt at defining a high level driver
infrastructure API for general use by xen split drivers.  This is the
xenidc API and, whilst it is designed for general use, it currently has
one client: the split USB driver.

I believe that xenidc completely decouples its clients from the memory
management architecture such that, for example, there should be no
changes required in the USB driver code when porting it from x86 to ia64
and PPC (this will be true whether or not the memory management
architecture for those platforms is changed to be more like x86).

All required changes ought to be contained within the xenidc
implementation and therefore would only need to be implemented once for
all clients of xenidc.

The choice of a common memory management architecture or different
memory management architectures across platforms or different options
for memory management architectures for a particular platform or
different options for memory management architecture at run-time for
transparent virtualization can all be contained within the xenidc
implementation.

In addition to decoupling the client driver code from the memory
management architecture, the xenidc API provides:

o - Convenient inter-domain communication primitives which encapsulate
the rather complicated state machine required for correct set-up and
tear down of inter-domain communication channels for (un)loadable driver
modules.

o - A convenient inter-domain bulk transport.

o - An up-front-reservation resource management strategy.

o - Driver forwards-compatibility with a network transparent xenidc
implementation.

I have attached the latest xenidc patch which includes documentation of
the xenidc API (added by the patch to the Xen interface document).

I have also attached the latest USB patch as an example of a client of
the xenidc API.

(Since the last time I posted these patches I have fixed a couple of
compiler warnings for the X86_64 build).

A few points to note:

o - xenidc is an infrastructure for the Xen-specific split drivers.
Xenidc doesn't directly address the issue of making the native drivers
work correctly under virtualization but does allow you to do that
however you like across different architectures whilst maintaining
common code for all the split drivers.

o - This is just a first attempt which I wrote mainly to decouple the
USB driver from churn in the underlying infrastructure.  The API is
generally useful but only covers the operations that were actually
required for the USB driver.  There is already enough in the API to base
other drivers on it but the API would need to be fleshed out with some
different kinds of operations before it would be possible to implement
all drivers with the same efficient primitives that are used today.

o - Unfortunately I didn't get funding to attend the Xen summit so I
won't be there to present on Xenidc.  I'm not concerned about whether
xenidc gets accepted as-is but I do hope it will be useful as an example
of the kind of API that we could have.  I'll be happy to answer any
questions on the list.

Harry.

[-- Attachment #2: latest-xenidc-patch.gz --]
[-- Type: application/x-gzip, Size: 62319 bytes --]

[-- Attachment #3: latest-usb-patch.gz --]
[-- Type: application/x-gzip, Size: 34642 bytes --]

[-- Attachment #4: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel