Re: Re: Interdomain comms

[Harry Butterworth <harry@hebutterworth.freeserve.co.uk>]

On Sat, 2005-05-07 at 10:15 -0600, Ronald G. Minnich wrote:
> 
> On Sat, 7 May 2005, Harry Butterworth wrote:
> > A significant difference between Plan 9 and Xen which is relevant to
> > this discussion is that Plan 9 is designed to construct a single shared
> > environment from multiple physical machines whereas Xen is designed to
> > partition a single physical machine into multiple isolated environments.
> 
> this is not in my view germane. We're proposing using 9p as the low level 
> interdomain protocol for xen. 9p stands alone from plan 9. Both Eric and I 
> have felt for some time that this would work well, and I'm pretty strongly 
> convinced that it would work better than what we have now for the various 
> devices. 

I agree it would work well, definitely better than the current code, but
I don't think it's right for Xen because it precludes the kind of
optimisations that are relevant to Xen but not Plan 9.

Eric mentioned adding scatter/gather type semantics to the 9P protocol
which would get you half-way to what I was proposing and start to look
similar to the remote buffer reference concept in that you pass around
references to the bulk data instead of the data itself but you still
wouldn't get the ability to manipulate the meta-data of a request in a
stack of I/O applications and then do a direct transfer of the bulk data
from the source to the target.

Of course, if you modify the 9P protocol enough then you can make it a
superset of what I was proposing but I think my API proposal captured
the essence of what was required at the low level without complicating
the description with the higher level organisation aspects of the
protocol.

> 
> Your 1000 machine example is very interesting, however. I do know that 
> some folks who worked on the IBM hypervisor had a lot of concerns about 
> the page flipping done in the xen network FE/BE drivers, and I wonder if 
> their concerns would apply to your proposed implementation -- I wish we 
> could find a way to hear from them. 

Rusty and I exchanged some mail about the page flipping and I think
Rusty demonstrated that their concerns were unfounded.  Perhaps he can
comment.

In any case, my proposal decouples the IDC API clients from the actual
memory management implementation so different page-flipping behaviours
could be evaluated without requiring code changes in the clients.

Harry.