Re: Re: Interdomain comms

[Andrew Warfield <andrew.warfield@gmail.com>]

> It should be possible to still use the page mapping in the i/o transport.
> The issue right now is that the i/o interface is very low-level and
> intimately tangled up with the structs being transported.

I don't doubt that it is possible.  The point I was making is that the
current i/o interfaces are low level for a reason, and that
generalizing this to a higher-level communications primitive is a
non-trivial thing.  Just considering the disk and net interfaces, the
current device channels each make particular decisions regarding (a)
what to copy and what to map, (b) when to send notification to get
efficient batching through the scheduler, and most recently (c) which
grant mechanism to use to pass pages securely across domains.

Having a higher-level API to make all this easier, and especially to
reduce the code/complexity required to build new drivers etc is
something that will be fantastic to have.  I think though that at
least some of these underlying issues will need to be exposed for it
to be useful.  I'm not convinced that reimplementing the sockets API
for interdomain communication is a very good solution... The
buffer_reference struct that Harry mentioned looks quite interesting
as a start though in terms of describing a variety of underlying
transports.  Do you have a paper reference on that work, Harry?

With regards forwarding device channels across a network, I think we
can expect application-level involvement for shifting device messages
across a cluster.  If this is down the road, and it's certainly
something that has been discussed, a device channel is potentially two
local shared memory device channels between VMs on local hosts, and a
network connection between the physical hosts.  Beyond the more
complicated error cases that this obviously involves, we can then make
this as arbitrarily more complex by discussing HA or security
concerns... for the moment though, I think it would be interesting to
see how well the existing local host cases can be generalized.  ;)
 
> And with the domain control channel there's an implicit assumption
> that 'there can be only one'. This means for example, that domain A
> using a device with backend in domain B can't connect directly to domain B,
> but has to be 'introduced' by xend. It'd be better if it could connect
> directly.

This is not a flaw with the current implementation -- it's completely
intentional.  By forcing control through xend we ensure that there is
a single point for control logic, and for managing state.  Why do you
feel it would be better to provide arbitrary point-to-point comms in a
VMM environment that is specifically trying to provide isolation
between guests?

> Something like what Harry proposes should still be able to use
> page mapping for efficient local comms, but without _requiring_
> it. This opens the way for alternative transports, such as network.
> 
> Rather than going straight for something very high-level, I'd prefer
> to build up gradually, starting with a more general message transport
> api that includes analogues to listen/connect/recv/send.

As I said, I'm unconvinced that trying to mimic the sockets API is the
right way to go -- I think the communicating parties often want to see
and work with batches of messages without having to do extra copies or
have event notification made implicit.  I think you are completely
right about a gradual approach though -- having a generalized
host-local device channel would be very interesting to see...
especially if it could be shown to apply to the existing block, net,
usb, and control channels in a simplifying fashion.

a.