Re: Inter-domain Communication using Virtual Sockets (high-level design)

[David Vrabel <david.vrabel@citrix.com>]

On 17/06/13 19:28, Ross Philipson wrote:
> On 06/13/2013 12:27 PM, Tim Deegan wrote:
>> Hi,
>>
>> At 19:07 +0100 on 11 Jun (1370977636), David Vrabel wrote:
>>> This is a high-level design document for an inter-domain communication
>>> system under the virtual sockets API (AF_VSOCK) recently added to Linux.
>>
>> I'd be very interested to hear the v4v authors' opinions on this VSOCK
>> draft, btw -- in particular if it (or something similar) can provide all
>> v4v's features without new hypervisor code, I'd very much prefer it.
> 
> I guess I cannot be 100% just by reading the part of the spec on the low
> level transport mechanism. We originally tried to use a grant based
> model and ran into issue. Two of the most pronounced were:
> 
>  - Failure of grantees to release grants would cause hung domains under
> certain situations. This was discussed early in the V4V RFC work that
> Jean G. did. I am not sure if this has been fixed and if so, how. There
> was a suggestion about a fix in a reply from Daniel a while back.

The use of grants that only permit copying (i.e., no map/unmap) should
avoid any issues like these.  The granter can't revoke a copy-only grant
at any time.

>  - Synchronization between guests was very complicated without a central
> arbitrator like the hypervisor.

I'm not sure what you mean here.  What are you synchronizing?

> Also this solution may have some scaling issues. If I understand the
> model being proposed here, each ring which I guess is a connection
> consumes an event channel. In the large number of connections scenario
> is this not a scaling problem? I may not fully understand the proposed
> low level transport spec.

If there are N bits of work to do, N messages to resend for example,
then it doesn't matter if we have N notifications via event channels or
1 notification and some other data structure listing the N peers that
need work -- it's the same amount of work.

The number of event channels being a hard scalability limit will be
removed in Xen 4.4 (using one of the two proposals for an extended event
channel ABI).

David