Re: making xenstore domain easy configurable

[Juergen Gross <jgross@suse.com>]

On 28/06/16 15:59, Andrew Cooper wrote:
> On 28/06/16 14:36, Juergen Gross wrote:
>> On 28/06/16 14:42, Andrew Cooper wrote:
>>> On 28/06/16 12:56, Juergen Gross wrote:
>>>> On 28/06/16 13:03, Ian Jackson wrote:
>>>>> Juergen Gross writes ("Re: [Xen-devel] making xenstore domain easy configurable"):
>>>>>> So you are telling me the xenstore domain won't work for this case?
>>>>> Yes.
>>>> That's rather unfortunate. So in order to be able to make xenstore
>>>> domain a common setup we need to find a solution for support of
>>>> xs_restrict() via xenbus, right?
>>>>
>>>> TBH, the way xs_restrict() was introduced is rather weird. It is
>>>> completely bound to the socket interface of oxenstored. So anyone
>>>> wanting to use xs_restrict() is limited to oxenstored running in
>>>> dom0. No way to use xenstored or a xenstore domain. I'm really
>>>> disappointed such a design was accepted and is now the reason for
>>>> not being able to disaggregate dom0.
>>>>
>>>> I've searched through the xen-devel archives and found a very
>>>> interesting mail:
>>>>
>>>> http://lists.xen.org/archives/html/xen-devel/2010-04/msg01318.html
>>>>
>>>> The "restrict" feature was added without any further discussion how
>>>> it is implemented and that the C-variant doesn't support it. The
>>>> explicit question about non-existing features in the C xenstored was
>>>> answered just with "the xenstore wire protocol doesn't change".
>>>>
>>>> With:
>>>>
>>>> http://lists.xen.org/archives/html/xen-devel/2010-07/msg00091.html
>>>>
>>>> the XS_RESTRICT value in xs_wire.h (aah, suddenly it was changed?)
>>>> was added. Again no mentioning of the special implementation in
>>>> oxenstored.
>>>>
>>>> Really, this is not how open source development should be done!
>>>> Maybe I'm just upset now, but I'm in favor of dropping xs_restrict()
>>>> support as it has been introduced in a foul way.
>>> I don't think the lack of xs_restrict() working over the ring should
>>> preclude these improvements to the configuration of how xenstored starts up.
>> It is limiting the solution by not allowing me to drop the sockets
>> completely.
> 
> I don't think dropping the sockets completely is a sensible course of
> action.  I had come the conclusion that you were just not going to use
> them, as opposed to removing them entirely.

If they are not going to be used they can be dropped, no?

Again: the main problem with the sockets is their systemd definition in
combination of their existence being used for the connection type with
xenstore (socket vs. kernel).

So either I always connect via the kernel making the sockets useless
(then I can remove them completely) or I have a way creating the
sockets only in case of the daemon case which is currently available
only by removing the systemd definition of the sockets.

> For xenstored running in the same domain as the toolstack, sockets are
> less overhead than the shared memory ring, as no hypercalls are
> involved.  There is also the unfortunate problem that one of the two
> linux devices for xenstored *still* causes deadlocks when used; a
> problem which is unresolved from Linux 3.14.

So this would mean we should keep the sockets and just remove their
systemd definition.

>>> Longterm, a sensible solution would be to make a xenstore protocol
>>> extension to wrap an existing xenstore message in a restrict wrapper,
>>> where the kernel device can apply the appropriate restrict around user
>>> requests.
>> I'd rather let xs_restrict() work in a clean way: setup a new ring and
>> event channel with the appropriate privileges and let the connection
>> to xenstore run via this ring.
> 
> Domains currently don't have any notion of multiple xenstore-rings to
> the same domain.  Somewhere (i.e. in xenstored itself) there would have
> to be some kind of hard upper limit to avoid resource exhaustion, and
> guest kernels would still have to have some privileged way of
> negotiating the setup.  Also, how do you plan to make any of this work
> with xenstored not in a subdomain?

The multiple ring setup should be allowed for dom0 (or similar
privileged domains) only in order to avoid resource exhaustion.
xs_restrict() is limited to dom0, too.

I haven't worked on a design for the implementation yet. I've just
presented a rough interface idea making it rather easy to distinguish
multiple instances by xenstore regardless whether those instances are
located in the same domain or not.


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel