xenfs aka /proc/xen

xenfs aka /proc/xen

[Bastian Blank]

Hi

The current tree includes a xenfs to be used on /proc/xen. This always
looked wrong to me as it merges arbitrary information and access to Xen
infrastructure. Also it is the only hypervisor support that needs such
filesystem while everyone else and even several part of the Xen support
uses the traditional ways, devices and sysfs.

One of the entries (capabilities) provides information about the
proviledge level to the hypervisor and belongs more likely into
/sys/hypervisor. The four other are plain old devices and belongs into
/dev.

Bastian

-- 
There's coffee in that nebula!
		-- Capt. Kathryn Janeway, Star Trek: Voyager, "The Cloud"

Re: xenfs aka /proc/xen

[Keir Fraser]

On 27/05/2010 23:54, "Bastian Blank" <waldi@debian.org> wrote:

> Hi
> 
> The current tree includes a xenfs to be used on /proc/xen. This always
> looked wrong to me as it merges arbitrary information and access to Xen
> infrastructure. Also it is the only hypervisor support that needs such
> filesystem while everyone else and even several part of the Xen support
> uses the traditional ways, devices and sysfs.
> 
> One of the entries (capabilities) provides information about the
> proviledge level to the hypervisor and belongs more likely into
> /sys/hypervisor. The four other are plain old devices and belongs into
> /dev.

If that stuff got moved somewhere 'better', it would be fine to update the
tools to check both new and old locations.

 -- Keir

> Bastian

Re: xenfs aka /proc/xen

[Bastian Blank]

On Fri, May 28, 2010 at 06:40:02AM +0100, Keir Fraser wrote:
> On 27/05/2010 23:54, "Bastian Blank" <waldi@debian.org> wrote:
> > One of the entries (capabilities) provides information about the
> > proviledge level to the hypervisor and belongs more likely into
> > /sys/hypervisor. The four other are plain old devices and belongs into
> > /dev.

Okay. I thought about it and would settle for the following:
* $SYSFS/hypervisor/properties/guest_capabilites
  It includes the same value then $XENFS/capabilities. Or should that be
  changed as the meaning of "control_d" is not really clear (like
  "control-domain")?
* $DEV/xen/xenbus
  Merge into builtin xenbus support or own module xenbus-user
* $DEV/xen/privcmd
  - Module xen-control or so
  - *Needs to check for CAP_ADMIN*
* $DEV/xen/xenstored
  - Module xen-control or so
  - Merges xsd_kva and xsd_port
  - Supports:
    - mmap, only support pagesized maps
    - ioctl: get event channel port, get size (page size may be different)
  - Security constraints needs check. What can a user with access to
    this device do?
* Core kernel may trigger loading of xen-control module by some means
  (to be defined).

> If that stuff got moved somewhere 'better', it would be fine to update the
> tools to check both new and old locations.

I would start with adding the new stuff then check the tools to work
with both and just ignore for another merge try or remove xenfs later.

Bastian

-- 
Power is danger.
		-- The Centurion, "Balance of Terror", stardate 1709.2

Re: xenfs aka /proc/xen

[Jeremy Fitzhardinge]

On 06/05/2010 09:29 AM, Bastian Blank wrote:
> On Fri, May 28, 2010 at 06:40:02AM +0100, Keir Fraser wrote:
>   
>> On 27/05/2010 23:54, "Bastian Blank" <waldi@debian.org> wrote:
>>     
>>> One of the entries (capabilities) provides information about the
>>> proviledge level to the hypervisor and belongs more likely into
>>> /sys/hypervisor. The four other are plain old devices and belongs into
>>> /dev.
>>>       
> Okay. I thought about it and would settle for the following:
> * $SYSFS/hypervisor/properties/guest_capabilites
>   It includes the same value then $XENFS/capabilities. Or should that be
>   changed as the meaning of "control_d" is not really clear (like
>   "control-domain")?
>   

The general rule for sysfs is one value per file.  It would probably be
more consistent to have a (guest_)capabilities directory, with a file
per capability (containing 1/0, or some other value if appropriate).

> * $DEV/xen/xenbus
>   Merge into builtin xenbus support or own module xenbus-user
>   

Would you expect to change the actual ABI/protocol?

> * $DEV/xen/privcmd
>   - Module xen-control or so
>   - *Needs to check for CAP_ADMIN*
> * $DEV/xen/xenstored
>   - Module xen-control or so
>   - Merges xsd_kva and xsd_port
>   - Supports:
>     - mmap, only support pagesized maps
>     - ioctl: get event channel port, get size (page size may be different)
>   

Yes, the interface of exposing the xenstore mfn to userspace has always
seemed a bit mad.  The kernel driver should do the mapping for
usermode.  Does it also need to expose the xs event channel?  Or can the
kernel just handle it internally and expose a normal blocking interface.

In fact, does it needs its own separate driver?  This is just
symmetrical with /{proc,dev}/xen/xenbus.  Can that driver be made to
handle both ends?  Or at least a driver which looks symmetric to the
guest-side xenbus?

>   - Security constraints needs check. What can a user with access to
>     this device do?
>   

Policy should be enforced by xenstored itself.  Guests are not
trustworthy in general, so there's no point in enforcing anything within
the guest kernel.

> * Core kernel may trigger loading of xen-control module by some means
>   (to be defined).
>   

All a bit awkward, since there's no obvious trigger to hang the load
event off.  Maybe key them off Xen or xenbus bringup as some kind of
adjunct pseudo device?

    J

Re: xenfs aka /proc/xen

[Bastian Blank]

On Sat, Jun 05, 2010 at 05:48:21PM -0700, Jeremy Fitzhardinge wrote:
> On 06/05/2010 09:29 AM, Bastian Blank wrote:
> > Okay. I thought about it and would settle for the following:
> > * $SYSFS/hypervisor/properties/guest_capabilites
> >   It includes the same value then $XENFS/capabilities. Or should that be
> >   changed as the meaning of "control_d" is not really clear (like
> >   "control-domain")?
> The general rule for sysfs is one value per file.  It would probably be
> more consistent to have a (guest_)capabilities directory, with a file
> per capability (containing 1/0, or some other value if appropriate).

You are right. However, the current sys/hypervisor interface already
uses multi-valued items.

> > * $DEV/xen/xenbus
> >   Merge into builtin xenbus support or own module xenbus-user
> Would you expect to change the actual ABI/protocol?

To reduce the complexity, partial messages could be disallowed.

> > * $DEV/xen/privcmd
> >   - Module xen-control or so
> >   - *Needs to check for CAP_ADMIN*
> > * $DEV/xen/xenstored
> >   - Module xen-control or so
> >   - Merges xsd_kva and xsd_port
> >   - Supports:
> >     - mmap, only support pagesized maps
> >     - ioctl: get event channel port, get size (page size may be different)
> Yes, the interface of exposing the xenstore mfn to userspace has always
> seemed a bit mad.  The kernel driver should do the mapping for
> usermode.  Does it also need to expose the xs event channel?  Or can the
> kernel just handle it internally and expose a normal blocking interface.

Sure, it can. However it moves the complexity from the userspace into
the kernel. As there are only two users right now, I doubt that the
easier userspace implementations would outweigh the possible problems.

> In fact, does it needs its own separate driver?  This is just
> symmetrical with /{proc,dev}/xen/xenbus.  Can that driver be made to
> handle both ends?  Or at least a driver which looks symmetric to the
> guest-side xenbus?

Not sure if this is worth the problems: Only one access-control path for
both client and server access.

However, if you want to go this way, I would propose a different
solution that looks the same in all the different access paths from
userspace: Netlink. This is than a complete overhaul.[1]

> >   - Security constraints needs check. What can a user with access to
> >     this device do?
> Policy should be enforced by xenstored itself.  Guests are not
> trustworthy in general, so there's no point in enforcing anything within
> the guest kernel.

I thought about the capacity of the server part to do damage.

> > * Core kernel may trigger loading of xen-control module by some means
> >   (to be defined).
> All a bit awkward, since there's no obvious trigger to hang the load
> event off.  Maybe key them off Xen or xenbus bringup as some kind of
> adjunct pseudo device?

Yeah. Something like that. Maybe just use the control domain capability
in the sys/hypervisor tree for that. The uevent response can set
MODALIAS.

Bastian

[1]: The problem is that this interface would be completely asynchron
with no notion of a statefull connection. However it would be a large
reduction in complexity and state.

Currently xenstore supports three types of messages: Transactional,
Read/Write and Watch. Transactional commands would go away. Every
message woule be its own transaction as such a protocol can not handle
long running transactions (are they really needed anyway?). Read/Write
would just work. Watch would be replaced by relayed change requests like
udev uses to acknowledge its work finished on an event.

-- 
Klingon phaser attack from front!!!!!
100% Damage to life support!!!!

Re: xenfs aka /proc/xen

[Jeremy Fitzhardinge]

On 06/06/2010 04:21 AM, Bastian Blank wrote:
> On Sat, Jun 05, 2010 at 05:48:21PM -0700, Jeremy Fitzhardinge wrote:
>   
>> On 06/05/2010 09:29 AM, Bastian Blank wrote:
>>     
>>> Okay. I thought about it and would settle for the following:
>>> * $SYSFS/hypervisor/properties/guest_capabilites
>>>   It includes the same value then $XENFS/capabilities. Or should that be
>>>   changed as the meaning of "control_d" is not really clear (like
>>>   "control-domain")?
>>>       
>> The general rule for sysfs is one value per file.  It would probably be
>> more consistent to have a (guest_)capabilities directory, with a file
>> per capability (containing 1/0, or some other value if appropriate).
>>     
> You are right. However, the current sys/hypervisor interface already
> uses multi-valued items.
>   

Yes, I've always been a bit disappointed with /sys/hypervisor.   I think
it came from the ppc side, and it should be much more useful than it is,
I think.  I'm not sure that anybody actually uses it for Xen at the moment.

Also, given that all (? I think there's just one?) the guest
capabilities are Xen specific, as are the other things we're talking
about, there should probably be a xen/ directory to stick all these
things into.

>>> * $DEV/xen/xenbus
>>>   Merge into builtin xenbus support or own module xenbus-user
>>>       
>> Would you expect to change the actual ABI/protocol?
>>     
> To reduce the complexity, partial messages could be disallowed.
>
>   
>>> * $DEV/xen/privcmd
>>>   - Module xen-control or so
>>>   - *Needs to check for CAP_ADMIN*
>>> * $DEV/xen/xenstored
>>>   - Module xen-control or so
>>>   - Merges xsd_kva and xsd_port
>>>   - Supports:
>>>     - mmap, only support pagesized maps
>>>     - ioctl: get event channel port, get size (page size may be different)
>>>       
>> Yes, the interface of exposing the xenstore mfn to userspace has always
>> seemed a bit mad.  The kernel driver should do the mapping for
>> usermode.  Does it also need to expose the xs event channel?  Or can the
>> kernel just handle it internally and expose a normal blocking interface.
>>     
> Sure, it can. However it moves the complexity from the userspace into
> the kernel. As there are only two users right now, I doubt that the
> easier userspace implementations would outweigh the possible problems.
>   

Well, the kernel already handles all the client-side stuff.  The kernel
needn't care much about the content of the data, just provide a
mechanism to shove it over a shared memory ring and provide
notifications, as it does with all the other xen devices.

Also, we have the irritating problem that xenstored is not restartable,
and I think that's partly because there's some limitation about mapping
the shared page or binding the event channel.  If the kernel does that
in a device, then it can deal with userspace reopening the device on
restart.  (The actual persistance of the data is a separate question.)

>> In fact, does it needs its own separate driver?  This is just
>> symmetrical with /{proc,dev}/xen/xenbus.  Can that driver be made to
>> handle both ends?  Or at least a driver which looks symmetric to the
>> guest-side xenbus?
>>     
> Not sure if this is worth the problems: Only one access-control path for
> both client and server access.
>   

I don't follow your point here.

> However, if you want to go this way, I would propose a different
> solution that looks the same in all the different access paths from
> userspace: Netlink. This is than a complete overhaul.[1]
>   

Erm, maybe.  I think a plain device is probably a better match though,
and I don't see why there would be much difference in complexity.  I
guess the device is rather ioctl-heavy, and having an explicit delimited
format might be nice.  But as you mention, the stateless connectionless
property is hard to reconcile with current xenbus.

>>>   - Security constraints needs check. What can a user with access to
>>>     this device do?
>>>       
>> Policy should be enforced by xenstored itself.  Guests are not
>> trustworthy in general, so there's no point in enforcing anything within
>> the guest kernel.
>>     
> I thought about the capacity of the server part to do damage.
>   

The server is pretty inherently trusted in the Xen model, so I don't
think that's a huge concern.

>>> * Core kernel may trigger loading of xen-control module by some means
>>>   (to be defined).
>>>       
>> All a bit awkward, since there's no obvious trigger to hang the load
>> event off.  Maybe key them off Xen or xenbus bringup as some kind of
>> adjunct pseudo device?
>>     
> Yeah. Something like that. Maybe just use the control domain capability
> in the sys/hypervisor tree for that. The uevent response can set
> MODALIAS.
>   

evtchn and gntdev are usable from domU.

    J