back/frontend drivers HelloWorld

back/frontend drivers HelloWorld

[Anton Korenyushkin]

Hi,

I'm trying to write 2 simple xen0/xenU modules, which only transfer string 
"Hello world" by grant tables interface. But in docs/misc/grant-tables.txt 
there is no explanation of how to transmit grant table reference to another 
domain:
.. gref is then somehow transmitted to domB for use.

I'm reading blkback/front driver's code, but it is too complicated. I've 
understood that i should use message rings for my task (i've read 
docs/misc/blkif-drivers-explained.txt too). But how to get such ring?

May be a simple example of solution such problem exists, so could you point me 
on this example?

Sorry for bad English.

-- 
Best regards,
Anton Korenyushkin
mailto:tiger@swsoft.mipt.ru

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 22 Sep 2005, at 14:46, Anton Korenyushkin wrote:

>
> I'm reading blkback/front driver's code, but it is too complicated. 
> I've
> understood that i should use message rings for my task (i've read
> docs/misc/blkif-drivers-explained.txt too). But how to get such ring?
>
> May be a simple example of solution such problem exists, so could you 
> point me
> on this example?

You bootstrap yourself via xenbus. You have a pre-agreed location in 
the xenstore hieararchy where the frontend driver writes key-value 
pairs containing the grant reference for the page containing the 
message rings, and an event channel for async notifications.

See the xenbus.c files in each backend driver directory for examples.

  -- Keir

Re: back/frontend drivers HelloWorld

[Mark Williamson]

> I'm trying to write 2 simple xen0/xenU modules, which only transfer string
> "Hello world" by grant tables interface. But in docs/misc/grant-tables.txt
> there is no explanation of how to transmit grant table reference to another
> domain:
> .. gref is then somehow transmitted to domB for use.
>
> I'm reading blkback/front driver's code, but it is too complicated. I've
> understood that i should use message rings for my task (i've read
> docs/misc/blkif-drivers-explained.txt too). But how to get such ring?

You share it using grant tables ;-)  I'll assume that since you're using 
grant-tables you're developing on 3.0-unstable (the control plane was very 
different on Xen 2.0).

> May be a simple example of solution such problem exists, so could you point
> me on this example?

To share the initial grant reference you using "xenstore".  This is a 
hierarchical directory structure, holding names "keys" which contain 
arbitrailty structured data.  Think "registry" ;-)

The kernel interface to Xenstore is the Xenbus - you use Xenbus API functions 
in the kernel to write and read the keys in the store.  By writing the grant 
reference into a key at a standard location, you can communicate it from the 
frontend to the backend.  This is what the block and network devices do.

For a very simple driver, you might want to bypass the complexity of using 
Xenstore at all...  How about passing the grant reference on the command 
line?  Add a printk to the frontend, outputting the reference ID to the dmesg 
log, then pass this as an argument when you load the module for the backend 
driver.  Just a thought...  (side note: I'm not sure if we currently export 
all the symbols required for an unloadable backend - if not, we probably 
should).

> Sorry for bad English.

Way better than my Russian ever was.  For that matter, probably better than my 
English too ;-)

Poka!
Mark

Re: back/frontend drivers HelloWorld

[NAHieu]

On 9/22/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>
> On 22 Sep 2005, at 14:46, Anton Korenyushkin wrote:
>
> >
> > I'm reading blkback/front driver's code, but it is too complicated.
> > I've
> > understood that i should use message rings for my task (i've read
> > docs/misc/blkif-drivers-explained.txt too). But how to get such ring?
> >
> > May be a simple example of solution such problem exists, so could you
> > point me
> > on this example?
>
> You bootstrap yourself via xenbus. You have a pre-agreed location in
> the xenstore hieararchy where the frontend driver writes key-value
> pairs containing the grant reference for the page containing the
> message rings, and an event channel for async notifications.
>

I imagine that we can get rid of event channel, and use xenstore to
notify. That is certainly OK, but I doubt about its efficientcy. What
do you think?

Thanks,
NAH

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 23 Sep 2005, at 10:53, NAHieu wrote:

>> You bootstrap yourself via xenbus. You have a pre-agreed location in
>> the xenstore hieararchy where the frontend driver writes key-value
>> pairs containing the grant reference for the page containing the
>> message rings, and an event channel for async notifications.
>>
>
> I imagine that we can get rid of event channel, and use xenstore to
> notify. That is certainly OK, but I doubt about its efficientcy. What
> do you think?

Very mad. It's easy to set up an inter-domain event channel betwene 
domains A and B:
  1. A executes something like evtchn = EVTCHNOP_alloc_unbound(B)
  2. A writes 'evtchn' into xenstore.
  3. B reads 'evtchn' from xenstore
  4. B executes something like EVTCHNOP_bind_interdomain(A,evtchn)

  -- Keir

Re: back/frontend drivers HelloWorld

[NAHieu]

On 9/23/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>
> On 23 Sep 2005, at 10:53, NAHieu wrote:
>
> >> You bootstrap yourself via xenbus. You have a pre-agreed location in
> >> the xenstore hieararchy where the frontend driver writes key-value
> >> pairs containing the grant reference for the page containing the
> >> message rings, and an event channel for async notifications.
> >>
> >
> > I imagine that we can get rid of event channel, and use xenstore to
> > notify. That is certainly OK, but I doubt about its efficientcy. What
> > do you think?
>
> Very mad.

Yes, it is. But looks like xenstore is the only way to "notify"
between userspace applications (like one runs in domU and another runs
in dom0). Or there is another good solution ?

Thanks,
NAH

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 23 Sep 2005, at 12:37, NAHieu wrote:

> Yes, it is. But looks like xenstore is the only way to "notify"
> between userspace applications (like one runs in domU and another runs
> in dom0). Or there is another good solution ?

You can execute all the event-channel binding and sending commands via 
libxenctrl (see the interfaces defintiions in xenctrl.h).

You can receive asynchronous notifications via /dev/xen/evtchn. You 
execute a special ioctl to bind to particular event-channel ports, then 
you can read() from the device to read ports that have notifications 
pending, or you can add the fd to a select loop(), or you can set 
FASYNC and get SIGIO when there are notifications pending.

  -- Keir

Re: back/frontend drivers HelloWorld

[NAHieu]

On 9/23/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>
> On 23 Sep 2005, at 12:37, NAHieu wrote:
>
> > Yes, it is. But looks like xenstore is the only way to "notify"
> > between userspace applications (like one runs in domU and another runs
> > in dom0). Or there is another good solution ?
>
> You can execute all the event-channel binding and sending commands via
> libxenctrl (see the interfaces defintiions in xenctrl.h).

Thanks for the suggestion. I will take another look.

As there is not much docs about xenstore, confuses may lead to abuse
that interface. Could you give us a brief summary about the proper
usage of xenbus/xenstore?

Many thanks.
NAH

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 23 Sep 2005, at 13:37, NAHieu wrote:

> Thanks for the suggestion. I will take another look.
>
> As there is not much docs about xenstore, confuses may lead to abuse
> that interface. Could you give us a brief summary about the proper
> usage of xenbus/xenstore?

It's intended for control plane use. That is, things like connection 
setup, teardown, error recovery, status reporting. It's not intended 
for communications of any significant bandwidth.

  -- Keir

Re: back/frontend drivers HelloWorld

[NAHieu]

On 9/23/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>
> On 23 Sep 2005, at 10:53, NAHieu wrote:
>
> >> You bootstrap yourself via xenbus. You have a pre-agreed location in
> >> the xenstore hieararchy where the frontend driver writes key-value
> >> pairs containing the grant reference for the page containing the
> >> message rings, and an event channel for async notifications.
> >>
> >
> > I imagine that we can get rid of event channel, and use xenstore to
> > notify. That is certainly OK, but I doubt about its efficientcy. What
> > do you think?
>
> Very mad. It's easy to set up an inter-domain event channel betwene
> domains A and B:
>   1. A executes something like evtchn = EVTCHNOP_alloc_unbound(B)
>   2. A writes 'evtchn' into xenstore.
>   3. B reads 'evtchn' from xenstore
>   4. B executes something like EVTCHNOP_bind_interdomain(A,evtchn)

It seems I also need to use bind_evtchn_to_irq() to handle event sent
from event-channel?


Could you tell me which code in Xen uses the above 1->4 setup
procedure? I searched for some examples, but couldnt find any.


Great thanks,
Hieu

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 24 Sep 2005, at 18:40, NAHieu wrote:

> Could you tell me which code in Xen uses the above 1->4 setup
> procedure? I searched for some examples, but couldnt find any.

There are several examples in the kernel: grep for uses of 
'alloc_unbound' in every frontend driver and 'bind_interdomain' in 
every backend driver. Unfortunately I think we currently have no 
example of doing this from user space, but the principles are the same!

There are examples of communicating with xenstore in user space in the 
tools/ directory (e.g. xend uses a python binding to do so). The 
libxenctrl functions for setting up event channels are very 
straightforward and also have python bindings. There also will be some 
code there that opens /dev/xen/evtchn so you can grep for that to get 
examples of how to receive evtchn notifications into user space.

  -- Keir

Re: back/frontend drivers HelloWorld

[NAHieu]

On 9/25/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>
> On 24 Sep 2005, at 18:40, NAHieu wrote:
>
> > Could you tell me which code in Xen uses the above 1->4 setup
> > procedure? I searched for some examples, but couldnt find any.
>
> There are several examples in the kernel: grep for uses of
> 'alloc_unbound' in every frontend driver and 'bind_interdomain' in
> every backend driver. Unfortunately I think we currently have no
> example of doing this from user space, but the principles are the same!

I am writing a small "virtual" device driver for Xen (virtual means
not "real" device). After looking at those code, I think the steps we
must do to exchange data between dom0 and domU - using backend and
frontend method:

1. domU get the event channel, and write to  xenstore

2. dom0 watches the xenstore (xs_watch) for the event channel of domU,
to detect when domU writes channel value to xenstore. dom0 then bind
to domU's channel (using  bind_evtchn_to_irqhandler())

3. whenever dom0 and domU want to exchange data, they update the
corresponding share-ring/grant-table and then inform the other using
notify_via_evtchn()


- Is that correct? ( I have some doubts about (2), as I see that the
drivers available in xen uses xenbus_driver.probe to detect domU's
driver?)


- Another question: I found it is difficult to understand
xenbus_driver/xenbus_device architecture. They must be rooted from
Linux architecture? Any good documentation to understand those things,
in order to understand xenbus_device?

Many thanks,
Hieu

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 25 Sep 2005, at 17:24, NAHieu wrote:

> - Is that correct? ( I have some doubts about (2), as I see that the
> drivers available in xen uses xenbus_driver.probe to detect domU's
> driver?)

All correct, but you skip a few details.

In step 1, domU gets the event channel port via EVTCHNOP_alloc_unbound. 
It then writes that port value to xenstore.

In step 2, dom0 binds to the remote event-channel port via 
EVTCHNOP_bind_interdomain. This returns the allocated local port which 
is what is passed to bind_evtchn_to_irqhandler().
See code in drivers/xen/blkback/interface.c for a precise example of 
doing this -- probably best to steal that portion of code as a 
template.

Also, you can find code in blkfront/blkback to allocate a shared page 
in blkfront, allocate a grant reference for it, pass that to blkback 
and map the page thru the grant reference.

  -- Keir

Re: back/frontend drivers HelloWorld

[NAHieu]

On 9/25/05, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:
>
> On 25 Sep 2005, at 17:24, NAHieu wrote:
>
> > - Is that correct? ( I have some doubts about (2), as I see that the
> > drivers available in xen uses xenbus_driver.probe to detect domU's
> > driver?)
>
> All correct, but you skip a few details.
>
> In step 1, domU gets the event channel port via EVTCHNOP_alloc_unbound.
> It then writes that port value to xenstore.
>
> In step 2, dom0 binds to the remote event-channel port via
> EVTCHNOP_bind_interdomain. This returns the allocated local port which
> is what is passed to bind_evtchn_to_irqhandler().
> See code in drivers/xen/blkback/interface.c for a precise example of
> doing this -- probably best to steal that portion of code as a
> template.
>
> Also, you can find code in blkfront/blkback to allocate a shared page
> in blkfront, allocate a grant reference for it, pass that to blkback
> and map the page thru the grant reference.
>

I looked at some frontend code, and they all exploit the xenbus_driver
(and the needed common fields are .ids, .probe and .remove ). So it is
mandatory for all backend/frontend (regardless of real or virtual
device) to do like that ?

Keir, thank you a lot.
Hieu

Re: back/frontend drivers HelloWorld

[Keir Fraser]

On 26 Sep 2005, at 02:38, NAHieu wrote:

> I looked at some frontend code, and they all exploit the xenbus_driver
> (and the needed common fields are .ids, .probe and .remove ). So it is
> mandatory for all backend/frontend (regardless of real or virtual
> device) to do like that ?

I didn't develop xenbus or the new driver interfacing code, but I'm 
certain you need to interact with xenbus to some extent -- it's the 
only way for a guest kernel to access xenstore. I suspect you are best 
off taking that xenbus-interfacing code from one of the split drivers, 
to use as an initial template.

  -- Keir