Re: Where can I find some tutorial or manual on how to use xenstore?

Re: Where can I find some tutorial or manual on how to use xenstore?

[Nick Logan]

It's certainly possible, I've used this code as the basis for a new 
driver, which is similar to the vbd driver. The driver starts fine but I 
am unable to have it participate in a domain save/restore. I guess that 
this is because xend is not aware of  the new driver and the open 
devices supported by this driver and hence the driver does not receive 
resume calls when the domain is restored..

Is there any way that a new driver can make it's existance known to xend?

Nick

>
>Message: 3
>Date: Sun, 25 Jun 2006 15:00:49 -0400
>From: "Lily Huang" <ushuanglily@gmail.com>
>Subject: Re: [Xen-devel] Where can I find some tutorial or manual on
>	how to	use xenstore?
>To: "Jacob Gorm Hansen" <jacobg@diku.dk>
>Cc: xen-devel <xen-devel@lists.xensource.com>
>Message-ID:
>	<4d616cca0606251200q1103ec01q90799667882c4fcb@mail.gmail.com>
>Content-Type: text/plain; charset="iso-8859-1"
>
>Thanks a lot!
>
>This seems to be good for setting up block and network driver. But can the
>same code used for new drivers?
>
>BTW: why do you call your repository evilman? ;-)
>
>lily
>
>On 6/25/06, Jacob Gorm Hansen <jacobg@diku.dk> wrote:
>  
>
>>On 6/19/06, Lily Huang <ushuanglily@gmail.com> wrote:
>>    
>>
>>>I found from xen block and network driver that xenstore seems to need
>>>      
>>>
>>python
>>    
>>
>>>code for initialization and configuration. Is this true?
>>>But the python code seems to be complicated.
>>>
>>>Is there any complete tutorial or manual on how to use xenstore?
>>>      
>>>
>>Check out the tools/migrate/buscreate.c tool in
>>http://www.distlab.dk/hg/index.cgi/xen-evilman.hg , it creates a
>>domain with disk and network access, without needing python.
>>
>>Jacob
>>
>>    
>>

Re: Where can I find some tutorial or manual on how to use xenstore?

[Ewan Mellor]

On Wed, Jun 28, 2006 at 12:08:33PM +0100, Nick Logan wrote:

> It's certainly possible, I've used this code as the basis for a new 
> driver, which is similar to the vbd driver. The driver starts fine but I 
> am unable to have it participate in a domain save/restore. I guess that 
> this is because xend is not aware of  the new driver and the open 
> devices supported by this driver and hence the driver does not receive 
> resume calls when the domain is restored..
> 
> Is there any way that a new driver can make it's existance known to xend?

If you've managed to get the driver to start through xend, I'd be surprised if
it's xend's fault that save/restore does not work.

In any case, the main driver integration point is at the bottom of
tools/python/xen/xend/XendDomainInfo.py, where you can register a subclass of
xen.xend.server.DevController.py to manage the device.  You also need to make
sure that the driver's config is parsed properly by xm.

Take a look at Julian Chesterfield and Andrew Warfield's recent blktap patches
to see how driver tool integration is done:

http://lists.xensource.com/archives/html/xen-devel/2006-06/msg00738.html

Cheers,

Ewan.

Re: Where can I find some tutorial or manual on how to use xenstore?

[Nick Logan]

Thanks Ewan,

See below.

Cheers,

Nick

Ewan Mellor wrote:

>On Wed, Jun 28, 2006 at 12:08:33PM +0100, Nick Logan wrote:
>
>  
>
>>It's certainly possible, I've used this code as the basis for a new 
>>driver, which is similar to the vbd driver. The driver starts fine but I 
>>am unable to have it participate in a domain save/restore. I guess that 
>>this is because xend is not aware of  the new driver and the open 
>>devices supported by this driver and hence the driver does not receive 
>>resume calls when the domain is restored..
>>
>>Is there any way that a new driver can make it's existance known to xend?
>>    
>>
>
>If you've managed to get the driver to start through xend, I'd be surprised if
>it's xend's fault that save/restore does not work.
>  
>
The driver was started outside of xend, using a varient of Jacob's 
buscreate program to set the necessary values in xenstore. As this a 3rd 
party driver, I'm looking for a solution that does not involve xm or 
xend changes, if that's possible. I'll take a look at the blktap patches 
to see if that helps.

>In any case, the main driver integration point is at the bottom of
>tools/python/xen/xend/XendDomainInfo.py, where you can register a subclass of
>xen.xend.server.DevController.py to manage the device.  You also need to make
>sure that the driver's config is parsed properly by xm.
>
>Take a look at Julian Chesterfield and Andrew Warfield's recent blktap patches
>to see how driver tool integration is done:
>
>http://lists.xensource.com/archives/html/xen-devel/2006-06/msg00738.html
>
>Cheers,
>
>Ewan.
>  
>

Re: Where can I find some tutorial or manual on how to use xenstore?

[Ewan Mellor]

On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:

> The driver was started outside of xend, using a varient of Jacob's 
> buscreate program to set the necessary values in xenstore. As this a 3rd 
> party driver, I'm looking for a solution that does not involve xm or 
> xend changes, if that's possible. I'll take a look at the blktap patches 
> to see if that helps.

Xend is explicitly bringing the devices back up on restore.  If you
deliberately bypass it, then you are going to have to do that bringup
yourself.

Ewan.

Re: Where can I find some tutorial or manual on how to use xenstore?

[Nick Logan]

Ewan Mellor wrote:

>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
>
>  
>
>>The driver was started outside of xend, using a varient of Jacob's 
>>buscreate program to set the necessary values in xenstore. As this a 3rd 
>>party driver, I'm looking for a solution that does not involve xm or 
>>xend changes, if that's possible. I'll take a look at the blktap patches 
>>to see if that helps.
>>    
>>
>
>Xend is explicitly bringing the devices back up on restore.  If you
>deliberately bypass it, then you are going to have to do that bringup
>yourself.
>
>Ewan.
>  
>
I guessed that would be the case. The bringup for a restore would be 
quite straighforward but more complex for migration. Has anyone 
suggested hooks for xend to deal with 3rd party drivers so that it could 
initialise and restore devices that are supported by these drivers?  
This would enable new drivers to be implemented without changes to xend.

Cheers,

Nick

Re: Where can I find some tutorial or manual on how to use xenstore?

[Ewan Mellor]

On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:

> Ewan Mellor wrote:
> 
> >On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
> >
> > 
> >
> >>The driver was started outside of xend, using a varient of Jacob's 
> >>buscreate program to set the necessary values in xenstore. As this a 3rd 
> >>party driver, I'm looking for a solution that does not involve xm or 
> >>xend changes, if that's possible. I'll take a look at the blktap patches 
> >>to see if that helps.
> >>   
> >>
> >
> >Xend is explicitly bringing the devices back up on restore.  If you
> >deliberately bypass it, then you are going to have to do that bringup
> >yourself.
> >
> >Ewan.
> > 
> >
> I guessed that would be the case. The bringup for a restore would be 
> quite straighforward but more complex for migration. Has anyone 
> suggested hooks for xend to deal with 3rd party drivers so that it could 
> initialise and restore devices that are supported by these drivers?  
> This would enable new drivers to be implemented without changes to xend.

One would have to write a device handler that parsed generic config, and then
passed it through to the store unaltered, and then have hotplug scripts that
could cope with this unparsed config.  At the moment, the driver backends do
special-case things like generating a MAC address when none is supplied,
converting device names to their major:minor, that kind of thing.  There isn't
a generic device path; adding one wouldn't be hard.

Ewan.

Re: Where can I find some tutorial or manual on how to use xenstore?

[Nick Logan]

Ewan Mellor wrote:

>On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
>
>  
>
>>Ewan Mellor wrote:
>>
>>    
>>
>>>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
>>>
>>>
>>>
>>>      
>>>
>>>>The driver was started outside of xend, using a varient of Jacob's 
>>>>buscreate program to set the necessary values in xenstore. As this a 3rd 
>>>>party driver, I'm looking for a solution that does not involve xm or 
>>>>xend changes, if that's possible. I'll take a look at the blktap patches 
>>>>to see if that helps.
>>>>  
>>>>
>>>>        
>>>>
>>>Xend is explicitly bringing the devices back up on restore.  If you
>>>deliberately bypass it, then you are going to have to do that bringup
>>>yourself.
>>>
>>>Ewan.
>>>
>>>
>>>      
>>>
>>I guessed that would be the case. The bringup for a restore would be 
>>quite straighforward but more complex for migration. Has anyone 
>>suggested hooks for xend to deal with 3rd party drivers so that it could 
>>initialise and restore devices that are supported by these drivers?  
>>This would enable new drivers to be implemented without changes to xend.
>>    
>>
>
>One would have to write a device handler that parsed generic config, and then
>passed it through to the store unaltered, and then have hotplug scripts that
>could cope with this unparsed config.  At the moment, the driver backends do
>special-case things like generating a MAC address when none is supplied,
>converting device names to their major:minor, that kind of thing.  There isn't
>a generic device path; adding one wouldn't be hard.
>
>Ewan.
>  
>
Can I just confirm what your proposal is:

Add a generic device config to the xm config file that would contain the 
driver id, virtual device, physical device .....
Add a generic device handler to xend that would pass the unparsed 
generic config through to xenstore.
Add a hotplug script that would parse the unparsed config in xenstore 
and start the driver.
When saving a domain, xend would save the generic device config.
xend would call restore for the generic devices.

If  that's correct, I'll find some time to investigate how this could done.

Nick

Re: Where can I find some tutorial or manual on how to use xenstore?

[Ewan Mellor]

On Thu, Jun 29, 2006 at 02:53:26PM +0100, Nick Logan wrote:

> Ewan Mellor wrote:
> 
> >On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
> >
> > 
> >
> >>Ewan Mellor wrote:
> >>
> >>   
> >>
> >>>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
> >>>
> >>>
> >>>
> >>>     
> >>>
> >>>>The driver was started outside of xend, using a varient of Jacob's 
> >>>>buscreate program to set the necessary values in xenstore. As this a 
> >>>>3rd party driver, I'm looking for a solution that does not involve xm 
> >>>>or xend changes, if that's possible. I'll take a look at the blktap 
> >>>>patches to see if that helps.
> >>>> 
> >>>>
> >>>>       
> >>>>
> >>>Xend is explicitly bringing the devices back up on restore.  If you
> >>>deliberately bypass it, then you are going to have to do that bringup
> >>>yourself.
> >>>
> >>>Ewan.
> >>>
> >>>
> >>>     
> >>>
> >>I guessed that would be the case. The bringup for a restore would be 
> >>quite straighforward but more complex for migration. Has anyone 
> >>suggested hooks for xend to deal with 3rd party drivers so that it could 
> >>initialise and restore devices that are supported by these drivers?  
> >>This would enable new drivers to be implemented without changes to xend.
> >>   
> >>
> >
> >One would have to write a device handler that parsed generic config, and 
> >then
> >passed it through to the store unaltered, and then have hotplug scripts 
> >that
> >could cope with this unparsed config.  At the moment, the driver backends 
> >do
> >special-case things like generating a MAC address when none is supplied,
> >converting device names to their major:minor, that kind of thing.  There 
> >isn't
> >a generic device path; adding one wouldn't be hard.
> >
> >Ewan.
> > 
> >
> Can I just confirm what your proposal is:
> 
> Add a generic device config to the xm config file that would contain the 
> driver id, virtual device, physical device .....
> Add a generic device handler to xend that would pass the unparsed 
> generic config through to xenstore.
> Add a hotplug script that would parse the unparsed config in xenstore 
> and start the driver.
> When saving a domain, xend would save the generic device config.
> xend would call restore for the generic devices.
> 
> If  that's correct, I'll find some time to investigate how this could done.

Yes, that's it.  It sounds like something that would be very useful.

Cheers,

Ewan.

Re: Where can I find some tutorial or manual on how to use xenstore?

[Andrew Warfield]

I also agree that this would be very useful, and I think the drivers
are stable enough now that it's a good idea.

In addition to the generic xend integration, it would be quite good if
the skeleton frontend/backend contained minimal demonstrator code to
set up a device channel (a.k.a. shared memory page and event mapping)
between the two VMs.

We've talked about having something like this in the tree in the past,
it would clearly be useful for people wanting to write new drivers,
but I think there was fear that it would rot too quickly (and it
certainly would have in the past).  If the skeleton driver did
something simple but measurable (for correctness) like ping-pong
between the frontend and backend, we could add it to XenRT and make
sure that it stayed up to date.

just a thought...

a.

On 6/29/06, Ewan Mellor <ewan@xensource.com> wrote:
> On Thu, Jun 29, 2006 at 02:53:26PM +0100, Nick Logan wrote:
>
> > Ewan Mellor wrote:
> >
> > >On Wed, Jun 28, 2006 at 04:20:51PM +0100, Nick Logan wrote:
> > >
> > >
> > >
> > >>Ewan Mellor wrote:
> > >>
> > >>
> > >>
> > >>>On Wed, Jun 28, 2006 at 02:38:15PM +0100, Nick Logan wrote:
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>>The driver was started outside of xend, using a varient of Jacob's
> > >>>>buscreate program to set the necessary values in xenstore. As this a
> > >>>>3rd party driver, I'm looking for a solution that does not involve xm
> > >>>>or xend changes, if that's possible. I'll take a look at the blktap
> > >>>>patches to see if that helps.
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>Xend is explicitly bringing the devices back up on restore.  If you
> > >>>deliberately bypass it, then you are going to have to do that bringup
> > >>>yourself.
> > >>>
> > >>>Ewan.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>I guessed that would be the case. The bringup for a restore would be
> > >>quite straighforward but more complex for migration. Has anyone
> > >>suggested hooks for xend to deal with 3rd party drivers so that it could
> > >>initialise and restore devices that are supported by these drivers?
> > >>This would enable new drivers to be implemented without changes to xend.
> > >>
> > >>
> > >
> > >One would have to write a device handler that parsed generic config, and
> > >then
> > >passed it through to the store unaltered, and then have hotplug scripts
> > >that
> > >could cope with this unparsed config.  At the moment, the driver backends
> > >do
> > >special-case things like generating a MAC address when none is supplied,
> > >converting device names to their major:minor, that kind of thing.  There
> > >isn't
> > >a generic device path; adding one wouldn't be hard.
> > >
> > >Ewan.
> > >
> > >
> > Can I just confirm what your proposal is:
> >
> > Add a generic device config to the xm config file that would contain the
> > driver id, virtual device, physical device .....
> > Add a generic device handler to xend that would pass the unparsed
> > generic config through to xenstore.
> > Add a hotplug script that would parse the unparsed config in xenstore
> > and start the driver.
> > When saving a domain, xend would save the generic device config.
> > xend would call restore for the generic devices.
> >
> > If  that's correct, I'll find some time to investigate how this could done.
>
> Yes, that's it.  It sounds like something that would be very useful.
>
> Cheers,
>
> Ewan.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Re: Where can I find some tutorial or manual on how to use xenstore?

[Nick Logan]

>> Can I just confirm what your proposal is:
>>
>> Add a generic device config to the xm config file that would contain the 
>> driver id, virtual device, physical device .....
>> Add a generic device handler to xend that would pass the unparsed 
>> generic config through to xenstore.
>> Add a hotplug script that would parse the unparsed config in xenstore 
>> and start the driver.
>> When saving a domain, xend would save the generic device config.
>> xend would call restore for the generic devices.
>>
>> If  that's correct, I'll find some time to investigate how this could done.
>>     
>
> Yes, that's it.  It sounds like something that would be very useful.
>
> Cheers,
>
> Ewan.
>   

I can't promise that I'll get onto this straightaway, but if we're going 
to go ahead with a new driver, we'll have to have some way of dealing 
with save/restore and migration. This looks like a way to solve the problem.

Cheers,

Nick

Where can I find some tutorial or manual on how to use xenstore?

[Lily Huang]

[-- Attachment #1.1: Type: text/plain, Size: 260 bytes --]

I found from xen block and network driver that xenstore seems to need python
code for initialization and configuration. Is this true?
But the python code seems to be complicated.

Is there any complete tutorial or manual on how to use xenstore?

Thanks!
-lily

[-- Attachment #1.2: Type: text/html, Size: 286 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Where can I find some tutorial or manual on how to use?xenstore?

[Horms]

In article <4d616cca0606190013h3922ae8i155de76067d35083@mail.gmail.com> you wrote:
> [-- multipart/alternative, encoding 7bit, 2 lines --]
> 
>    [-- text/plain, encoding 7bit, charset: ISO-8859-1, 9 lines --]
> 
> I found from xen block and network driver that xenstore seems to need python
> code for initialization and configuration. Is this true?
> But the python code seems to be complicated.

I believe that the answer is that xenstore needs asistance from
code running in userspace in domain U, and currently the only
implemntation of that code is in python.

-- 
Horms                                           http://www.vergenet.net/~horms/

Re: Where can I find some tutorial or manual on how to use xenstore?

[Jacob Gorm Hansen]

On 6/19/06, Lily Huang <ushuanglily@gmail.com> wrote:
> I found from xen block and network driver that xenstore seems to need python
> code for initialization and configuration. Is this true?
> But the python code seems to be complicated.
>
> Is there any complete tutorial or manual on how to use xenstore?

Check out the tools/migrate/buscreate.c tool in
http://www.distlab.dk/hg/index.cgi/xen-evilman.hg , it creates a
domain with disk and network access, without needing python.

Jacob

Re: Where can I find some tutorial or manual on how to use xenstore?

[Lily Huang]

[-- Attachment #1.1: Type: text/plain, Size: 752 bytes --]

Thanks a lot!

This seems to be good for setting up block and network driver. But can the
same code used for new drivers?

BTW: why do you call your repository evilman? ;-)

lily

On 6/25/06, Jacob Gorm Hansen <jacobg@diku.dk> wrote:
>
> On 6/19/06, Lily Huang <ushuanglily@gmail.com> wrote:
> > I found from xen block and network driver that xenstore seems to need
> python
> > code for initialization and configuration. Is this true?
> > But the python code seems to be complicated.
> >
> > Is there any complete tutorial or manual on how to use xenstore?
>
> Check out the tools/migrate/buscreate.c tool in
> http://www.distlab.dk/hg/index.cgi/xen-evilman.hg , it creates a
> domain with disk and network access, without needing python.
>
> Jacob
>

[-- Attachment #1.2: Type: text/html, Size: 1180 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Where can I find some tutorial or manual on how to use xenstore?

[Jacob Gorm Hansen]

On 6/25/06, Lily Huang <ushuanglily@gmail.com> wrote:
> Thanks a lot!
>
> This seems to be good for setting up block and network driver. But can the
> same code used for new drivers?

No, you will need to adapt it a bit, but you can likely use one of the
setup_vbd() or setup_vif() functions as template. Turning on the
printf-enabled version of the xs_w() macro will probably also provide
helpful information.

> BTW: why do you call your repository evilman? ;-)

That is a good question, please see the explanation in the paper "The
Laundromat Model for Autonomic Cluster Computing", which just appear
at ICAC06 and is available from my web page, at
http://www.diku.dk/~jacobg ;-)

Regards,
Jacob