OF_DYNAMIC usage

OF_DYNAMIC usage

[Michal Simek]

Hi,

can someone tell me usage of OF_DYNAMIC?
How this can be used by user?

Is it possible to exchange device node?
Add new one, delete one, etc?

Any user guide/log will be helpful to see what I can do with it.

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: OF_DYNAMIC usage

[Rob Herring]

On 07/05/2012 07:21 AM, Michal Simek wrote:
> Hi,
> 
> can someone tell me usage of OF_DYNAMIC?
> How this can be used by user?
> 
> Is it possible to exchange device node?
> Add new one, delete one, etc?
> 
> Any user guide/log will be helpful to see what I can do with it.

2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I
would suggest looking at the existing users (only powerpc iseries and
pseries). My concern using would be that the reference counting is not
correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.

You can also activate/deactivate nodes with the status property.

Rob

Re: OF_DYNAMIC usage

[Benjamin Herrenschmidt]

On Thu, 2012-07-05 at 08:20 -0500, Rob Herring wrote:
> On 07/05/2012 07:21 AM, Michal Simek wrote:
> > Hi,
> > 
> > can someone tell me usage of OF_DYNAMIC?
> > How this can be used by user?
> > 
> > Is it possible to exchange device node?
> > Add new one, delete one, etc?
> > 
> > Any user guide/log will be helpful to see what I can do with it.
> 
> 2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I
> would suggest looking at the existing users (only powerpc iseries and
> pseries). My concern using would be that the reference counting is not
> correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.
> 
> You can also activate/deactivate nodes with the status property.
> 

No doc that I know of. It's used by pseries only (iseries is gone btw)
for "DLPAR" (aka Dynamic Reconfiguration aka hotplug).

The way it works at the moment is that when something new is plugged in,
the hypervisor talks to a proprietary crap daemon in userspace which
talks to a special tool (that one we have the source code) which then
obtains via some FW interfaces a "blob" of bits of device-tree to add
(or to remove), using a phyp specific format, and echo that stuff
into /proc/ppc64/ofdt.

It's pretty fugly...

Because nodes can come and go, we thus add refcounting. There's various
places where the refcounting is not quite right but mostly things that
don't get plugged/unplugged, actual vio devices or PCI devices tend to
get it right.

Also we don't have good rules for properties lifetime, as it is, it's
assumed to be the same as the node and there's no proper way to
add/remove/modify a property "live". I would recommend at least
something RCUish if you're going to try to be safe to delay the freeing
of the old value in the dynamic case.

There's definitely room for improvements in that area...

Cheers,
Ben.

Re: OF_DYNAMIC usage

[Michal Simek]

On 07/05/2012 10:46 PM, Benjamin Herrenschmidt wrote:
> On Thu, 2012-07-05 at 08:20 -0500, Rob Herring wrote:
>> On 07/05/2012 07:21 AM, Michal Simek wrote:
>>> Hi,
>>>
>>> can someone tell me usage of OF_DYNAMIC?
>>> How this can be used by user?
>>>
>>> Is it possible to exchange device node?
>>> Add new one, delete one, etc?
>>>
>>> Any user guide/log will be helpful to see what I can do with it.
>>
>> 2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I
>> would suggest looking at the existing users (only powerpc iseries and
>> pseries). My concern using would be that the reference counting is not
>> correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.
>>
>> You can also activate/deactivate nodes with the status property.
>>
>
> No doc that I know of. It's used by pseries only (iseries is gone btw)
> for "DLPAR" (aka Dynamic Reconfiguration aka hotplug).

ok. This is for Xilinx partial reconfiguration.

>
> The way it works at the moment is that when something new is plugged in,
> the hypervisor talks to a proprietary crap daemon in userspace which
> talks to a special tool (that one we have the source code) which then
> obtains via some FW interfaces a "blob" of bits of device-tree to add
> (or to remove), using a phyp specific format, and echo that stuff
> into /proc/ppc64/ofdt.

Where is that source code for the special tool?

Can you point me to the "phyp specific format"?

 From reconfig.c it looks like that there are some key words like
add_node/remove_node/add_property... follow by space and node name +
options which lookes like dtb format.

>
> It's pretty fugly...

Yes, I see.

>
> Because nodes can come and go, we thus add refcounting. There's various
> places where the refcounting is not quite right but mostly things that
> don't get plugged/unplugged, actual vio devices or PCI devices tend to
> get it right.

Some driver validation has to be made to get it work. I have done some correction
for xilinx drivers in past.

>
> Also we don't have good rules for properties lifetime, as it is, it's
> assumed to be the same as the node and there's no proper way to
> add/remove/modify a property "live". I would recommend at least
> something RCUish if you're going to try to be safe to delay the freeing
> of the old value in the dynamic case.
>
> There's definitely room for improvements in that area...

:-)

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: OF_DYNAMIC usage

[Benjamin Herrenschmidt]

On Fri, 2012-07-06 at 08:03 +0200, Michal Simek wrote:
> >
> > The way it works at the moment is that when something new is plugged in,
> > the hypervisor talks to a proprietary crap daemon in userspace which
> > talks to a special tool (that one we have the source code) which then
> > obtains via some FW interfaces a "blob" of bits of device-tree to add
> > (or to remove), using a phyp specific format, and echo that stuff
> > into /proc/ppc64/ofdt.
> 
> Where is that source code for the special tool?

I can dig that for you, however ...

> Can you point me to the "phyp specific format"?

Same deal, I don't think there's a public doc, however..

>  From reconfig.c it looks like that there are some key words like
> add_node/remove_node/add_property... follow by space and node name +
> options which lookes like dtb format.

Right, I would just recommend you don't do that.

The need to "hotplug" bits of device-tree is going to be generic enough
that we should come up with something better and more generic than that
pseries stuff.

IE. Some way to pass bits of ftb blob rather than this specific format
to begin with, etc...

So I'd say just ignore the pseries stuff, I can dig the tool etc... if
you -really- need them but I'd rather you don't base your stuff on it,
just make up something better & more generic for you. It will be useful
to others.

Cheers,
Ben.

Re: OF_DYNAMIC usage

[Michal Simek]

On 07/06/2012 08:19 AM, Benjamin Herrenschmidt wrote:
> On Fri, 2012-07-06 at 08:03 +0200, Michal Simek wrote:
>>>
>>> The way it works at the moment is that when something new is plugged in,
>>> the hypervisor talks to a proprietary crap daemon in userspace which
>>> talks to a special tool (that one we have the source code) which then
>>> obtains via some FW interfaces a "blob" of bits of device-tree to add
>>> (or to remove), using a phyp specific format, and echo that stuff
>>> into /proc/ppc64/ofdt.
>>
>> Where is that source code for the special tool?
>
> I can dig that for you, however ...
>
>> Can you point me to the "phyp specific format"?
>
> Same deal, I don't think there's a public doc, however..

Why is it in the kernel something which does not have any public documentation?
It seems like that it is more proprietary code.


>
>>    From reconfig.c it looks like that there are some key words like
>> add_node/remove_node/add_property... follow by space and node name +
>> options which lookes like dtb format.
>
> Right, I would just recommend you don't do that.
>
> The need to "hotplug" bits of device-tree is going to be generic enough
> that we should come up with something better and more generic than that
> pseries stuff.
>
> IE. Some way to pass bits of ftb blob rather than this specific format
> to begin with, etc...

Can we discuss this a little bit more?

 From my partial reconfiguration understanding is that you have partial bitstream
which you pass through pcap(IP and driver which can do it) to specific location
which they are known.

It means you have to unplug device which is in the same location,
load partial bitstream via pcap
register driver and probe it.

I expect that pcap driver could be aware which driver is at that location
and is able to plug and unplug it based on description.

I will ask Xilinx how this is exactly done and I hope I will get any example
to be able to do some experiments.

>
> So I'd say just ignore the pseries stuff, I can dig the tool etc... if
> you -really- need them but I'd rather you don't base your stuff on it,
> just make up something better&  more generic for you. It will be useful
> to others.

My point here is just to try to understand current working version
to get the better overview how it is done and probably working somehow.

Of course some ideas how this can/should be done will be helpful.

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: OF_DYNAMIC usage

[Benjamin Herrenschmidt]

On Fri, 2012-07-06 at 08:51 +0200, Michal Simek wrote:
> On 07/06/2012 08:19 AM, Benjamin Herrenschmidt wrote:
> > On Fri, 2012-07-06 at 08:03 +0200, Michal Simek wrote:
> >>>
> >>> The way it works at the moment is that when something new is plugged in,
> >>> the hypervisor talks to a proprietary crap daemon in userspace which
> >>> talks to a special tool (that one we have the source code) which then
> >>> obtains via some FW interfaces a "blob" of bits of device-tree to add
> >>> (or to remove), using a phyp specific format, and echo that stuff
> >>> into /proc/ppc64/ofdt.
> >>
> >> Where is that source code for the special tool?
> >
> > I can dig that for you, however ...
> >
> >> Can you point me to the "phyp specific format"?
> >
> > Same deal, I don't think there's a public doc, however..
> 
> Why is it in the kernel something which does not have any public documentation?
> It seems like that it is more proprietary code.

Don't ask me, it's been there since day 1 of the ppc64 port afaik,
before I was involved in it. I've been wanting to deprecate that
interface for a while, but haven't got to it yet.

> >
> >>    From reconfig.c it looks like that there are some key words like
> >> add_node/remove_node/add_property... follow by space and node name +
> >> options which lookes like dtb format.
> >
> > Right, I would just recommend you don't do that.
> >
> > The need to "hotplug" bits of device-tree is going to be generic enough
> > that we should come up with something better and more generic than that
> > pseries stuff.
> >
> > IE. Some way to pass bits of ftb blob rather than this specific format
> > to begin with, etc...
> 
> Can we discuss this a little bit more?

Sure :-)

>  From my partial reconfiguration understanding is that you have partial bitstream
> which you pass through pcap(IP and driver which can do it) to specific location
> which they are known.
> 
> It means you have to unplug device which is in the same location,
> load partial bitstream via pcap
> register driver and probe it.
> 
> I expect that pcap driver could be aware which driver is at that location
> and is able to plug and unplug it based on description.
> 
> I will ask Xilinx how this is exactly done and I hope I will get any example
> to be able to do some experiments.
>
> >
> > So I'd say just ignore the pseries stuff, I can dig the tool etc... if
> > you -really- need them but I'd rather you don't base your stuff on it,
> > just make up something better&  more generic for you. It will be useful
> > to others.
> 
> My point here is just to try to understand current working version
> to get the better overview how it is done and probably working somehow.
> 
> Of course some ideas how this can/should be done will be helpful.

Well, I think the right way is to have some mechanism (TBD) to be able
to inject into the kernel a bit of device-tree (or remove a bit of
device-tree). Ideally by passing it an FDT blob segment which is a  well
known & documented format.

The kernel would then expand it and call some notifiers which we can use
to create platform devices if needed etc...

Cheers,
Ben.

Re: OF_DYNAMIC usage

[Michal Simek]

On 07/06/2012 09:08 AM, Benjamin Herrenschmidt wrote:
> On Fri, 2012-07-06 at 08:51 +0200, Michal Simek wrote:
>> On 07/06/2012 08:19 AM, Benjamin Herrenschmidt wrote:
>>> On Fri, 2012-07-06 at 08:03 +0200, Michal Simek wrote:
>>>>>
>>>>> The way it works at the moment is that when something new is plugged in,
>>>>> the hypervisor talks to a proprietary crap daemon in userspace which
>>>>> talks to a special tool (that one we have the source code) which then
>>>>> obtains via some FW interfaces a "blob" of bits of device-tree to add
>>>>> (or to remove), using a phyp specific format, and echo that stuff
>>>>> into /proc/ppc64/ofdt.
>>>>
>>>> Where is that source code for the special tool?
>>>
>>> I can dig that for you, however ...
>>>
>>>> Can you point me to the "phyp specific format"?
>>>
>>> Same deal, I don't think there's a public doc, however..
>>
>> Why is it in the kernel something which does not have any public documentation?
>> It seems like that it is more proprietary code.
>
> Don't ask me, it's been there since day 1 of the ppc64 port afaik,
> before I was involved in it. I've been wanting to deprecate that
> interface for a while, but haven't got to it yet.

:-)


>>>>      From reconfig.c it looks like that there are some key words like
>>>> add_node/remove_node/add_property... follow by space and node name +
>>>> options which lookes like dtb format.
>>>
>>> Right, I would just recommend you don't do that.
>>>
>>> The need to "hotplug" bits of device-tree is going to be generic enough
>>> that we should come up with something better and more generic than that
>>> pseries stuff.
>>>
>>> IE. Some way to pass bits of ftb blob rather than this specific format
>>> to begin with, etc...
>>
>> Can we discuss this a little bit more?
>
> Sure :-)
>
>>    From my partial reconfiguration understanding is that you have partial bitstream
>> which you pass through pcap(IP and driver which can do it) to specific location
>> which they are known.
>>
>> It means you have to unplug device which is in the same location,
>> load partial bitstream via pcap
>> register driver and probe it.
>>
>> I expect that pcap driver could be aware which driver is at that location
>> and is able to plug and unplug it based on description.
>>
>> I will ask Xilinx how this is exactly done and I hope I will get any example
>> to be able to do some experiments.
>>
>>>
>>> So I'd say just ignore the pseries stuff, I can dig the tool etc... if
>>> you -really- need them but I'd rather you don't base your stuff on it,
>>> just make up something better&   more generic for you. It will be useful
>>> to others.
>>
>> My point here is just to try to understand current working version
>> to get the better overview how it is done and probably working somehow.
>>
>> Of course some ideas how this can/should be done will be helpful.
>
> Well, I think the right way is to have some mechanism (TBD) to be able
> to inject into the kernel a bit of device-tree (or remove a bit of
> device-tree). Ideally by passing it an FDT blob segment which is a  well
> known&  documented format.

ok. How that FDT blob segment should look like?
It can't be just one node because it also requires path where it is connected.

It means at least the part like below for injecting.

/dts-v1/;
/ {
	#address-cells = <1>;
	#size-cells = <1>;
	compatible = "xlnx,microblaze";

	mb_plb: plb@0 {
		#address-cells = <1>;
		#size-cells = <1>;
		compatible = "xlnx,plb-v46-1.00.a", "simple-bus";
		DIP_Switches_4Bit: gpio@81440000 {
			#gpio-cells = <2>;
			compatible = "xlnx,xps-gpio-1.00.a";
			gpio-controller ;
			reg = < 0x81440000 0x10000 >;
			/* ... */
		} ;
	} ;
};

Not sure if this can be used for removing. I mean if you want to remove node
if make sense to pass the whole node.


Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: OF_DYNAMIC usage

[Benjamin Herrenschmidt]

On Fri, 2012-07-06 at 10:02 +0200, Michal Simek wrote:

> ok. How that FDT blob segment should look like?
> It can't be just one node because it also requires path where it is connected.
> 
> It means at least the part like below for injecting.

No, my idea was to pass 3 arguments:

 - action (enum)
 - path (string)
 - segment (blob)

action would be typically add/remove

 * add: add node under <path> whose content is in <segment>

 * remove: remove node <path> and all children

(or we could require children removal to be done explicitly).

Cheers,
Ben.

> /dts-v1/;
> / {
> 	#address-cells = <1>;
> 	#size-cells = <1>;
> 	compatible = "xlnx,microblaze";
> 
> 	mb_plb: plb@0 {
> 		#address-cells = <1>;
> 		#size-cells = <1>;
> 		compatible = "xlnx,plb-v46-1.00.a", "simple-bus";
> 		DIP_Switches_4Bit: gpio@81440000 {
> 			#gpio-cells = <2>;
> 			compatible = "xlnx,xps-gpio-1.00.a";
> 			gpio-controller ;
> 			reg = < 0x81440000 0x10000 >;
> 			/* ... */
> 		} ;
> 	} ;
> };
> 
> Not sure if this can be used for removing. I mean if you want to remove node
> if make sense to pass the whole node.
> 
> 
> Thanks,
> Michal
> 
> 

Re: OF_DYNAMIC usage

[Michal Simek]

On 07/06/2012 10:10 AM, Benjamin Herrenschmidt wrote:
> On Fri, 2012-07-06 at 10:02 +0200, Michal Simek wrote:
>
>> ok. How that FDT blob segment should look like?
>> It can't be just one node because it also requires path where it is connected.
>>
>> It means at least the part like below for injecting.
>
> No, my idea was to pass 3 arguments:
>
>   - action (enum)
>   - path (string)
>   - segment (blob)
>
> action would be typically add/remove
>
>   * add: add node under<path>  whose content is in<segment>
>
>   * remove: remove node<path>  and all children
>
> (or we could require children removal to be done explicitly).

ok.

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

RE: OF_DYNAMIC usage

[Stephen Neuendorffer]

> -----Original Message-----
> From: devicetree-discuss [mailto:devicetree-discuss-
> bounces+stephen.neuendorffer=xilinx.com-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org] On Behalf Of
> Benjamin Herrenschmidt
> Sent: Friday, July 06, 2012 1:10 AM
> To: monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org
> Cc: devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
> Subject: Re: OF_DYNAMIC usage
> 
> On Fri, 2012-07-06 at 10:02 +0200, Michal Simek wrote:
> 
> > ok. How that FDT blob segment should look like?
> > It can't be just one node because it also requires path where it is
> connected.
> >
> > It means at least the part like below for injecting.
> 
> No, my idea was to pass 3 arguments:
> 
>  - action (enum)
>  - path (string)
>  - segment (blob)
> 
> action would be typically add/remove
> 
>  * add: add node under <path> whose content is in <segment>
> 
>  * remove: remove node <path> and all children
> 
> (or we could require children removal to be done explicitly).

I've implemented something like Michal's suggestion, although I never
worked out the details of removing.  Primarily, I was interested in the
slightly simpler case of a PCIe endpoint, whose contents are described
by a device tree.  This means that the device tree fragment has
to be able to be independent of its context.  Basically, I added a
function: 

+/**
+ * unflatten_partial_device_tree - create tree of device_nodes from
flat blob
+ *
+ * unflattens the device-tree passed by the firmware, creating the
+ * tree of struct device_node. It also fills the "name" and "type"
+ * pointers of the nodes so the normal device-tree walking functions
+ * can be used.
+ */
+void unflatten_partial_device_tree(unsigned long *blob, struct
device_node **mynodes)

That can be called by a device driver, like:

+       unflatten_partial_device_tree(drvdata->fw_entry->data,
+                                     &drvdata->child_nodes);
+
+       /* Generate child devices from the device tree */
+       of_platform_bus_probe(drvdata->child_nodes, NULL, &pdev->dev);

Part of the reason for implementing it this way is that it allows the
result to work on x86, where
there may not be a toplevel device tree, so adding and deleting device
nodes is handled at the device level, rather than having the device
trees themselves be dynamic.  The downside, of course it that it is
harder to get references from one device tree fragment into another.
However, in the case of either partial reconfiguration or PCIe, I think
this makes more sense than the OF_DYNAMIC way.

We demonstrated this working in a PCIe system and posted the patches
several years ago, but they were never mainlined, and the
architecture-independent device tree pieces have gone through a lot of
change since then.  Maybe Michal can get them to compile and post
updated versions for more comment?

Steve


> 
> Cheers,
> Ben.
> 
> > /dts-v1/;
> > / {
> > 	#address-cells = <1>;
> > 	#size-cells = <1>;
> > 	compatible = "xlnx,microblaze";
> >
> > 	mb_plb: plb@0 {
> > 		#address-cells = <1>;
> > 		#size-cells = <1>;
> > 		compatible = "xlnx,plb-v46-1.00.a", "simple-bus";
> > 		DIP_Switches_4Bit: gpio@81440000 {
> > 			#gpio-cells = <2>;
> > 			compatible = "xlnx,xps-gpio-1.00.a";
> > 			gpio-controller ;
> > 			reg = < 0x81440000 0x10000 >;
> > 			/* ... */
> > 		} ;
> > 	} ;
> > };
> >
> > Not sure if this can be used for removing. I mean if you want to
> remove node
> > if make sense to pass the whole node.
> >
> >
> > Thanks,
> > Michal
> >
> >
> 
> 
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss


This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.

Re: OF_DYNAMIC usage

[Michal Simek]

On 07/05/2012 03:20 PM, Rob Herring wrote:
> On 07/05/2012 07:21 AM, Michal Simek wrote:
>> Hi,
>>
>> can someone tell me usage of OF_DYNAMIC?
>> How this can be used by user?
>>
>> Is it possible to exchange device node?
>> Add new one, delete one, etc?
>>
>> Any user guide/log will be helpful to see what I can do with it.
>
> 2nd question on OF_DYNAMIC in a week... I haven't really looked at it. I
> would suggest looking at the existing users (only powerpc iseries and
> pseries). My concern using would be that the reference counting is not
> correct for many drivers as of_node_get/put is a nop without OF_DYNAMIC.

Agree, I know that reference counting is not correct for many drivers.

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian