Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

Hi Benoit,

On Nov 2, 2012, at 10:15 AM, Cousson, Benoit wrote:

> On 11/1/2012 1:00 PM, Koen Kooi wrote:
>> tl;dr: please suggest an actual solution that allows plug&play when plugging in multiple capes and applying power after that. Preferably one that doesn't pass the buck to u-boot.
>> 
>> Op 1 nov. 2012, om 12:26 heeft "Cousson, Benoit" <b-cousson@ti.com> het volgende geschreven:
>> 
>>> Hi Panto,
>>> 
>>> On 11/1/2012 11:39 AM, Pantelis Antoniou wrote:
>>>> Hi Benoit,
>>>> 
>>>> On Nov 1, 2012, at 12:23 PM, Cousson, Benoit wrote:
>>>> 
>>>>> On 11/1/2012 8:02 AM, Pantelis Antoniou wrote:
>>>>>> Hi Felibe,
>>>>>> 
>>>>>> On Nov 1, 2012, at 12:14 AM, Felipe Balbi wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> On Wed, Oct 31, 2012 at 11:36:25PM +0200, Pantelis Antoniou wrote:
>>>>>>>>> * Pantelis Antoniou <panto@antoniou-consulting.com> [121031 13:14]:
>>>>>>>>>> On Oct 31, 2012, at 9:55 PM, Benoit Cousson wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Yeah, I do agree. I'm confused as well. Only OMAP IPs under PRCM control
>>>>>>>>>>> could have an hwmod and thus must be handled by an omap_device.
>>>>>>>>>>> 
>>>>>>>>>>> Any devices that is created later cannot be omap_device. The DT core
>>>>>>>>>>> will create regular platform_device for them.
>>>>>>>>>>> 
>>>>>>>>>>> Since cape is an external board, it should have nothing to do with
>>>>>>>>>>> omap_device.
>>>>>>>>>>> 
>>>>>>>>>>> Looking at your patch:
>>>>>>>>>>> da8xx-dt: Create da8xx DT adapter device
>>>>>>>>>>> 
>>>>>>>>>>> I understand why you do that, but in fact that patch does not make sense
>>>>>>>>>>> to me :-(
>>>>>>>>>>> 
>>>>>>>>>>> Why do you have to create an omap_device from the driver probe?
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> The problem is that capes are not external boards in the normal sense
>>>>>>>>>> as a PCI board is. In the PCI case the hardware that implements the
>>>>>>>>>> desired functionality is on the PCI board, while in the cape case the
>>>>>>>>>> hardware module is in the SoC. The cape most of the times is quite
>>>>>>>>>> simple and contains passive components, leds or some kind of I2C/SPI devices.
>>>>>>>>> 
>>>>>>>>> Ah now I see, you're talking about the beaglebone extension
>>>>>>>>> boards :)
>>>>>>>>> 
>>>>>>>>> The way to deal with those properly is to just edit the
>>>>>>>>> board .dts entry to include omap-beaglebone-cape-xyz.dtsi
>>>>>>>>> whatever.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> That is what is being done...
>>>>>>>> This is the fallout.
>>>>>>>> 
>>>>>>>>>> You can't instantiate the omap_device early in the boot process like it was done up to
>>>>>>>>>> now in the board file. You can only do that later in the boot process (for built-in
>>>>>>>>>> cape drivers), or even after user-space has booted and the matching cape driver module
>>>>>>>>>> has been loaded.
>>>>>>>>> 
>>>>>>>>> Yes you can, just edit the .dts file for the extension
>>>>>>>>> board you have connected. This stuff should be DT
>>>>>>>>> only for sure, let's not even start adding platform_data
>>>>>>>>> entries for that.
>>>>>>>>> 
>>>>>>>>> The omap_device entries for the omap internal devices
>>>>>>>>> will be created automatically during startup based on the
>>>>>>>>> .dts. Note that you can set status = "disabled" for the
>>>>>>>>> omap internal devices in the .dts file, the devices are
>>>>>>>>> there in the hardware.
>>>>>>>>> 
>>>>>>>>> If you are sure the extension boards are safely hot
>>>>>>>>> pluggable, then all you need is some interface to enable
>>>>>>>>> and disable devices after booting. But that should be
>>>>>>>>> done in Linux generic way.
>>>>>>>>> 
>>>>>>>>> So we should not export any omap_hwmod or omap_device
>>>>>>>>> things, and want to keep it __init only, and local to
>>>>>>>>> arch/arm/mach-omap2.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> I disagree. This is not something that is handled by just
>>>>>>>> editing the dts. The way it is handled, is in the Linux
>>>>>>>> generic way, we have a proper bus, and the drivers as compiled
>>>>>>>> as standalone file.
>>>>>>> 
>>>>>>> when you have an actual bus, that'd be correct.
>>>>>> 
>>>>>> What do you think capebus is? It is an actual bus that allows you
>>>>>> to do so.
>>>>>> 
>>>>>>> 
>>>>>>>> We're hitting a use case that wasn't there in omap until now.
>>>>>>>> 
>>>>>>>> There a a whole bunch of conflicting capes. There's no
>>>>>>>> way to instantiate them together. They must be instantiated
>>>>>>>> only after their EEPROMs are read and they are matched
>>>>>>>> to their corresponding cape drivers.
>>>>>>> 
>>>>>>> why this requirement of instantiating them only after reading EEPROMs ?
>>>>>>> 
>>>>>>> It's unnecessary to add that requirement, just do what Tony said
>>>>>>> (include my-awesome-cape.dtsi to bone.dts) and all i2c/spi devices
>>>>>>> should be created automatically.
>>>>>>> 
>>>>>>> The thing is that there is no such thing as a cape-device. The cape is
>>>>>>> just a collection of I2C, 1-wire and SPI devices anyway. What we should
>>>>>>> instantiate is bmp085, tls2550, sht21, instead of instantiating
>>>>>>> beagle-bone-weather-cape.
>>>>>>> 
>>>>>>> What's the problem in just instantiating all of those from bone.dts ?
>>>>>>> Expose the EEPROMs to userland so whatever SW you guys have running on
>>>>>>> the bone can figure out what features to expose to the SDK which the
>>>>>>> user sees, but the kernel doesn't need to know that there is a
>>>>>>> weather-cape attached to the bone.
>>>>>>> 
>>>>>> 
>>>>>> The I2C/SPI devices are not a problem at all.
>>>>>> 
>>>>>> Let me try to explain what the problem is, and why it all this
>>>>>> is needed.
>>>>>> 
>>>>>> First of all, capes may be relatively simple boards, which may function
>>>>>> as a generic device - like the generic capes. They might not necessarily
>>>>>> be simple. Some capes, for example the geiger cape have a number of
>>>>>> components that perform a specific task; in this instance the driving
>>>>>> circuit of the geiger tube and the event detector input. Other capes
>>>>>> that are being designed now are even more complex, i.e. there's a
>>>>>> radar cape, an fpga cape and so on. So for these kind of boards
>>>>>> you will need a specific driver. That driver will probably use some
>>>>>> generic-cape bits (like gpios, pwms, keys what-ever).
>>>>>> But it will put them to use in the custom in-kernel driver in it's own
>>>>>> way. You can't put that task to user-space, if it's ever slightly complex.
>>>>>> 
>>>>>> So for really simple capes, after considerable pain you might, just
>>>>>> might, dump the problem to user-space and try to make it work.
>>>>>> People have tried that and it's a total pain. There is no way that this will
>>>>>> work with anything more complex than a generic cape.
>>>>>> 
>>>>>> Then there's the out of the box experience problem.
>>>>>> 
>>>>>> What a customer that buys one of these boards along with a number of
>>>>>> capes wants is to plug them, and have them work. You cannot have
>>>>>> the DTS file activating all the drivers for each possible cape since
>>>>>> they conflict; the pins that one SPI bus that one cape uses might be a
>>>>>> PWM output for another, and so on.
>>>>>> 
>>>>>> Asking the customer to edit the distro supplied kernel's dts and recompiling,
>>>>>> while figuring out how to solve conflicts is not going to fly.
>>>>> 
>>>>> Well you do not have to rebuild the kernel if you just want to update the DTS.
>>>>> The way DT is done, the best you can do is to rely on the bootloader in your case to select the proper DTB. You should try to merge dynamically AMxx + beaglebone + capeXXX based on some EEPROM id.
>>>>> 
>>>> 
>>>> That is what the capebus does. It inserts
>>> 
>>> It inserts what?
>>> 
>>>>> FWIW, we do have a similar, but simpler, problem with the beagle / beagle-xm and panda / panda-es. But for the moment we are just using a different DTS for each board.
>>>> 
>>>> A different DTS for each board combination... There alot more capes for the bone that they are for the beagle & the panda.
>>>> And more are going to come, but not necessarily from people that have any connection to TI or CCO.
>>> 
>>> Sure, but my point is that your solution will not solve our problem :-)
>>> This is a generic problem that you address with a very custom / specific approach.
>>> 
>>>> You still haven't described how I could solve the issue of conflicting capes using a single DTS.
>>> 
>>> Well I don't have the solution otherwise I will have done it already :-)
>>> My point is that the solution has to be in the DT core if not in the bootloader.
>> 
>> <snarky comment>So when we at beagleboard.org handled the beagleboard and beagleboard xM expansionboards in the bootloader (detection, muxing, etc) we were told it was wrong and we should handle it in the kernel and if we waited a bit, DT would solve everything. And now that we actually handle it in the kernel you are saying that it is wrong and we should handle it in the bootloader and that DT won't solve everything. I guess someone will now tell us that uEFI will fix everything.</snarky comment>
> 
> Well, at that time I guess people were not that aware of the DT limitation.
> In fact at that time, this kind of stuff were done by bootloader.
> 
> But I'm quite sure uEFI will fix that :-)
> 
> The point being, there is indeed no proper solution as of today beside the bootloader hack.
> 
>> Apart from that, you and others still fail to tell us how you want to handle a user (or customer) that buys a beaglebone, an LCD cape, a weatherstation cape and a geiger counter cape and plugs those together and powers them on. With the evil TI vendor tree and extra patches the boardfile reads the eeproms and tries its best to instantiate all the platform data. One of the capes won't have working LEDs since appending to the leds-gpio struct is pretty much impossible in this situation. But it gets a picture on the screen, blinks LEDs and does largely what the user expects.
> 
> As I told Panto, it is not like it is a straightforward problem. My point is still valid, this is a generic issue that should not be fixed using some capebus hack.
> 
>> With capebus we get:
>> 
>> 1) da8xx-fb which lacks DT bindingsp[1] receives plaftorm data to match the LCD
>> 2) the i2c sensors on the weathercape are registered
>> 3) the one-wire bus on the weathercape gets registered
>> 4) the LEDs on the lcd cape get registered5
>> 5) the LED on the geigercape gets registered and adds a custom trigger
>> 6) the ADC, which again, lacks DT bindings[2], receives plaftorm data and a custom IIO binding
>> 7) pinctrl sets the pinmuxes for the above
>> 
>> In other words: plug & play, even for devices with drivers that are still lacking DT support.
>> 
>> Now please explain to me why you think it's such an awesome idea that the users will need to find the right dtsi files (multiple revisions of the lcd cape exist), include them in the dts, run dtc, add a few missing semicolons, run dtc again, copy it over to /boot and reboot to have a change of making it work?
> 
> Because, anyway, any new complex capes will require a new DTS to handle it if it requires some parameters.
> The whole issue is the lack of discoverable bus, and there is nothing you can do to fix that.
> 
>> I know you can't escape that for new or custom capes, but I do want all the capes my company assembles work out of the box. (Cross)compiling a kernel is a bridge too far for 95% of the intended audience.
> 
> The whole point of DTS is that you don't have to recompile to describe a new board. So why do you want to compile a kernel?
> 
>> With capebus most capes can be supported by editing the DTS, your bootloader proposal involves updating u-boot for every new cape as well. That is downright scary. The RMA department will get flooded.
> 
> Fair enough, hacking boot loader sucks.
> 
>> More importantly: capebus is generic enough to work on beagleboard, beagleboard xm, panda, panda es and even raspberry-pi. Basically on any DT capable platform.
> 
> Then make it part of DT core.
> 
> If you do want a pure dynamic stuff that minimize rebuild kernel, potentially loadable module and loadable DTB sub tree makes more sense than the capebus infrastructure.
> 
> The whole point here is to be able to build a DT dynamically.
> 
> We can boot with the default one.
> 
> beaglebone.dts:
> /{
> 	i2c@0 {
> 		status = "disabled";
> 	};
> 
> 	spi@0 {
> 		status = "disabled";
> 	};
> 
> 	gmpc {
> 		status = "disabled";
> 	};
> }
> 
> 
> And then later load that one using a bone-cape driver that can detect revision.
> 
> super-cape.dts:
> {
> 	i2c@0 {
> 		toaster@42 {
> 			blabla...
> 		};
> 	};
> 
> 	gmpc {
> 		mega-fpga@42 {
> 			blabla...
> 		};
> 	};
> }
> 
> And then use the standard DT support to create later the platform_device that does represent the new super-cape devices.
> 

We know this is the ideal case. In fact that's the long term goal and we had internal discussions about it.

DT is nowhere near being able to do it.

That would require the introduction of a DT object file format with aliases being capable to be
resolved dynamically. We're talking about major changes here in the way DT files are being compiled and
used in practice.

So yes, in an ideal world that would be great. We're nowhere near close today unfortunately.

Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic 
DT core.

The question is that this will take time, with no guarantees that this would be acceptable from
the device tree maintainers. So I am putting them in the CC list, to see what they think about it.

> Regards,
> Benoit
> 

Regards

-- Pantelis

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

Hi Alan,

On Nov 2, 2012, at 1:21 PM, Alan Cox wrote:

>>>> Fair enough. But there's no such thing a 'hotplug enumeration
>>>> construct' in Linux yet, and a bus is the closest thing to it. It does
>>>> take advantage of the nice way device code matches drivers and devices
>>>> though.
> 
> A bus is the wrong construct. You need something to add devices onto the
> busses. You can do that. The Intel SFI layer on phones for example
> enumerates devices through a firmware table set and creates them on the
> right actual physical bus not on their own fake one.
> 
> It's not hotplug in the sense that the phone happens be a fixed
> configuration but it does support hotplug behaviour because the order of
> the drivers and enumeration is undefined (and both orders work).

I see. The firmware table format is similar to our case where we use DT.
We don't have to come up with another firmware table format, like SFI here.
The problem is that we have to inject DT fragments to the kernel's runtime
device tree, at some arbitrary point in time, after the cape probing is complete.
Not only that, the DT fragment for each device is not self contained; it contains
references to other nodes which have to be resolved in runtime.

Let me demonstrate what the problem is - take for example the very simple
example of a (non-versioned) cape, with a single I2C device.

------8<-----------8<--------------
Kernel's boot DT:

i2c2: i2c@4819c000 {
        compatible = "ti,omap4-i2c";
        #address-cells = <1>;
        #size-cells = <0>;
        ti,hwmods = "i2c3";     /* TODO: Fix hwmod */
        reg = <0x4819c000 0x1000>;
        interrupt-parent = <&intc>;
        interrupts = <30>;

        status = "on-demand";	/* there's no on-demand status property defined yet */
				/* means that it will be set to "okay" when a i2c   */
				/* device node turn it to "okay"                    */
				/* we can't enable it at boot because of possible   */
				/* conflicts with other peripherals                 */ 
        pinctrl-names = "default";
        pinctrl-0 = <&i2c2_pins>;

        clock-frequency = <100000>;

};
------8<-----------8<--------------

A cape DT fragment (probable syntax) :

i2c2-devices {
	parent = <&i2c2>;

        /* Ambient light sensor */
         tsl2550@39 {
         	compatible = "tsl,tsl2550";
                reg = <0x39>;
        };      
};

------8<-----------8<--------------

The i2c2 alias cannot be resolved at compile time; there has to be

a) A DT object format where unresolved aliases (symbols) are tracked
b) A runtime DT linker that will resolve the alias, and will insert the
   i2c2-devices child nodes as children in the i2c2 node.
c) A method to trigger platform device creation for the child nodes just
   inserted.

DT core changes aren't bound to be easily accepted, so without having a clear
signal from the DT maintainers that they would consider such a method people
are just hesitant to go down this road.


> 
>>>> 
>>>> I'm afraid that having the I2C/SPI drivers doing the detection won't
>>>> work.  The capes can have arbitrary I2C/SPI devices (and even more
>>>> weird components).  There is no way to assure for example that the I2C
>>>> device responding to address 'foo' in cape A is the same I2C device
>>>> responding to the same address in cape B.
>>> 
>>> your ->detect() method should take care of that.
>> 
>> There isn't some magical serial number in I²C devices that a
>> ->detect() method can read and the implementation of I²C is somewhat
> 
> It doesn't matter.
> 
> What you are basically talking about is
> 
> 
> 	cape layer
> 		- wtf is this
> 		- how do I plumb it
> 
> 		- create device nodes with correct name for
> 		  binding, address etc on the right bus
> 
> 
> 	i2c layer
> 		- ooh a new i2c device
> 		- probe as indicated by device name
> 		- attach correct driver
> 
> 
> Architecturally its possible you want to make some caps MFDs if they have
> their own bus heirarchies etc but generally I doubt it.
> 
> 
> Take a look at arch/x86/platform/mrst/mrst.c. It's a specific example of
> a platform which parses tables and attaches devices to the right physical
> bus in a manner they can be reliably probed even when the device has no
> sane autodetect.
> 
> Alan

Regards

-- Pantelis

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Kevin Hilman]

On 11/02/2012 09:43 AM, Pantelis Antoniou wrote:
[...]

>>
>> And then use the standard DT support to create later the platform_device that does represent the new super-cape devices.
>>
>
> We know this is the ideal case. In fact that's the long term goal and we had internal discussions about it.

Since you already had internal discussions about this, it would have 
been a great help in avoiding lots of this discussion if you would've 
summarized this ideal case from the beginning, then describe the 
weaknesses and limitations of DT for handling hotplug/dynamic devices 
and thus the reasoning behind creating capebus.  Now it's taken this 
long thread for others to try to convince you about something you 
already knew to be the ideal case.  Seems a bit wasteful.

Kernel development typically works by building/extending infrastructure 
that is already there.  Only when it's obvious that the current 
infrastructure cannot be extended for a new kind of usage do we build 
new infrastruture.

In this case, DT is the obvious infrastructure that needs extending.  At 
least we can all agree on that, for starters.

> DT is nowhere near being able to do it.
>
> That would require the introduction of a DT object file format with aliases being capable to be
> resolved dynamically. We're talking about major changes here in the way DT files are being compiled and
> used in practice.
>
> So yes, in an ideal world that would be great. We're nowhere near close today unfortunately.
>
> Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
> then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic
> DT core.
>
> The question is that this will take time, with no guarantees that this would be acceptable from
> the device tree maintainers. So I am putting them in the CC list, to see what they think about it.

Since this thread has already ventured into the weeds a few times, I 
would suggest that you summarize the DT limitations (focusing on they 
dynamic/hotplug needs) and start a thread on devicetree-discuss, so that 
the DT maintainers can be helpful without having to follow this thread.

IMO, the path forward is clear (though probably longer than you would 
like):  Let's first try and extend DT infrastructure.  If that is 
obviously going nowhere, and DT maintainers are against it.  Then, let's 
revisit capebus.

Kevin

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Fri, Nov 2, 2012 at 8:43 AM, Pantelis Antoniou
<panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org> wrote:
> Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
> then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic
> DT core.
>
> The question is that this will take time, with no guarantees that this would be acceptable from
> the device tree maintainers. So I am putting them in the CC list, to see what they think about it.

This is actually exactly the direction I want to go with DT, which the
ability to load supplemental DT data blobs from either a kernel module
or userspace using the firmware loading infrastructure.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Fri, Nov 2, 2012 at 12:32 PM, Pantelis Antoniou
<panto@antoniou-consulting.com> wrote:
> The i2c2 alias cannot be resolved at compile time; there has to be
>
> a) A DT object format where unresolved aliases (symbols) are tracked
> b) A runtime DT linker that will resolve the alias, and will insert the
>    i2c2-devices child nodes as children in the i2c2 node.
> c) A method to trigger platform device creation for the child nodes just
>    inserted.
>
> DT core changes aren't bound to be easily accepted, so without having a clear
> signal from the DT maintainers that they would consider such a method people
> are just hesitant to go down this road.

I do agree and will accept such a method.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

On Nov 5, 2012, at 1:22 AM, Grant Likely wrote:

> On Fri, Nov 2, 2012 at 8:43 AM, Pantelis Antoniou
> <panto@antoniou-consulting.com> wrote:
>> Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
>> then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic
>> DT core.
>> 
>> The question is that this will take time, with no guarantees that this would be acceptable from
>> the device tree maintainers. So I am putting them in the CC list, to see what they think about it.
> 
> This is actually exactly the direction I want to go with DT, which the
> ability to load supplemental DT data blobs from either a kernel module
> or userspace using the firmware loading infrastructure.
> 
> g.

Hi Grant,

That's pretty much our use case.

Regards

-- Pantelis

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Mon, Nov 5, 2012 at 1:25 PM, Pantelis Antoniou
<panto@antoniou-consulting.com> wrote:
>
> On Nov 5, 2012, at 1:22 AM, Grant Likely wrote:
>
>> On Fri, Nov 2, 2012 at 8:43 AM, Pantelis Antoniou
>> <panto@antoniou-consulting.com> wrote:
>>> Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
>>> then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic
>>> DT core.
>>>
>>> The question is that this will take time, with no guarantees that this would be acceptable from
>>> the device tree maintainers. So I am putting them in the CC list, to see what they think about it.
>>
>> This is actually exactly the direction I want to go with DT, which the
>> ability to load supplemental DT data blobs from either a kernel module
>> or userspace using the firmware loading infrastructure.
>>
>> g.
>
> Hi Grant,
>
> That's pretty much our use case.
>
> Regards

Good. I'm about 80% though putting together a project plan of what is
required to implement this. I'll post it for RFC shortly. I would
appreciate feedback and help on flushing out the design.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Tony Lindgren]

* Grant Likely <grant.likely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org> [121105 06:36]:
> On Mon, Nov 5, 2012 at 1:25 PM, Pantelis Antoniou
> <panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org> wrote:
> >
> > On Nov 5, 2012, at 1:22 AM, Grant Likely wrote:
> >
> >> On Fri, Nov 2, 2012 at 8:43 AM, Pantelis Antoniou
> >> <panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org> wrote:
> >>> Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
> >>> then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic
> >>> DT core.
> >>>
> >>> The question is that this will take time, with no guarantees that this would be acceptable from
> >>> the device tree maintainers. So I am putting them in the CC list, to see what they think about it.
> >>
> >> This is actually exactly the direction I want to go with DT, which the
> >> ability to load supplemental DT data blobs from either a kernel module
> >> or userspace using the firmware loading infrastructure.
> >>
> >> g.
> >
> > Hi Grant,
> >
> > That's pretty much our use case.
> >
> > Regards
> 
> Good. I'm about 80% though putting together a project plan of what is
> required to implement this. I'll post it for RFC shortly. I would
> appreciate feedback and help on flushing out the design.

Great, sounds like almost-a-plan then :)

Regards,

Tony

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

Hi Grant,

On Nov 5, 2012, at 1:37 AM, Grant Likely wrote:

> On Fri, Nov 2, 2012 at 12:32 PM, Pantelis Antoniou
> <panto@antoniou-consulting.com> wrote:
>> The i2c2 alias cannot be resolved at compile time; there has to be
>> 
>> a) A DT object format where unresolved aliases (symbols) are tracked
>> b) A runtime DT linker that will resolve the alias, and will insert the
>>   i2c2-devices child nodes as children in the i2c2 node.
>> c) A method to trigger platform device creation for the child nodes just
>>   inserted.
>> 
>> DT core changes aren't bound to be easily accepted, so without having a clear
>> signal from the DT maintainers that they would consider such a method people
>> are just hesitant to go down this road.
> 
> I do agree and will accept such a method.
> 
> g.

Understood.

I'll think about the issues and come up with a design for the format, but
let's talk about this in the open for a while.

I don't want to modify the DTB format, in order to avoid impacting any other
DT users.

So would something like this work for you?

-----8<--------8<--------

/* booting DT (all symbols are resolved) */
/* this is a valid DTB today */

/ {

	i2c2: i2c@4819c000 { 
		compatible = "ti,omap4-i2c";
		#address-cells = <1>;
		#size-cells = <0>;
		ti,hwmods = "i2c3"; /* TODO: Fix hwmod */
		reg = <0x4819c000 0x1000>;
		interrupt-parent = <&intc>;
		interrupts = <30>;
		status = "on-demand";
	}; 
	
	/* .... others nodes .... */

	/* the symbols node can be generated by the DTC */
	symbols {
		compatible = "dt,symbols";

		symbol@0 {
			name = "i2c2";
			phandle = <&i2c2>;
		};

		...
	};
};

-----8<--------8<--------

/* fragment DT (not all symbols are resolved) */

/ {
	i2c2-append {
		compatible = "dt,append";
		#address-cells = <1>;
		#size-cells = <0>;

		append-at = <&i2c2>;	/* offset F00, either -1 (end of list) */
					/* or next offset in the list of i2c2 refs. */

		/* Ambient light sensor */ 
		tsl2550@39 {
			compatible = "tsl,tsl2550";
			reg = <0x39>;
		};
	};
};

-----8<--------8<--------

The <&i2c2> value can't be resolved at compile time.

We can use a simple linked list method for keeping track of the
locations in the fragment which we need to resolve, terminating with -1.

The compiler can add the following nodes to mark the entries requiring
resolution.

For example i2c2...

dt-resolution {
	i2c2 = <0xf00>; /* first offset in the dtb fragment requiring */
			/* resolution of i2c2 */
	....
};

The dt-resolution node, can be appended to the dt fragment contents and
can be generated by the compiler.

Upon injection of the DT fragment the in-kernel loader can perform
symbol resolution and replace the phandles with the in-kernel values.

I know there's an aliases node, but I don't know if it's wise to modify it.

What do you think?

Regards

-- Pantelis

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Rob Herring]

On 11/05/2012 08:34 AM, Grant Likely wrote:
> On Mon, Nov 5, 2012 at 1:25 PM, Pantelis Antoniou
> <panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org> wrote:
>>
>> On Nov 5, 2012, at 1:22 AM, Grant Likely wrote:
>>
>>> On Fri, Nov 2, 2012 at 8:43 AM, Pantelis Antoniou
>>> <panto-wVdstyuyKrO8r51toPun2/C9HSW9iNxf@public.gmane.org> wrote:
>>>> Assuming that we do work on a DT object format, and that the runtime resolution mechanism is approved,
>>>> then I agree that this part of the capebus patches can be dropped and the functionality assumed by generic
>>>> DT core.
>>>>
>>>> The question is that this will take time, with no guarantees that this would be acceptable from
>>>> the device tree maintainers. So I am putting them in the CC list, to see what they think about it.
>>>
>>> This is actually exactly the direction I want to go with DT, which the
>>> ability to load supplemental DT data blobs from either a kernel module
>>> or userspace using the firmware loading infrastructure.
>>>
>>> g.
>>
>> Hi Grant,
>>
>> That's pretty much our use case.
>>
>> Regards
> 
> Good. I'm about 80% though putting together a project plan of what is
> required to implement this. I'll post it for RFC shortly. I would
> appreciate feedback and help on flushing out the design.

The FPGA folks are also interested in dynamic DT configuration. There's
been some discussion and postings on the DT list in the last few months
you may have missed. It's maybe not completely the same use case, but
there is some overlap here.

Rob

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Mon, Nov 5, 2012 at 3:37 PM, Pantelis Antoniou
<panto@antoniou-consulting.com> wrote:
> Hi Grant,
>
> On Nov 5, 2012, at 1:37 AM, Grant Likely wrote:
>
>> On Fri, Nov 2, 2012 at 12:32 PM, Pantelis Antoniou
>> <panto@antoniou-consulting.com> wrote:
>>> The i2c2 alias cannot be resolved at compile time; there has to be
>>>
>>> a) A DT object format where unresolved aliases (symbols) are tracked
>>> b) A runtime DT linker that will resolve the alias, and will insert the
>>>   i2c2-devices child nodes as children in the i2c2 node.
>>> c) A method to trigger platform device creation for the child nodes just
>>>   inserted.
>>>
>>> DT core changes aren't bound to be easily accepted, so without having a clear
>>> signal from the DT maintainers that they would consider such a method people
>>> are just hesitant to go down this road.
>>
>> I do agree and will accept such a method.
>>
>> g.
>
> Understood.
>
> I'll think about the issues and come up with a design for the format, but
> let's talk about this in the open for a while.

Agreed. I'm planning to post my first draft/RFC tonight.

> I don't want to modify the DTB format, in order to avoid impacting any other
> DT users.

Ditto. It should be a direct extension.

> So would something like this work for you?
[...]
> The dt-resolution node, can be appended to the dt fragment contents and
> can be generated by the compiler.
>
> Upon injection of the DT fragment the in-kernel loader can perform
> symbol resolution and replace the phandles with the in-kernel values.
>
> I know there's an aliases node, but I don't know if it's wise to modify it.
>
> What do you think?

If I'm reading your intent correctly, your primary worry is the
problem of matching up phandles from the base dt with (potentially
different) phandle values in the overlay. This solution solves it by
encoding all of the phandle locations as offsets from the start of the
file into the dt-resolution node. Correct? My concern with this
approach is that it is fragile. Any changes to the fragment file, such
as to add extra properties or nodes, or even to repack the tree will
break all the offsets; probably silently.

It would be less fragile if each property containing phandles had some
kind of .<prop>-phandle-offsets companion property that listed the
phandles that need to be fixed up as an offset to the beginning of
only that properties' data. Better, in that modifying the tree won't
break the links, but I still worry that it is fragile and possibly too
complex.

However, the problem is based on the assumption that phandles are
effectively random and could change ever time the tree is recompiled.
Well, what if they weren't? What if dtc generated phandles using a
hash of the node full name so that the value changed rarely? Also,
what if the format was oriented around detecting if the phandles don't
match instead of fixing things up? The solution becomes a lot simpler
if the parser only has to verify that the referenced phandles already
exist at the right path in the tree.

Something like this:

/include/ "base-file.dts"  /* include might not be the right syntax here */
&i2c0 {         /* i2c0 resolved by label */
        touchpad@10 {
                compatible = "acme,touchpad";
                reg = <0x10>;
                interrupt-parent = <&intc>;
                interrupts = <100>;
        };
};

And the generated overlay dtb may look like this:

/ {
        .readonly;
        interrupt-controller@0x10005000 {
                .readonly;
                phandle = <0x1234>;
        };

        peripheral-bus {
                .readonly;
                i2c@20001000 {
                        touchpad@10 {
                                compatible = "acme,touchpad";
                                reg = <0x10>;
                                interrupt-parent = <0x1234>;
                                interrupts = <100>;
                        };
                };
        };
};

Which is obviously missing a bunch of information for the rest of the
system, but contains enough data to ensure that the path to the touchpad
node is valid and that the phandle has not changed.

This handles many of the use cases, but it assumes that an overlay is
board specific. If it ever is required to support multiple base boards
with a single overlay file then there is a problem. The .dtb overlays
generated in this manor cannot handle different phandles or nodes that
are in a different place. On the other hand, the overlay source files
should have no problem being compiled for multiple targets, so maybe
it isn't an issue. Plus if dtc is installed on the target, then the
live tree from /proc can be used as the reference when compiling the
overlay.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Mon, Nov 5, 2012 at 3:56 PM, Rob Herring <robherring2@gmail.com> wrote:
> On 11/05/2012 08:34 AM, Grant Likely wrote:
>> Good. I'm about 80% though putting together a project plan of what is
>> required to implement this. I'll post it for RFC shortly. I would
>> appreciate feedback and help on flushing out the design.
>
> The FPGA folks are also interested in dynamic DT configuration. There's
> been some discussion and postings on the DT list in the last few months
> you may have missed. It's maybe not completely the same use case, but
> there is some overlap here.

It should be very similar. I'll see if I can dig up those conversations.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

Hi Grant,

On Nov 5, 2012, at 8:10 PM, Grant Likely wrote:

> On Mon, Nov 5, 2012 at 3:37 PM, Pantelis Antoniou
> <panto@antoniou-consulting.com> wrote:
>> Hi Grant,
>> 
>> On Nov 5, 2012, at 1:37 AM, Grant Likely wrote:
>> 
>>> On Fri, Nov 2, 2012 at 12:32 PM, Pantelis Antoniou
>>> <panto@antoniou-consulting.com> wrote:
>>>> The i2c2 alias cannot be resolved at compile time; there has to be
>>>> 
>>>> a) A DT object format where unresolved aliases (symbols) are tracked
>>>> b) A runtime DT linker that will resolve the alias, and will insert the
>>>>  i2c2-devices child nodes as children in the i2c2 node.
>>>> c) A method to trigger platform device creation for the child nodes just
>>>>  inserted.
>>>> 
>>>> DT core changes aren't bound to be easily accepted, so without having a clear
>>>> signal from the DT maintainers that they would consider such a method people
>>>> are just hesitant to go down this road.
>>> 
>>> I do agree and will accept such a method.
>>> 
>>> g.
>> 
>> Understood.
>> 
>> I'll think about the issues and come up with a design for the format, but
>> let's talk about this in the open for a while.
> 
> Agreed. I'm planning to post my first draft/RFC tonight.
> 
>> I don't want to modify the DTB format, in order to avoid impacting any other
>> DT users.
> 
> Ditto. It should be a direct extension.
> 

Avoiding mobs of angry users with broken bootloader/kernel combos is a good
thing...

>> So would something like this work for you?
> [...]
>> The dt-resolution node, can be appended to the dt fragment contents and
>> can be generated by the compiler.
>> 
>> Upon injection of the DT fragment the in-kernel loader can perform
>> symbol resolution and replace the phandles with the in-kernel values.
>> 
>> I know there's an aliases node, but I don't know if it's wise to modify it.
>> 
>> What do you think?
> 
> If I'm reading your intent correctly, your primary worry is the
> problem of matching up phandles from the base dt with (potentially
> different) phandle values in the overlay. This solution solves it by
> encoding all of the phandle locations as offsets from the start of the
> file into the dt-resolution node. Correct? My concern with this
> approach is that it is fragile. Any changes to the fragment file, such
> as to add extra properties or nodes, or even to repack the tree will
> break all the offsets; probably silently.
> 

Yes, this will not survive modification of the fragment file.
For the use case I'm targeting the DT fragment is going to be quite
minimal, a few tens of nodes at the most. Modification of a single
fragment file is not expected.

Modifications to the base tree, those would work just fine. The base
tree by definition will not have external references.

> It would be less fragile if each property containing phandles had some
> kind of .<prop>-phandle-offsets companion property that listed the
> phandles that need to be fixed up as an offset to the beginning of
> only that properties' data. Better, in that modifying the tree won't
> break the links, but I still worry that it is fragile and possibly too
> complex.

I believe this is considerably more complex, without fixing the 
fragility problem.

> 
> However, the problem is based on the assumption that phandles are
> effectively random and could change ever time the tree is recompiled.
> Well, what if they weren't? What if dtc generated phandles using a
> hash of the node full name so that the value changed rarely? Also,
> what if the format was oriented around detecting if the phandles don't
> match instead of fixing things up? The solution becomes a lot simpler
> if the parser only has to verify that the referenced phandles already
> exist at the right path in the tree.
> 
> Something like this:
> 
> /include/ "base-file.dts"  /* include might not be the right syntax here */
> &i2c0 {         /* i2c0 resolved by label */
>        touchpad@10 {
>                compatible = "acme,touchpad";
>                reg = <0x10>;
>                interrupt-parent = <&intc>;
>                interrupts = <100>;
>        };
> };
> 
> And the generated overlay dtb may look like this:
> 
> / {
>        .readonly;
>        interrupt-controller@0x10005000 {
>                .readonly;
>                phandle = <0x1234>;
>        };
> 
>        peripheral-bus {
>                .readonly;
>                i2c@20001000 {
>                        touchpad@10 {
>                                compatible = "acme,touchpad";
>                                reg = <0x10>;
>                                interrupt-parent = <0x1234>;
>                                interrupts = <100>;
>                        };
>                };
>        };
> };
> 
> Which is obviously missing a bunch of information for the rest of the
> system, but contains enough data to ensure that the path to the touchpad
> node is valid and that the phandle has not changed.

I see what you mean. It will work, provided you 'link' (I don't know what
the proper terminology is) against the correct board dts file.
> 
> This handles many of the use cases, but it assumes that an overlay is
> board specific. If it ever is required to support multiple base boards
> with a single overlay file then there is a problem. The .dtb overlays
> generated in this manor cannot handle different phandles or nodes that
> are in a different place. On the other hand, the overlay source files
> should have no problem being compiled for multiple targets, so maybe
> it isn't an issue. Plus if dtc is installed on the target, then the
> live tree from /proc can be used as the reference when compiling the
> overlay.

My worry is that this format is dependent on linking against the board
DTS file. One of the ideas thrown around here was that it might make
sense to store the DTB fragment in the EEPROM of the device. 

In that case you have a OS independent hardware description, which can 
be even used even by the bootloader to access devices it knows not about
at compile time.

Other than that, I have no other objections.

> 
> g.

Regards

-- Pantelis

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Mon, Nov 5, 2012 at 7:54 PM, Pantelis Antoniou
<panto@antoniou-consulting.com> wrote:
>> This handles many of the use cases, but it assumes that an overlay is
>> board specific. If it ever is required to support multiple base boards
>> with a single overlay file then there is a problem. The .dtb overlays
>> generated in this manor cannot handle different phandles or nodes that
>> are in a different place. On the other hand, the overlay source files
>> should have no problem being compiled for multiple targets, so maybe
>> it isn't an issue. Plus if dtc is installed on the target, then the
>> live tree from /proc can be used as the reference when compiling the
>> overlay.
>
> My worry is that this format is dependent on linking against the board
> DTS file. One of the ideas thrown around here was that it might make
> sense to store the DTB fragment in the EEPROM of the device.

Right, that wouldn't work well if the base DT changed, or if a
BeagleBone2 is released that has the same header configuration, but
different backing devices. It would be nice to have a solution for
that.

> In that case you have a OS independent hardware description, which can
> be even used even by the bootloader to access devices it knows not about
> at compile time.
>
> Other than that, I have no other objections.

I'm open to suggestions if anyone has any. I have not objections to a
fixup approach, but I'm not comfortable with anything that is fragile
to modifications to the fragment.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Joel A Fernandes]

Hi Grant,

On Mon, Nov 5, 2012 at 2:14 PM, Grant Likely <grant.likely@secretlab.ca> wrote:
> On Mon, Nov 5, 2012 at 7:54 PM, Pantelis Antoniou
> <panto@antoniou-consulting.com> wrote:
>>> This handles many of the use cases, but it assumes that an overlay is
>>> board specific. If it ever is required to support multiple base boards
>>> with a single overlay file then there is a problem. The .dtb overlays
>>> generated in this manor cannot handle different phandles or nodes that
>>> are in a different place. On the other hand, the overlay source files
>>> should have no problem being compiled for multiple targets, so maybe
>>> it isn't an issue. Plus if dtc is installed on the target, then the
>>> live tree from /proc can be used as the reference when compiling the
>>> overlay.
>>
>> My worry is that this format is dependent on linking against the board
>> DTS file. One of the ideas thrown around here was that it might make
>> sense to store the DTB fragment in the EEPROM of the device.
>
> Right, that wouldn't work well if the base DT changed, or if a
> BeagleBone2 is released that has the same header configuration, but
> different backing devices. It would be nice to have a solution for
> that.
>
>> In that case you have a OS independent hardware description, which can
>> be even used even by the bootloader to access devices it knows not about
>> at compile time.
>>
>> Other than that, I have no other objections.
>
> I'm open to suggestions if anyone has any. I have not objections to a
> fixup approach, but I'm not comfortable with anything that is fragile
> to modifications to the fragment.

I am fairly new to the DT world so please bear with me, but how about
a method that resolves symbols the same way that the linux dynamic
linker does when shared libraries are loaded?

A separate table (similar to .PLT/GOT sections  in the ELF object
format) could be created when the fragment is loaded, and the phandle
references could be fixed to point to the table offsets during compile
time. This table would be a second level indirection and the kernel
would populate this table with the in-kernel values of the phandles
that the dt fragment refers to.

This might involve changes to the DT core, but as such, this method
wouldn't suffer from the fragility problem of either base or fragment
DT trees being modified.

The table itself could be added to the tree by the compiler, and the
phandles could point to it (fixed). such phandles could be marked for
special handling to facilitate the 1-level indirection.

What do you think about this?

Regards,
Joel

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

Joel A Fernandes <agnel.joel@gmail.com> wrote:

>Hi Grant,
>
>On Mon, Nov 5, 2012 at 2:14 PM, Grant Likely
><grant.likely@secretlab.ca> wrote:
>> I'm open to suggestions if anyone has any. I have not objections to a
>> fixup approach, but I'm not comfortable with anything that is fragile
>> to modifications to the fragment.
>
>I am fairly new to the DT world so please bear with me, but how about
>a method that resolves symbols the same way that the linux dynamic
>linker does when shared libraries are loaded?
>
>A separate table (similar to .PLT/GOT sections  in the ELF object
>format) could be created when the fragment is loaded, and the phandle
>references could be fixed to point to the table offsets during compile
>time. This table would be a second level indirection and the kernel
>would populate this table with the in-kernel values of the phandles
>that the dt fragment refers to.

That's an interesting idea that is worth exploring. That would make it possible to avoid a fixup stage, but it also means that any parsing code must know how to handle the got-like table. It won't be backwards compatible with existing tools. It also wouldn't easily support the case of firmware applying the overlay and passing the resulting tree to the kernel. Hmmm.... Not being backwards compatible at the data level is a big problem. I really want a method that can resolve back to the current data format or is a compatible extension of it.

>
>This might involve changes to the DT core, but as such, this method
>wouldn't suffer from the fragility problem of either base or fragment
>DT trees being modified.
>
>The table itself could be added to the tree by the compiler, and the
>phandles could point to it (fixed). such phandles could be marked for
>special handling to facilitate the 1-level indirection.

That's part of the problem. Property values are essentially anaonymous data. There is no mechanism currently for marking data such as indicate which data values are phandles. If there were then it would be a simple matter to find all the phandles and fix them up.

We could possibly add data type suppplementary properties to the tree to solve that problem. They would have to be optional for the base tree to retain backwards compatibility, but could be required on overlays.

g.

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Joel A Fernandes]

Hi Grant,

On Mon, Nov 5, 2012 at 5:58 PM, Grant Likely <grant.likely@secretlab.ca> wrote:
>
>
> Joel A Fernandes <agnel.joel@gmail.com> wrote:
>
>>Hi Grant,
>>
>>On Mon, Nov 5, 2012 at 2:14 PM, Grant Likely
>><grant.likely@secretlab.ca> wrote:
>>> I'm open to suggestions if anyone has any. I have not objections to a
>>> fixup approach, but I'm not comfortable with anything that is fragile
>>> to modifications to the fragment.
>>
>>I am fairly new to the DT world so please bear with me, but how about
>>a method that resolves symbols the same way that the linux dynamic
>>linker does when shared libraries are loaded?
>>
>>A separate table (similar to .PLT/GOT sections  in the ELF object
>>format) could be created when the fragment is loaded, and the phandle
>>references could be fixed to point to the table offsets during compile
>>time. This table would be a second level indirection and the kernel
>>would populate this table with the in-kernel values of the phandles
>>that the dt fragment refers to.
>
> That's an interesting idea that is worth exploring. That would make
> it possible to avoid a fixup stage, but it also means that any parsing
> code must know how to handle the got-like table. It won't be
> backwards compatible with existing tools. It also wouldn't easily
> support the case of firmware applying the overlay and passing the
> resulting tree to the kernel. Hmmm.... Not being backwards compatible
> at the data level is a big problem. I really want a method that can
> resolve back to the current data format or is a compatible extension
> of it.
>

Is it meaningful or feasible to make the table a part of the tree
structure itself? the table would initially be empty. If I'm right,
this is how the GOT tables in ELF objects that refer to unresolved
symbols in shared libraries are implemented as well, they are a part
of the object and are loaded into memory and fixed up. If the table is
a part of the DT data, the tools would then be able to parse such a
tree. We could enforce that the got-like tree would strictly follow a
predefined format.

Something along these lines would also avoid the need for a tree fix
up. Do you think this reduces the difficulty of backward compatibility
with the parsing tools and firmware loading?

>>
>>This might involve changes to the DT core, but as such, this method
>>wouldn't suffer from the fragility problem of either base or fragment
>>DT trees being modified.
>>
>>The table itself could be added to the tree by the compiler, and the
>>phandles could point to it (fixed). such phandles could be marked for
>>special handling to facilitate the 1-level indirection.
>
> That's part of the problem. Property values are essentially
> anaonymous data. There is no mechanism currently for marking data
> such as indicate which data values are phandles. If there were then
> it would be a simple matter to find all the phandles and fix them up.
>
> We could possibly add data type suppplementary properties to the tree
> to solve that problem. They would have to be optional for the base
> tree to retain backwards compatibility, but could be required on
> overlays.
>

Sure, so if we add data type supplementary properties to the tree to
indicate the data type as "indirect phandle", then kernel could refer
to the index in the got-like table to fetch the actual phandle address
(1-level of indirection), instead of directly using the address in the
data field.


Thanks and Regards,

Joel

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

Hi Joel,

On Nov 6, 2012, at 4:06 AM, Joel A Fernandes wrote:

> Hi Grant,
> 
> On Mon, Nov 5, 2012 at 5:58 PM, Grant Likely <grant.likely@secretlab.ca> wrote:
>> 
>> 
>> Joel A Fernandes <agnel.joel@gmail.com> wrote:
>> 
>>> Hi Grant,
>>> 
>>> On Mon, Nov 5, 2012 at 2:14 PM, Grant Likely
>>> <grant.likely@secretlab.ca> wrote:
>>>> I'm open to suggestions if anyone has any. I have not objections to a
>>>> fixup approach, but I'm not comfortable with anything that is fragile
>>>> to modifications to the fragment.
>>> 
>>> I am fairly new to the DT world so please bear with me, but how about
>>> a method that resolves symbols the same way that the linux dynamic
>>> linker does when shared libraries are loaded?
>>> 
>>> A separate table (similar to .PLT/GOT sections  in the ELF object
>>> format) could be created when the fragment is loaded, and the phandle
>>> references could be fixed to point to the table offsets during compile
>>> time. This table would be a second level indirection and the kernel
>>> would populate this table with the in-kernel values of the phandles
>>> that the dt fragment refers to.
>> 
>> That's an interesting idea that is worth exploring. That would make
>> it possible to avoid a fixup stage, but it also means that any parsing
>> code must know how to handle the got-like table. It won't be
>> backwards compatible with existing tools. It also wouldn't easily
>> support the case of firmware applying the overlay and passing the
>> resulting tree to the kernel. Hmmm.... Not being backwards compatible
>> at the data level is a big problem. I really want a method that can
>> resolve back to the current data format or is a compatible extension
>> of it.
>> 
> 
> Is it meaningful or feasible to make the table a part of the tree
> structure itself? the table would initially be empty. If I'm right,
> this is how the GOT tables in ELF objects that refer to unresolved
> symbols in shared libraries are implemented as well, they are a part
> of the object and are loaded into memory and fixed up. If the table is
> a part of the DT data, the tools would then be able to parse such a
> tree. We could enforce that the got-like tree would strictly follow a
> predefined format.
> 
> Something along these lines would also avoid the need for a tree fix
> up. Do you think this reduces the difficulty of backward compatibility
> with the parsing tools and firmware loading?

To be honest here, we are discussing a new object format. 
There are a few twists IMO.

a) We definitely, absolutely, don't want to introduce anything that will
risk compatibility with devices already out there. The binary format
for device trees that don't need the dynamic resolution features we're
talking about here, should be be usable for those older devices.

b) Device tree is flexible enough to store the additional data in it's own
node format. So we shouldn't have any kind of binary data tacked on; this
ties with a) - make sure that the binary format doesn't change.

c) There is no need (at least AFAIKT) of having any other resolution type
than a phandle of a node.

d) Finally, for some use-cases the problem is simplified by not having
all the features of a true dynamic linker. For example for the capebus
case the 'base' DTS won't have any references to any fragments. It is only
the fragments that have unresolved references and only to the 'base' DTS.

> 
>>> 
>>> This might involve changes to the DT core, but as such, this method
>>> wouldn't suffer from the fragility problem of either base or fragment
>>> DT trees being modified.
>>> 
>>> The table itself could be added to the tree by the compiler, and the
>>> phandles could point to it (fixed). such phandles could be marked for
>>> special handling to facilitate the 1-level indirection.
>> 
>> That's part of the problem. Property values are essentially
>> anaonymous data. There is no mechanism currently for marking data
>> such as indicate which data values are phandles. If there were then
>> it would be a simple matter to find all the phandles and fix them up.
>> 
>> We could possibly add data type suppplementary properties to the tree
>> to solve that problem. They would have to be optional for the base
>> tree to retain backwards compatibility, but could be required on
>> overlays.
>> 
> 
> Sure, so if we add data type supplementary properties to the tree to
> indicate the data type as "indirect phandle", then kernel could refer
> to the index in the got-like table to fetch the actual phandle address
> (1-level of indirection), instead of directly using the address in the
> data field.
> 

I'm fine with this.

> 
> Thanks and Regards,
> 
> Joel

Regards

-- Pantelis

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Grant Likely]

On Tue, Nov 6, 2012 at 8:14 AM, Pantelis Antoniou
<panto@antoniou-consulting.com> wrote:
> On Nov 6, 2012, at 4:06 AM, Joel A Fernandes wrote:
>> Sure, so if we add data type supplementary properties to the tree to
>> indicate the data type as "indirect phandle", then kernel could refer
>> to the index in the got-like table to fetch the actual phandle address
>> (1-level of indirection), instead of directly using the address in the
>> data field.
>>
>
> I'm fine with this.

But if the data about phandles is already in the tree, then the need
for a GOT simply goes away. The phandles can be fixed up directly and
it is so much better because it works with existing parsing code after
the merge is applied.

g.

Re: [PATCH 0/3] capebus moving omap_devices to mach-omap2

[Pantelis Antoniou]

Από το iPhone μου

6 Νοε 2012, 12:16, ο/η Grant Likely <grant.likely@secretlab.ca> έγραψε:

> On Tue, Nov 6, 2012 at 8:14 AM, Pantelis Antoniou
> <panto@antoniou-consulting.com> wrote:
>> On Nov 6, 2012, at 4:06 AM, Joel A Fernandes wrote:
>>> Sure, so if we add data type supplementary properties to the tree to
>>> indicate the data type as "indirect phandle", then kernel could refer
>>> to the index in the got-like table to fetch the actual phandle address
>>> (1-level of indirection), instead of directly using the address in the
>>> data field.
>> 
>> I'm fine with this.
> 
> But if the data about phandles is already in the tree, then the need
> for a GOT simply goes away. The phandles can be fixed up directly and
> it is so much better because it works with existing parsing code after
> the merge is applied.
> 

Either way works. It is your call after all. 
I agree that your method is simpler. 
> g.

Regards

-- Pantelis--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html