[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Tero Kristo]

On 17/04/18 17:49, Tony Lindgren wrote:
> * Tero Kristo <t-kristo@ti.com> [180417 09:36]:
>> In typical setup, you can boot a large number of different configs via:
>>
>> bootm 0x82000000#dra71-evm#nand#lcd-auo-g101evn01.0
>>
>> ... assuming the configs were named like that, and assuming they would be
>> compatible with each other. The am57xx-evm example provided is better, as
>> you can chain the different cameras to the available evm configs.
> 
> Why not just do it in the bootloader to put together the dtb?
> 
> Then for external devices, you could just pass info on the
> kernel cmdline with lcd=foo camera=bar if they cannot be
> detected over I2C.

(Added Linux ARM list to CC, this was not part of the original delivery.)

Ok trying to resurrect this thread a bit. Is there any kind of consensus 
how things like this should be handled? Should we add the DT overlay 
files to kernel tree or not?

Should we add any kind of build infra to kernel tree, and at what level 
would this be? Just DT overlay file building support, and drop the FIT 
build support as was proposed in this RFC series or...?

U-boot can obviously parse the base DTB + overlay DTB:s into a single 
DTB, but this is somewhat clumsy approach and is relatively error prone 
to get it right.

Building the FIT image post kernel build would also be possible, but who 
would be doing this, is there any need to get this done in generic 
manner or shall we just add SoC vendor specific tools for this?

-Tero
--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Rob Herring]

On Mon, May 21, 2018 at 09:57:54AM +0300, Tero Kristo wrote:
> On 17/04/18 17:49, Tony Lindgren wrote:
> > * Tero Kristo <t-kristo@ti.com> [180417 09:36]:
> > > In typical setup, you can boot a large number of different configs via:
> > > 
> > > bootm 0x82000000#dra71-evm#nand#lcd-auo-g101evn01.0
> > > 
> > > ... assuming the configs were named like that, and assuming they would be
> > > compatible with each other. The am57xx-evm example provided is better, as
> > > you can chain the different cameras to the available evm configs.
> > 
> > Why not just do it in the bootloader to put together the dtb?
> > 
> > Then for external devices, you could just pass info on the
> > kernel cmdline with lcd=foo camera=bar if they cannot be
> > detected over I2C.
> 
> (Added Linux ARM list to CC, this was not part of the original delivery.)
> 
> Ok trying to resurrect this thread a bit. Is there any kind of consensus how
> things like this should be handled? Should we add the DT overlay files to
> kernel tree or not?

IMO, yes.

> Should we add any kind of build infra to kernel tree, and at what level
> would this be? Just DT overlay file building support, and drop the FIT build
> support as was proposed in this RFC series or...?

I think I mentioned this already, but I expect that this is going to 
cause a number of conversions of dtsi + dtsi -> dtb into base dts and 
overlay(s) dts files. In doing so, we still need to be able to build the 
original, full dtb.

> U-boot can obviously parse the base DTB + overlay DTB:s into a single DTB,
> but this is somewhat clumsy approach and is relatively error prone to get it
> right.

Why? How is the kernel better?

> Building the FIT image post kernel build would also be possible, but who
> would be doing this, is there any need to get this done in generic manner or
> shall we just add SoC vendor specific tools for this?

I'll tell you up front, I'm not a fan of FIT image (nor uImage, 
Android boot image, $bootloader image). If you want a collection of 
files and some configuration data, use a filesystem and a text file.

Rob

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Tero Kristo]

On 22/05/18 23:01, Rob Herring wrote:
> On Mon, May 21, 2018 at 09:57:54AM +0300, Tero Kristo wrote:
>> On 17/04/18 17:49, Tony Lindgren wrote:
>>> * Tero Kristo <t-kristo@ti.com> [180417 09:36]:
>>>> In typical setup, you can boot a large number of different configs via:
>>>>
>>>> bootm 0x82000000#dra71-evm#nand#lcd-auo-g101evn01.0
>>>>
>>>> ... assuming the configs were named like that, and assuming they would be
>>>> compatible with each other. The am57xx-evm example provided is better, as
>>>> you can chain the different cameras to the available evm configs.
>>>
>>> Why not just do it in the bootloader to put together the dtb?
>>>
>>> Then for external devices, you could just pass info on the
>>> kernel cmdline with lcd=foo camera=bar if they cannot be
>>> detected over I2C.
>>
>> (Added Linux ARM list to CC, this was not part of the original delivery.)
>>
>> Ok trying to resurrect this thread a bit. Is there any kind of consensus how
>> things like this should be handled? Should we add the DT overlay files to
>> kernel tree or not?
> 
> IMO, yes.
> 
>> Should we add any kind of build infra to kernel tree, and at what level
>> would this be? Just DT overlay file building support, and drop the FIT build
>> support as was proposed in this RFC series or...?
> 
> I think I mentioned this already, but I expect that this is going to
> cause a number of conversions of dtsi + dtsi -> dtb into base dts and
> overlay(s) dts files. In doing so, we still need to be able to build the
> original, full dtb.

So you mean like breaking apart the existing .dts files? Are there any 
plans to get that done (I know the android folks talk about this but I 
don't like their idea.) If we do the split, how are we going to 
determine which dts + overlay files are required to get a specific DTB 
done? I actually wrote a tool for this purpose which parses the FIT 
image configurations and generates plain dtb files out of the info there 
if needed, but assuming FIT is abandoned then what...?

> 
>> U-boot can obviously parse the base DTB + overlay DTB:s into a single DTB,
>> but this is somewhat clumsy approach and is relatively error prone to get it
>> right.
> 
> Why? How is the kernel better?

I am mostly speaking about runtime applying of the overlays. If done 
build time, it is obviously on same level. If you apply the base DTS + 
overlays from u-boot prompt, it is not too much fun, and if there are 
any failures it just won't work, but don't really tell you why not.

> 
>> Building the FIT image post kernel build would also be possible, but who
>> would be doing this, is there any need to get this done in generic manner or
>> shall we just add SoC vendor specific tools for this?
> 
> I'll tell you up front, I'm not a fan of FIT image (nor uImage,
> Android boot image, $bootloader image). If you want a collection of
> files and some configuration data, use a filesystem and a text file.

Ok, thanks for your frank comments. I believe based on this feedback 
I'll try to modify this series into bare minimal overlay support to 
kernel, and have the post processing done elsewhere (either u-boot build 
or possibly completely separate tool.)

-Tero
--
Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Rob Herring]

On Wed, May 23, 2018 at 12:55 AM, Tero Kristo <t-kristo@ti.com> wrote:
> On 22/05/18 23:01, Rob Herring wrote:
>>
>> On Mon, May 21, 2018 at 09:57:54AM +0300, Tero Kristo wrote:
>>>
>>> On 17/04/18 17:49, Tony Lindgren wrote:
>>>>
>>>> * Tero Kristo <t-kristo@ti.com> [180417 09:36]:
>>>>>
>>>>> In typical setup, you can boot a large number of different configs via:
>>>>>
>>>>> bootm 0x82000000#dra71-evm#nand#lcd-auo-g101evn01.0
>>>>>
>>>>> ... assuming the configs were named like that, and assuming they would
>>>>> be
>>>>> compatible with each other. The am57xx-evm example provided is better,
>>>>> as
>>>>> you can chain the different cameras to the available evm configs.
>>>>
>>>>
>>>> Why not just do it in the bootloader to put together the dtb?
>>>>
>>>> Then for external devices, you could just pass info on the
>>>> kernel cmdline with lcd=foo camera=bar if they cannot be
>>>> detected over I2C.
>>>
>>>
>>> (Added Linux ARM list to CC, this was not part of the original delivery.)
>>>
>>> Ok trying to resurrect this thread a bit. Is there any kind of consensus
>>> how
>>> things like this should be handled? Should we add the DT overlay files to
>>> kernel tree or not?
>>
>>
>> IMO, yes.
>>
>>> Should we add any kind of build infra to kernel tree, and at what level
>>> would this be? Just DT overlay file building support, and drop the FIT
>>> build
>>> support as was proposed in this RFC series or...?
>>
>>
>> I think I mentioned this already, but I expect that this is going to
>> cause a number of conversions of dtsi + dtsi -> dtb into base dts and
>> overlay(s) dts files. In doing so, we still need to be able to build the
>> original, full dtb.
>
>
> So you mean like breaking apart the existing .dts files? Are there any plans
> to get that done (I know the android folks talk about this but I don't like
> their idea.) If we do the split, how are we going to determine which dts +
> overlay files are required to get a specific DTB done? I actually wrote a
> tool for this purpose which parses the FIT image configurations and
> generates plain dtb files out of the info there if needed, but assuming FIT
> is abandoned then what...?

There aren't any plans that I'm aware of. I'm just assuming there are
some cases in the tree. It would just be some additional build rules:

board-kit.dtb: soc-som.dtb expansion-board.dtbo
  fdtoverlay -o $@ -i $^

We'd need to come up with some way to express the input and output
files in a kbuild way. Perhaps similar to how multiple module files
are done.

Even if you are adding new platforms, having the above would still be
good. It captures what base the overlays apply to and tests that
applying actually works (rather than leaving that for the user to do
in the bootloader). FIT images also have or need that information, so
perhaps we can define a format that works as input for both kbuild and
FIT images.

>>> U-boot can obviously parse the base DTB + overlay DTB:s into a single
>>> DTB,
>>> but this is somewhat clumsy approach and is relatively error prone to get
>>> it
>>> right.
>>
>>
>> Why? How is the kernel better?
>
>
> I am mostly speaking about runtime applying of the overlays. If done build
> time, it is obviously on same level. If you apply the base DTS + overlays
> from u-boot prompt, it is not too much fun, and if there are any failures it
> just won't work, but don't really tell you why not.

Runtime, but still at boot time, right?

How is the kernel going to have better error handling? Is libfdt error
handling worse than the kernel's overlay code?

I do think we need to standardize how we tell bootloaders what
overlays to apply. I expect that we'll do that as part of EBBR spec[1]
though that is not part of the immediate focus.

Rob

[1] https://github.com/ARM-software/ebbr

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Tony Lindgren]

* Tero Kristo <t-kristo@ti.com> [180523 05:58]:
> On 22/05/18 23:01, Rob Herring wrote:
> > On Mon, May 21, 2018 at 09:57:54AM +0300, Tero Kristo wrote:
> > > Ok trying to resurrect this thread a bit. Is there any kind of consensus how
> > > things like this should be handled? Should we add the DT overlay files to
> > > kernel tree or not?
> > 
> > IMO, yes.

I too agree the overlay files should be in the kernel rather than
people having to try to figure out where to find them.

No idea how we'll verify they'll build though :) Maybe we
just need randconfig support for DT overlay files too?

Regards,

Tony

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Frank Rowand]

On 05/22/18 13:01, Rob Herring wrote:
> On Mon, May 21, 2018 at 09:57:54AM +0300, Tero Kristo wrote:
>> On 17/04/18 17:49, Tony Lindgren wrote:
>>> * Tero Kristo <t-kristo@ti.com> [180417 09:36]:
>>>> In typical setup, you can boot a large number of different configs via:
>>>>
>>>> bootm 0x82000000#dra71-evm#nand#lcd-auo-g101evn01.0
>>>>
>>>> ... assuming the configs were named like that, and assuming they would be
>>>> compatible with each other. The am57xx-evm example provided is better, as
>>>> you can chain the different cameras to the available evm configs.
>>>
>>> Why not just do it in the bootloader to put together the dtb?
>>>
>>> Then for external devices, you could just pass info on the
>>> kernel cmdline with lcd=foo camera=bar if they cannot be
>>> detected over I2C.
>>
>> (Added Linux ARM list to CC, this was not part of the original delivery.)
>>
>> Ok trying to resurrect this thread a bit. Is there any kind of consensus how
>> things like this should be handled? Should we add the DT overlay files to
>> kernel tree or not?
> 
> IMO, yes.

Agreed.

> 
>> Should we add any kind of build infra to kernel tree, and at what level
>> would this be? Just DT overlay file building support, and drop the FIT build
>> support as was proposed in this RFC series or...?
> 
> I think I mentioned this already, but I expect that this is going to 
> cause a number of conversions of dtsi + dtsi -> dtb into base dts and 
> overlay(s) dts files. In doing so, we still need to be able to build the 
> original, full dtb.
> 
>> U-boot can obviously parse the base DTB + overlay DTB:s into a single DTB,
>> but this is somewhat clumsy approach and is relatively error prone to get it
>> right.
> 
> Why? How is the kernel better?
> 
>> Building the FIT image post kernel build would also be possible, but who
>> would be doing this, is there any need to get this done in generic manner or
>> shall we just add SoC vendor specific tools for this?
> 
> I'll tell you up front, I'm not a fan of FIT image (nor uImage, 
> Android boot image, $bootloader image). If you want a collection of 
> files and some configuration data, use a filesystem and a text file.

My gut feel is that using a filesystem and a text file is the easier way
to create the boot info.  But that also makes applying the overlay(s)
during early Linux boot (at the point of FDT unflattening) impractical
(can't access the file system without a driver, the driver depends on
the devicetree, the devicetree depends upon the overlay).

> 
> Rob
> 

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Russell King - ARM Linux]

On Tue, May 29, 2018 at 10:05:37AM -0700, Frank Rowand wrote:
> On 05/22/18 13:01, Rob Herring wrote:
> > I'll tell you up front, I'm not a fan of FIT image (nor uImage, 
> > Android boot image, $bootloader image). If you want a collection of 
> > files and some configuration data, use a filesystem and a text file.

Me neither.

> My gut feel is that using a filesystem and a text file is the easier way
> to create the boot info.  But that also makes applying the overlay(s)
> during early Linux boot (at the point of FDT unflattening) impractical
> (can't access the file system without a driver, the driver depends on
> the devicetree, the devicetree depends upon the overlay).

Why do you want to apply overlays during the kernel boot?  The boot
loader should be providing the kernel with the merged DT to describe
the system that the kernel is running on - it's not the kernel's
job to put that together.

The whole point of DT is to make the kernel _less_ tied to the hardware
and more generic.  If we're going to introduce an entirely new set of
drivers into the kernel to "probe" the hardware to determine which
overlays are required, then that's really defeating the purpose of DT.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Frank Rowand]

On 05/29/18 10:33, Russell King - ARM Linux wrote:
> On Tue, May 29, 2018 at 10:05:37AM -0700, Frank Rowand wrote:
>> On 05/22/18 13:01, Rob Herring wrote:
>>> I'll tell you up front, I'm not a fan of FIT image (nor uImage, 
>>> Android boot image, $bootloader image). If you want a collection of 
>>> files and some configuration data, use a filesystem and a text file.
> 
> Me neither.
> 
>> My gut feel is that using a filesystem and a text file is the easier way
>> to create the boot info.  But that also makes applying the overlay(s)
>> during early Linux boot (at the point of FDT unflattening) impractical
>> (can't access the file system without a driver, the driver depends on
>> the devicetree, the devicetree depends upon the overlay).
> 
> Why do you want to apply overlays during the kernel boot?  The boot
> loader should be providing the kernel with the merged DT to describe
> the system that the kernel is running on - it's not the kernel's
> job to put that together.

I would much prefer that overlays get applied before the unflattened
devicetree is made available to the kernel instead of applying overlays
late in the boot (after many susbsystems and drivers are initialized)
or after boot.

If overlays are applied before the unflattened devicetree is made
available to the kernel then subsystems and drivers do not have to be
aware of the possibility of the data in the devicetree changing instead
of being invariant.  (I will call this "apply early overlays" later in
this email.)

There are use cases of overlays where the overlay is not available until
late in the boot, or after the boot completes.  I am not suggesting that
those use cases be ignored.  That is a completely different conversation.

If overlays are applied before the unflattened devicetree is made
available to the kernel, there are at least two obvious methods
(and I can think of at least one more) -- either the kernel does it
or the bootloader does it.  I do not see either of the two as being
the obvious and only correct choice.  If the kernel does it, then
there is no need for multiple bootloaders to implement the same code.
If the bootloaders do it, then there is no need for multiple kernels
(Linux, BSD, etc) to implement the same code.  I do not have a
strong opinion about whether the bootloaders or the kernels are a
better place to apply early overlays.


> The whole point of DT is to make the kernel _less_ tied to the hardware
> and more generic.  If we're going to introduce an entirely new set of
> drivers into the kernel to "probe" the hardware to determine which
> overlays are required, then that's really defeating the purpose of DT.

If you read my circular dependency sentence carefully, I said a kernel
driver is impractical for the task of getting the overlay.  So I agree.

If the kernel applies the overlay, an alternative to drivers is that the
kernel could receive the overlay(s) in a similar manner as the method it
currently receives the base devicetree.  (The overlay(s) could be
appended to the base devicetree, a wrapper could be placed around the
base devicetree and the overlay(s), or some other implementation.  There
are multiple ways to achieve the same result.)

[RFC 12/13] ARM: dts: ti: add dra71-evm FIT description file

[Russell King - ARM Linux]

On Tue, May 29, 2018 at 03:45:12PM -0700, Frank Rowand wrote:
> If the kernel applies the overlay, an alternative to drivers is that the
> kernel could receive the overlay(s) in a similar manner as the method it
> currently receives the base devicetree.  (The overlay(s) could be
> appended to the base devicetree, a wrapper could be placed around the

No wrappers, sorry.  We already have too many places in the Linux
kernel and its decompressor that try to work out whether a device
tree or ATAGs are present and how large, and that is done in assembly
code.  We don't need more complexity to deal with some kind of
wrapping of device trees as well.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 8.8Mbps down 630kbps up
According to speedtest.net: 8.21Mbps down 510kbps up