[OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build

[OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build

[Petr Kubizňák]

Add DT_FILES variable to allow the user of the class to select specific
dts files to build. This is useful for packages featuring dts files
for multiple machines.

Since many machine configs contain a list of dtb files
(e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.

Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
---
 meta/classes-recipe/devicetree.bbclass | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
index ed2a92e447..bd50d7fa1d 100644
--- a/meta/classes-recipe/devicetree.bbclass
+++ b/meta/classes-recipe/devicetree.bbclass
@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
 
 DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
 DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
-DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
+DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
 DT_FILES_PATH ?= "${S}"
+DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
+DT_FILES ?= ""
 
 DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
 DT_PADDING_SIZE ??= "0x3000"
@@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
     subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
 
 python devicetree_do_compile() {
+    import re
     includes = expand_includes("DT_INCLUDE", d)
+    dtfiles = d.getVar("DT_FILES").split()
+    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
     listpath = d.getVar("DT_FILES_PATH")
-    for dts in os.listdir(listpath):
+    for dts in dtfiles or os.listdir(listpath):
         dtspath = os.path.join(listpath, dts)
         try:
             if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):
-- 
2.30.2

Re: [OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build

[Denys Dmytriyenko]

On Fri, Apr 21, 2023 at 05:23:25PM +0200, Petr Kubizňák wrote:
> Add DT_FILES variable to allow the user of the class to select specific
> dts files to build. This is useful for packages featuring dts files
> for multiple machines.
> 
> Since many machine configs contain a list of dtb files
> (e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.

Any examples of this being used? Not really clear from the description what's 
this really used for...


> Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
> ---
>  meta/classes-recipe/devicetree.bbclass | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
> index ed2a92e447..bd50d7fa1d 100644
> --- a/meta/classes-recipe/devicetree.bbclass
> +++ b/meta/classes-recipe/devicetree.bbclass
> @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
>  
>  DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
>  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
> +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
>  DT_FILES_PATH ?= "${S}"
> +DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
> +DT_FILES ?= ""
>  
>  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
>  DT_PADDING_SIZE ??= "0x3000"
> @@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
>      subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>  
>  python devicetree_do_compile() {
> +    import re
>      includes = expand_includes("DT_INCLUDE", d)
> +    dtfiles = d.getVar("DT_FILES").split()
> +    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
>      listpath = d.getVar("DT_FILES_PATH")
> -    for dts in os.listdir(listpath):
> +    for dts in dtfiles or os.listdir(listpath):
>          dtspath = os.path.join(listpath, dts)
>          try:
>              if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):
> -- 
> 2.30.2

Re: [OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build

[Petr Kubizňák - 2N]

[-- Attachment #1: Type: text/plain, Size: 3538 bytes --]

Hi Denys,

> Any examples of this being used? Not really clear from the description what's
> this really used for...

I'm not sure what is unclear to you - just the commit message, or the patch itself?

An example could be beaglebone-yocto.conf. If someone decides to move the device tree files from kernel to a standalone repository, then it is enough (e.g.) to rename KERNEL_DEVICETREE to EXTERNAL_DEVICETREE in the machine conf, and make an assignment DT_FILES = "${EXTERNAL_DEVICETREE}" in the devicetree recipe.

So how about to replace the unclear sentence by:
To make DT_FILES consistent with KERNEL_DEVICETREE, the list works with both dts and dtb files.

Would that make more sense to you? Or do you think the example should be part of that commit message, too?

Cheers,
Petr

________________________________________
From: Denys Dmytriyenko <denis@denix.org>
Sent: Sunday, April 23, 2023 12:07 AM
To: Petr Kubizňák - 2N
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build

On Fri, Apr 21, 2023 at 05:23:25PM +0200, Petr Kubizňák wrote:
> Add DT_FILES variable to allow the user of the class to select specific
> dts files to build. This is useful for packages featuring dts files
> for multiple machines.
>
> Since many machine configs contain a list of dtb files
> (e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.

Any examples of this being used? Not really clear from the description what's
this really used for...


> Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
> ---
>  meta/classes-recipe/devicetree.bbclass | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
> index ed2a92e447..bd50d7fa1d 100644
> --- a/meta/classes-recipe/devicetree.bbclass
> +++ b/meta/classes-recipe/devicetree.bbclass
> @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
>
>  DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
>  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
> +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
>  DT_FILES_PATH ?= "${S}"
> +DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
> +DT_FILES ?= ""
>
>  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
>  DT_PADDING_SIZE ??= "0x3000"
> @@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
>      subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>
>  python devicetree_do_compile() {
> +    import re
>      includes = expand_includes("DT_INCLUDE", d)
> +    dtfiles = d.getVar("DT_FILES").split()
> +    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
>      listpath = d.getVar("DT_FILES_PATH")
> -    for dts in os.listdir(listpath):
> +    for dts in dtfiles or os.listdir(listpath):
>          dtspath = os.path.join(listpath, dts)
>          try:
>              if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):
> --
> 2.30.2

[-- Attachment #2: Type: text/html, Size: 5065 bytes --]

Re: [OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build

[Denys Dmytriyenko]

On Mon, Apr 24, 2023 at 08:48:24AM +0000, Petr Kubizňák - 2N wrote:
> Hi Denys,
> 
> > Any examples of this being used? Not really clear from the description what's
> > this really used for...
> 
> I'm not sure what is unclear to you - just the commit message, or the patch itself?
> 
> An example could be beaglebone-yocto.conf. If someone decides to move the 
> device tree files from kernel to a standalone repository, then it is enough 
> (e.g.) to rename KERNEL_DEVICETREE to EXTERNAL_DEVICETREE in the machine 
> conf, and make an assignment DT_FILES = "${EXTERNAL_DEVICETREE}" in the 
> devicetree recipe.
> 
> So how about to replace the unclear sentence by:

> To make DT_FILES consistent with KERNEL_DEVICETREE, the list works with both 
> dts and dtb files.

Thanks, this sounds much better, I believe.


> Would that make more sense to you? Or do you think the example should be 
> part of that commit message, too?
> 
> Cheers,
> Petr
> 
> ________________________________________
> From: Denys Dmytriyenko <denis@denix.org>
> Sent: Sunday, April 23, 2023 12:07 AM
> To: Petr Kubizňák - 2N
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core][PATCH v3] devicetree.bbclass: Allow selection of dts files to build
> 
> On Fri, Apr 21, 2023 at 05:23:25PM +0200, Petr Kubizňák wrote:
> > Add DT_FILES variable to allow the user of the class to select specific
> > dts files to build. This is useful for packages featuring dts files
> > for multiple machines.
> >
> > Since many machine configs contain a list of dtb files
> > (e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.
> 
> Any examples of this being used? Not really clear from the description what's
> this really used for...
> 
> 
> > Signed-off-by: Petr Kubizňák <kubiznak@2n.com>
> > ---
> >  meta/classes-recipe/devicetree.bbclass | 9 +++++++--
> >  1 file changed, 7 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/classes-recipe/devicetree.bbclass b/meta/classes-recipe/devicetree.bbclass
> > index ed2a92e447..bd50d7fa1d 100644
> > --- a/meta/classes-recipe/devicetree.bbclass
> > +++ b/meta/classes-recipe/devicetree.bbclass
> > @@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \
> >
> >  DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion."
> >  DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}"
> > -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)."
> > +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory."
> >  DT_FILES_PATH ?= "${S}"
> > +DT_FILES[doc] = "Space-separated list of dts or dtb files (relative to DT_FILES_PATH) to build. If empty, all dts files are built."
> > +DT_FILES ?= ""
> >
> >  DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot."
> >  DT_PADDING_SIZE ??= "0x3000"
> > @@ -125,9 +127,12 @@ def devicetree_compile(dtspath, includes, d):
> >      subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
> >
> >  python devicetree_do_compile() {
> > +    import re
> >      includes = expand_includes("DT_INCLUDE", d)
> > +    dtfiles = d.getVar("DT_FILES").split()
> > +    dtfiles = [ re.sub(r"\.dtbo?$", ".dts", dtfile) for dtfile in dtfiles ]
> >      listpath = d.getVar("DT_FILES_PATH")
> > -    for dts in os.listdir(listpath):
> > +    for dts in dtfiles or os.listdir(listpath):
> >          dtspath = os.path.join(listpath, dts)
> >          try:
> >              if not(os.path.isfile(dtspath)) or not(dts.endswith(".dts") or devicetree_source_is_overlay(dtspath)):
> > --
> > 2.30.2