[meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Khem Raj]

trusted-firmware-m needs arm binary toolchain to build from another
layer, this is the only recipe thus far and its better to hide it to
majority users who may not be using trusted-firmware-m and thusly
meta-arm-toolchain, folks who need to use trusted-firmware-m should
explicitly add it to layer mix

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta-arm/conf/layer.conf                                    | 6 +++++-
 .../trusted-firmware-m/trusted-firmware-m_1.0.bb            | 0
 2 files changed, 5 insertions(+), 1 deletion(-)
 rename meta-arm/{ => dynamic-layers/arm-toolchain}/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb (100%)

diff --git a/meta-arm/conf/layer.conf b/meta-arm/conf/layer.conf
index 10a7951..2d0c49f 100644
--- a/meta-arm/conf/layer.conf
+++ b/meta-arm/conf/layer.conf
@@ -11,6 +11,10 @@ BBFILE_PRIORITY_meta-arm = "6"
 
 LAYERDEPENDS_meta-arm = " \
     core \
-    arm-toolchain \
 "
 LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
+
+BBFILES_DYNAMIC += " \
+    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bb \
+    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bbappend \
+"
diff --git a/meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb b/meta-arm/dynamic-layers/arm-toolchain/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
similarity index 100%
rename from meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
rename to meta-arm/dynamic-layers/arm-toolchain/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
-- 
2.27.0

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Richard Purdie]

On Fri, 2020-07-03 at 07:52 -0700, Khem Raj wrote:
> trusted-firmware-m needs arm binary toolchain to build from another
> layer, this is the only recipe thus far and its better to hide it to
> majority users who may not be using trusted-firmware-m and thusly
> meta-arm-toolchain, folks who need to use trusted-firmware-m should
> explicitly add it to layer mix
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta-arm/conf/layer.conf                                    | 6 +++++-
>  .../trusted-firmware-m/trusted-firmware-m_1.0.bb            | 0
>  2 files changed, 5 insertions(+), 1 deletion(-)
>  rename meta-arm/{ => dynamic-layers/arm-toolchain}/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb (100%)
> 
> diff --git a/meta-arm/conf/layer.conf b/meta-arm/conf/layer.conf
> index 10a7951..2d0c49f 100644
> --- a/meta-arm/conf/layer.conf
> +++ b/meta-arm/conf/layer.conf
> @@ -11,6 +11,10 @@ BBFILE_PRIORITY_meta-arm = "6"
>  
>  LAYERDEPENDS_meta-arm = " \
>      core \
> -    arm-toolchain \
>  "
>  LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
> +
> +BBFILES_DYNAMIC += " \
> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bb \
> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bbappend \
> +"
> diff --git a/meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb b/meta-arm/dynamic-layers/arm-toolchain/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
> similarity index 100%
> rename from meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
> rename to meta-arm/dynamic-layers/arm-toolchain/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb

Ross asked to add meta-arm to the autobuilder and I agreed on the basis
it was passing testing. Due to this change all full builds are now red
again :(

https://autobuilder.yoctoproject.org/typhoon/#/builders/113/builds/11

Cheers,

Richard

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Denys Dmytriyenko]

On Sun, Jul 05, 2020 at 09:15:45PM +0100, Richard Purdie wrote:
> On Fri, 2020-07-03 at 07:52 -0700, Khem Raj wrote:
> > trusted-firmware-m needs arm binary toolchain to build from another
> > layer, this is the only recipe thus far and its better to hide it to
> > majority users who may not be using trusted-firmware-m and thusly
> > meta-arm-toolchain, folks who need to use trusted-firmware-m should
> > explicitly add it to layer mix
> > 
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta-arm/conf/layer.conf                                    | 6 +++++-
> >  .../trusted-firmware-m/trusted-firmware-m_1.0.bb            | 0
> >  2 files changed, 5 insertions(+), 1 deletion(-)
> >  rename meta-arm/{ => dynamic-layers/arm-toolchain}/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb (100%)
> > 
> > diff --git a/meta-arm/conf/layer.conf b/meta-arm/conf/layer.conf
> > index 10a7951..2d0c49f 100644
> > --- a/meta-arm/conf/layer.conf
> > +++ b/meta-arm/conf/layer.conf
> > @@ -11,6 +11,10 @@ BBFILE_PRIORITY_meta-arm = "6"
> >  
> >  LAYERDEPENDS_meta-arm = " \
> >      core \
> > -    arm-toolchain \
> >  "
> >  LAYERSERIES_COMPAT_meta-arm = "warrior zeus dunfell"
> > +
> > +BBFILES_DYNAMIC += " \
> > +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bb \
> > +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bbappend \
> > +"
> > diff --git a/meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb b/meta-arm/dynamic-layers/arm-toolchain/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
> > similarity index 100%
> > rename from meta-arm/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
> > rename to meta-arm/dynamic-layers/arm-toolchain/recipes-bsp/trusted-firmware-m/trusted-firmware-m_1.0.bb
> 
> Ross asked to add meta-arm to the autobuilder and I agreed on the basis
> it was passing testing. Due to this change all full builds are now red
> again :(
> 
> https://autobuilder.yoctoproject.org/typhoon/#/builders/113/builds/11

FWIW, this has been discussed at length here:
https://lists.yoctoproject.org/g/meta-arm/topic/patch_4_6_arm/75033886

-- 
Denys

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Ross Burton]

On Sun, 5 Jul 2020 at 21:15, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Fri, 2020-07-03 at 07:52 -0700, Khem Raj wrote:
> > trusted-firmware-m needs arm binary toolchain to build from another
> > layer, this is the only recipe thus far and its better to hide it to
> > majority users who may not be using trusted-firmware-m and thusly
> > meta-arm-toolchain, folks who need to use trusted-firmware-m should
> > explicitly add it to layer mix

My fault: the layer ordering wasn't quite as important before, but it
is now.  Testing a fix to autobuilder-helper now.

Ross

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Ross Burton]

Update: Richard has merged the fix.  Hopefully that now results in a
green build!

Ross

On Mon, 6 Jul 2020 at 12:07, Ross Burton <ross@burtonini.com> wrote:
>
> On Sun, 5 Jul 2020 at 21:15, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Fri, 2020-07-03 at 07:52 -0700, Khem Raj wrote:
> > > trusted-firmware-m needs arm binary toolchain to build from another
> > > layer, this is the only recipe thus far and its better to hide it to
> > > majority users who may not be using trusted-firmware-m and thusly
> > > meta-arm-toolchain, folks who need to use trusted-firmware-m should
> > > explicitly add it to layer mix
>
> My fault: the layer ordering wasn't quite as important before, but it
> is now.  Testing a fix to autobuilder-helper now.
>
> Ross

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Khem Raj]

On 7/6/20 5:30 AM, Ross Burton wrote:
> Update: Richard has merged the fix.  Hopefully that now results in a
> green build!
> 

this does not fix the problem for other distributions which are using 
meta-arm as dependency. this patch would still be better option IMO to 
avoid this dependency when not needed.

> Ross
> 
> On Mon, 6 Jul 2020 at 12:07, Ross Burton <ross@burtonini.com> wrote:
>>
>> On Sun, 5 Jul 2020 at 21:15, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>>> On Fri, 2020-07-03 at 07:52 -0700, Khem Raj wrote:
>>>> trusted-firmware-m needs arm binary toolchain to build from another
>>>> layer, this is the only recipe thus far and its better to hide it to
>>>> majority users who may not be using trusted-firmware-m and thusly
>>>> meta-arm-toolchain, folks who need to use trusted-firmware-m should
>>>> explicitly add it to layer mix
>>
>> My fault: the layer ordering wasn't quite as important before, but it
>> is now.  Testing a fix to autobuilder-helper now.
>>
>> Ross

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Ross Burton]

On Fri, 3 Jul 2020 at 15:52, Khem Raj <raj.khem@gmail.com> wrote:
> +BBFILES_DYNAMIC += " \
> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bb \
> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bbappend \

What I dislike about dynamic layers is how nested everything gets.
How about something more like:

BBFILES_DYNAMIC +=
"arm-toolchain:${LAYERDIR}/with-arm-toolchain/recipes-*/*/*.bb"

One less directory level, and all of the dynamic layers are at the
top-level alongside the recipes.

I'm also still unsure why there's so much pushback towards adding
arm-toolchain.  It doesn't do anything by simply adding the layer, if
you don't build tf-m it won't get used at all, and I want to get rid
of it anyway (using multiconfig to build a gcc).  Can you explain why
this dependency is so important to you?  I'm obviously not
understanding something, and I'm on the record for hating on binary
toolchains over many years!

Ross

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Khem Raj]

On 7/6/20 2:23 PM, Ross Burton wrote:
> On Fri, 3 Jul 2020 at 15:52, Khem Raj <raj.khem@gmail.com> wrote:
>> +BBFILES_DYNAMIC += " \
>> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bb \
>> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bbappend \
> 
> What I dislike about dynamic layers is how nested everything gets.
> How about something more like:
> 
> BBFILES_DYNAMIC +=
> "arm-toolchain:${LAYERDIR}/with-arm-toolchain/recipes-*/*/*.bb"
> 
> One less directory level, and all of the dynamic layers are at the
> top-level alongside the recipes.
> 
> I'm also still unsure why there's so much pushback towards adding
> arm-toolchain.  

reason meta-arm is needed for me is becuase meta-ti asks for it and now 
there is another dependency added so I am a bit not-nice to this because 
this serves no use for me as tf-a and tf-m are not even needed for 
machines of interest,  but it does show up on world builds and broke 
last time.

It doesn't do anything by simply adding the layer, if
> you don't build tf-m it won't get used at all, and I want to get rid
> of it anyway (using multiconfig to build a gcc).  Can you explain why
> this dependency is so important to you?  I'm obviously not
> understanding something, and I'm on the record for hating on binary
> toolchains over many years!
> 
> Ross
> 

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Khem Raj]

On 7/6/20 2:23 PM, Ross Burton wrote:
> On Fri, 3 Jul 2020 at 15:52, Khem Raj <raj.khem@gmail.com> wrote:
>> +BBFILES_DYNAMIC += " \
>> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bb \
>> +    arm-toolchain:${LAYERDIR}/dynamic-layers/arm-toolchain/*/*/*.bbappend \
> 
> What I dislike about dynamic layers is how nested everything gets.
> How about something more like:
> 
> BBFILES_DYNAMIC +=
> "arm-toolchain:${LAYERDIR}/with-arm-toolchain/recipes-*/*/*.bb"
> 
> One less directory level, and all of the dynamic layers are at the
> top-level alongside the recipes.
> 
> I'm also still unsure why there's so much pushback towards adding
> arm-toolchain.  It doesn't do anything by simply adding the layer, if
> you don't build tf-m it won't get used at all, and I want to get rid
> of it anyway (using multiconfig to build a gcc).  Can you explain why
> this dependency is so important to you?  I'm obviously not
> understanding something, and I'm on the record for hating on binary
> toolchains over many years!
> 

So I have added meta-arm-toolchain to distro ( ever to reluctantly ) and 
it did not break anything so I intend to keep that change, therefore 
this patch is no longer required for my usecase and can be ignored.

> Ross
> 

Re: [meta-arm][PATCH] meta-arm: Use dynamic layer magic to build trusted-firmware-m

[Ross Burton]

On Mon, 13 Jul 2020 at 18:09, Khem Raj <raj.khem@gmail.com> wrote:
> So I have added meta-arm-toolchain to distro ( ever to reluctantly ) and
> it did not break anything so I intend to keep that change, therefore
> this patch is no longer required for my usecase and can be ignored.

Thanks for the update Khem.  I have removing the need for binary
toolchains on my todo list.

Ross