multiconfig: Build breaks parsing recipes for other config layer

multiconfig: Build breaks parsing recipes for other config layer

[Bryan Evenson]

I'm testing a new board with a BSP that is setup on the gatesgarth branch.  I've been able to build and load the demo images for this board.  I'm now trying to use multiconfig to build images for my old board and my new board.  So far the build hasn't got past parsing recipes and I'm wondering what I'm doing wrong.

For my setup, I have two separate layers for the two different boards.  Each layer has its own MACHINE and DISTRO definitions.  I setup the multiconfig files as follows:

conf/local.conf:
Added the following line:
BBMULTICONFIG = "configA configB"

conf/multiconfig/configA.conf: (config for old board)
MACHINE = "machineA"
TMPDIR = "${TOPDIR}/tmpConfigA"
DISTRO = "distroA"

conf/multiconfig/configB.conf: (config for new board)
MACHINE = "machineB"
TMPDIR = "${TOPDIR}/tmpConfigB"
DISTRO = "distroB"

If I try building the old image by doing:
bitbake mc:configA:image-nameA

I get errors while parsing the files.  It complains about a Network Manager bbappend in layer B that uses variables that depends on MACHINE being set to "machineB".  The image for configA does not use Network Manager.  I also tried removing the MACHINE and DISTRO settings from local.conf to make sure the multiconfig settings are used, but then I get an error that MACHINE has not been set.

Does layerB need something added to it so parsing ignores recipes in this layer when they aren't needed?  Or do the recipes need to change so they build for other machines?

Thanks,
Bryan

Re: [OE-core] multiconfig: Build breaks parsing recipes for other config layer

[Joshua Watt]

On 2/27/23 14:35, Bryan Evenson wrote:
> I'm testing a new board with a BSP that is setup on the gatesgarth branch.  I've been able to build and load the demo images for this board.  I'm now trying to use multiconfig to build images for my old board and my new board.  So far the build hasn't got past parsing recipes and I'm wondering what I'm doing wrong.
>
> For my setup, I have two separate layers for the two different boards.  Each layer has its own MACHINE and DISTRO definitions.  I setup the multiconfig files as follows:
>
> conf/local.conf:
> Added the following line:
> BBMULTICONFIG = "configA configB"
>
> conf/multiconfig/configA.conf: (config for old board)
> MACHINE = "machineA"
> TMPDIR = "${TOPDIR}/tmpConfigA"
> DISTRO = "distroA"
>
> conf/multiconfig/configB.conf: (config for new board)
> MACHINE = "machineB"
> TMPDIR = "${TOPDIR}/tmpConfigB"
> DISTRO = "distroB"
>
> If I try building the old image by doing:
> bitbake mc:configA:image-nameA
>
> I get errors while parsing the files.  It complains about a Network Manager bbappend in layer B that uses variables that depends on MACHINE being set to "machineB".  The image for configA does not use Network Manager.  I also tried removing the MACHINE and DISTRO settings from local.conf to make sure the multiconfig settings are used, but then I get an error that MACHINE has not been set.
>
> Does layerB need something added to it so parsing ignores recipes in this layer when they aren't needed?  Or do the recipes need to change so they build for other machines?

When you use multiconfig, all of you layers need to parse for all 
configurations (although this is generally good layer practice in 
general, not just when using multiconfig). Usually this involves writing 
the recipes to use overrides where appropriate


Although, if you _really_ don't want to do that, the BBMASK variable 
(https://docs.yoctoproject.org/ref-manual/variables.html#term-BBMASK) 
can now be set per-multiconfig so you can have layerA mask out layerB 
recipes and vice-versa.

>
> Thanks,
> Bryan
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#177808): https://lists.openembedded.org/g/openembedded-core/message/177808
> Mute This Topic: https://lists.openembedded.org/mt/97276005/3616693
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [JPEWhacker@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] multiconfig: Build breaks parsing recipes for other config layer

[Mikko Rapeli]

Hi,

On Mon, Feb 27, 2023 at 03:03:48PM -0600, Joshua Watt wrote:
> 
> On 2/27/23 14:35, Bryan Evenson wrote:
> > I'm testing a new board with a BSP that is setup on the gatesgarth branch.  I've been able to build and load the demo images for this board.  I'm now trying to use multiconfig to build images for my old board and my new board.  So far the build hasn't got past parsing recipes and I'm wondering what I'm doing wrong.
> > 
> > For my setup, I have two separate layers for the two different boards.  Each layer has its own MACHINE and DISTRO definitions.  I setup the multiconfig files as follows:
> > 
> > conf/local.conf:
> > Added the following line:
> > BBMULTICONFIG = "configA configB"
> > 
> > conf/multiconfig/configA.conf: (config for old board)
> > MACHINE = "machineA"
> > TMPDIR = "${TOPDIR}/tmpConfigA"
> > DISTRO = "distroA"
> > 
> > conf/multiconfig/configB.conf: (config for new board)
> > MACHINE = "machineB"
> > TMPDIR = "${TOPDIR}/tmpConfigB"
> > DISTRO = "distroB"
> > 
> > If I try building the old image by doing:
> > bitbake mc:configA:image-nameA
> > 
> > I get errors while parsing the files.  It complains about a Network Manager bbappend in layer B that uses variables that depends on MACHINE being set to "machineB".  The image for configA does not use Network Manager.  I also tried removing the MACHINE and DISTRO settings from local.conf to make sure the multiconfig settings are used, but then I get an error that MACHINE has not been set.
> > 
> > Does layerB need something added to it so parsing ignores recipes in this layer when they aren't needed?  Or do the recipes need to change so they build for other machines?
> 
> When you use multiconfig, all of you layers need to parse for all
> configurations (although this is generally good layer practice in general,
> not just when using multiconfig). Usually this involves writing the recipes
> to use overrides where appropriate
> 
> 
> Although, if you _really_ don't want to do that, the BBMASK variable
> (https://docs.yoctoproject.org/ref-manual/variables.html#term-BBMASK) can
> now be set per-multiconfig so you can have layerA mask out layerB recipes
> and vice-versa.

Another way to have distro, machine, version etc specific bbappends and recipes in
layers is to use DISTRO, MACHINE, DISTRO_CODENAME etc variables in
BBFILES path. Example layer.conf:

BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
            ${LAYERDIR}/recipes-*/*/*.bbappend \
            ${LAYERDIR}/${DISTRO_CODENAME}-recipes-*/*/*.bb \
            ${LAYERDIR}/${DISTRO_CODENAME}-recipes-*/*/*.bbappend \
"

Then the layer can have kirkstone-recipes* and mickledore-recipes*
directories for bbappends specific to those branches. This enables
supporting multiple poky/yocto major versions from a single meta layer
branch without having recipe version specific bbappends which cause
problems when there are no recipes matching those versions. Same
approach will work with DISTRO, MACHINE and other variables.

Cheers,

-Mikko

RE: [OE-core] multiconfig: Build breaks parsing recipes for other config layer

[Bryan Evenson]

Mikko and Joshua,

Thanks for the guidance.  I tried using BBMASK and that wasn't working for me.  After a little digging I figured out that the BSP for this new board is based on an old enough version of gatesgarth that it doesn't have the patch for BBMASK multiconfig support.

I've been looking at the meta-ti layer for an example of how to use distro overrides to set variables in recipes selectively based on the distro.  The build has been getting further as I update the recipes one at a time.

I may give the BBFILES filtering by different distro variables a try.  I think I can see a way I can set DISTRO_CODENAME in each multiconfig conf file that will set it up to ignore the layers I want it to ignore for each config.  Either way, I think I'll be with the company that created the BSP on possible ways to make their BSP portable for multiconfig setups.

Thanks,
Bryan

> -----Original Message-----
> From: Mikko Rapeli <mikko.rapeli@linaro.org>
> Sent: Tuesday, February 28, 2023 2:06 AM
> To: Joshua Watt <JPEWhacker@gmail.com>
> Cc: Bryan Evenson <bevenson@melinkcorp.com>; openembedded-
> core@lists.openembedded.org
> Subject: Re: [OE-core] multiconfig: Build breaks parsing recipes for other
> config layer
> 
> Hi,
> 
> On Mon, Feb 27, 2023 at 03:03:48PM -0600, Joshua Watt wrote:
> >
> > On 2/27/23 14:35, Bryan Evenson wrote:
> > > I'm testing a new board with a BSP that is setup on the gatesgarth branch.
> I've been able to build and load the demo images for this board.  I'm now
> trying to use multiconfig to build images for my old board and my new board.
> So far the build hasn't got past parsing recipes and I'm wondering what I'm
> doing wrong.
> > >
> > > For my setup, I have two separate layers for the two different boards.
> Each layer has its own MACHINE and DISTRO definitions.  I setup the
> multiconfig files as follows:
> > >
> > > conf/local.conf:
> > > Added the following line:
> > > BBMULTICONFIG = "configA configB"
> > >
> > > conf/multiconfig/configA.conf: (config for old board) MACHINE =
> > > "machineA"
> > > TMPDIR = "${TOPDIR}/tmpConfigA"
> > > DISTRO = "distroA"
> > >
> > > conf/multiconfig/configB.conf: (config for new board) MACHINE =
> > > "machineB"
> > > TMPDIR = "${TOPDIR}/tmpConfigB"
> > > DISTRO = "distroB"
> > >
> > > If I try building the old image by doing:
> > > bitbake mc:configA:image-nameA
> > >
> > > I get errors while parsing the files.  It complains about a Network
> Manager bbappend in layer B that uses variables that depends on MACHINE
> being set to "machineB".  The image for configA does not use Network
> Manager.  I also tried removing the MACHINE and DISTRO settings from
> local.conf to make sure the multiconfig settings are used, but then I get an
> error that MACHINE has not been set.
> > >
> > > Does layerB need something added to it so parsing ignores recipes in this
> layer when they aren't needed?  Or do the recipes need to change so they
> build for other machines?
> >
> > When you use multiconfig, all of you layers need to parse for all
> > configurations (although this is generally good layer practice in
> > general, not just when using multiconfig). Usually this involves
> > writing the recipes to use overrides where appropriate
> >
> >
> > Although, if you _really_ don't want to do that, the BBMASK variable
> > (https://docs.yoctoproject.org/ref-manual/variables.html#term-BBMASK)
> > can now be set per-multiconfig so you can have layerA mask out layerB
> > recipes and vice-versa.
> 
> Another way to have distro, machine, version etc specific bbappends and
> recipes in layers is to use DISTRO, MACHINE, DISTRO_CODENAME etc
> variables in BBFILES path. Example layer.conf:
> 
> BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
>             ${LAYERDIR}/recipes-*/*/*.bbappend \
>             ${LAYERDIR}/${DISTRO_CODENAME}-recipes-*/*/*.bb \
>             ${LAYERDIR}/${DISTRO_CODENAME}-recipes-*/*/*.bbappend \ "
> 
> Then the layer can have kirkstone-recipes* and mickledore-recipes*
> directories for bbappends specific to those branches. This enables supporting
> multiple poky/yocto major versions from a single meta layer branch without
> having recipe version specific bbappends which cause problems when there
> are no recipes matching those versions. Same approach will work with
> DISTRO, MACHINE and other variables.
> 
> Cheers,
> 
> -Mikko