Creating a machine specific recipe for config file

Creating a machine specific recipe for config file

[Saul Wold]

Folks,

We have had an open enhancement in the form of bugzilla #4011 
(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4011).

I am currently working on this and want to get some feedback regarding 
the design, the below list of config files would move to one recipe in 
recipes-bsp, which will reduce the number of .bbappends that a BSP 
writer might need to create in order to customize the configuration of 
the BSP.

Overall, my proposal is to move all the BSP related config files into 
one recipe directory tree. Create a recipe that can have a package or 
packages that are RRECOMMENDS on.

We have 2 choices on the packaging side:

1) 1 Package to rule them all (conffiles)
   - RPROVIDES PN-conf
   - conffile.bbclass
       RRECOMMENDS = "${PN}-conf"
       # Can be overriden in recipe
       CONFFILES_conffiles ?= "${PN}.conf"
   - Will provide files not needed on final image, small
     amount of extra space used.

2) 1 package / conf file (${PN}-conf)
   - exactly what's needed will be installed
   - no needs for additional RPROVIDES
   - More packaging overhead, package data might be bigger than actual 
contents!

Currently the list of recipes/config files affected include:


meta/recipes-bsp/pointercal/pointercal/*/pointercal
meta/recipes-bsp/formfactor/files/*/machconfig
meta/recipes-bsp/alsa-state/alsa-state/asound.conf
meta/recipes-graphics/xorg-xserver/xserver-xf86-config/*/xorg.conf
meta/recipes-bsp/keymaps/files/keymap.sh
meta/recipes-graphics/xinput-calibrator/pointercal-xinput/pointercal.xinput
meta/recipes-graphics/tslib/tslib/ts.conf


Possibly also:
meta/recipes-core/init-ifupdown/init-ifupdown-1.0
meta/recipes-connectivity/connman/connman-conf
meta/recipes-connectivity/bluez5/bluez5/bluetooth.conf
meta/recipes-bsp/apmd/apmd-3.2.2-14/apmd_proxy.conf

Comment, thoughts, ...

Thanks
-- 
     Sau!

Saul Wold
Yocto Component Wrangler @ Intel
Yocto Project / Poky Build System

Re: Creating a machine specific recipe for config file

[Stephen Arnold]

[-- Attachment #1: Type: text/plain, Size: 2532 bytes --]

Package management overhead seems like a small-ish price for being able to
manage/coordinate custom configs with real depends.  Did I mention I like
2) better?

Steve


On Tue, May 27, 2014 at 11:35 AM, Saul Wold <sgw@linux.intel.com> wrote:

>
> Folks,
>
> We have had an open enhancement in the form of bugzilla #4011 (
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=4011).
>
> I am currently working on this and want to get some feedback regarding the
> design, the below list of config files would move to one recipe in
> recipes-bsp, which will reduce the number of .bbappends that a BSP writer
> might need to create in order to customize the configuration of the BSP.
>
> Overall, my proposal is to move all the BSP related config files into one
> recipe directory tree. Create a recipe that can have a package or packages
> that are RRECOMMENDS on.
>
> We have 2 choices on the packaging side:
>
> 1) 1 Package to rule them all (conffiles)
>   - RPROVIDES PN-conf
>   - conffile.bbclass
>       RRECOMMENDS = "${PN}-conf"
>       # Can be overriden in recipe
>       CONFFILES_conffiles ?= "${PN}.conf"
>   - Will provide files not needed on final image, small
>     amount of extra space used.
>
> 2) 1 package / conf file (${PN}-conf)
>   - exactly what's needed will be installed
>   - no needs for additional RPROVIDES
>   - More packaging overhead, package data might be bigger than actual
> contents!
>
> Currently the list of recipes/config files affected include:
>
>
> meta/recipes-bsp/pointercal/pointercal/*/pointercal
> meta/recipes-bsp/formfactor/files/*/machconfig
> meta/recipes-bsp/alsa-state/alsa-state/asound.conf
> meta/recipes-graphics/xorg-xserver/xserver-xf86-config/*/xorg.conf
> meta/recipes-bsp/keymaps/files/keymap.sh
> meta/recipes-graphics/xinput-calibrator/pointercal-xinput/
> pointercal.xinput
> meta/recipes-graphics/tslib/tslib/ts.conf
>
>
> Possibly also:
> meta/recipes-core/init-ifupdown/init-ifupdown-1.0
> meta/recipes-connectivity/connman/connman-conf
> meta/recipes-connectivity/bluez5/bluez5/bluetooth.conf
> meta/recipes-bsp/apmd/apmd-3.2.2-14/apmd_proxy.conf
>
> Comment, thoughts, ...
>
> Thanks
> --
>     Sau!
>
> Saul Wold
> Yocto Component Wrangler @ Intel
> Yocto Project / Poky Build System
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 3446 bytes --]

Re: Creating a machine specific recipe for config file

[Darren Hart]

On 5/27/14, 11:35, "Saul Wold" <sgw@linux.intel.com> wrote:

>
>Folks,
>
>We have had an open enhancement in the form of bugzilla #4011
>(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4011).
>
>I am currently working on this and want to get some feedback regarding
>the design, the below list of config files would move to one recipe in
>recipes-bsp, which will reduce the number of .bbappends that a BSP
>writer might need to create in order to customize the configuration of
>the BSP.
>
>Overall, my proposal is to move all the BSP related config files into
>one recipe directory tree. Create a recipe that can have a package or
>packages that are RRECOMMENDS on.
>
>We have 2 choices on the packaging side:
>
>1) 1 Package to rule them all (conffiles)
>   - RPROVIDES PN-conf
>   - conffile.bbclass
>       RRECOMMENDS = "${PN}-conf"
>       # Can be overriden in recipe
>       CONFFILES_conffiles ?= "${PN}.conf"
>   - Will provide files not needed on final image, small
>     amount of extra space used.
>
>2) 1 package / conf file (${PN}-conf)
>   - exactly what's needed will be installed
>   - no needs for additional RPROVIDES
>   - More packaging overhead, package data might be bigger than actual
>contents!

The status quo would suggest that Option 2 is more consistent with what
people expect of the build system. However, if we were to do this, one
might question why we should bother at all and not just leave it in the
hands of MACHINE-specific overrides for the packages we're configuring, as
is done today with alsa-state/asound.conf (for example).

What was your idea here - to replace the MACHINE-specific config for these
packages - or to augment it with an optional mega-config package?

I think it would help to provide a bit of background/motivation regarding
what exactly we're trying to accomplish with this. That would help me form
an opinion on 1 vs 2 anyway.

-- 
Darren Hart					Open Source Technology Center
darren.hart@intel.com				            Intel Corporation

Re: Creating a machine specific recipe for config file

[Christopher Larson]

[-- Attachment #1: Type: text/plain, Size: 2796 bytes --]

On Tue, May 27, 2014 at 1:39 PM, Darren Hart <dvhart@linux.intel.com> wrote:

> On 5/27/14, 11:35, "Saul Wold" <sgw@linux.intel.com> wrote:
>
> >
> >Folks,
> >
> >We have had an open enhancement in the form of bugzilla #4011
> >(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4011).
> >
> >I am currently working on this and want to get some feedback regarding
> >the design, the below list of config files would move to one recipe in
> >recipes-bsp, which will reduce the number of .bbappends that a BSP
> >writer might need to create in order to customize the configuration of
> >the BSP.
> >
> >Overall, my proposal is to move all the BSP related config files into
> >one recipe directory tree. Create a recipe that can have a package or
> >packages that are RRECOMMENDS on.
> >
> >We have 2 choices on the packaging side:
> >
> >1) 1 Package to rule them all (conffiles)
> >   - RPROVIDES PN-conf
> >   - conffile.bbclass
> >       RRECOMMENDS = "${PN}-conf"
> >       # Can be overriden in recipe
> >       CONFFILES_conffiles ?= "${PN}.conf"
> >   - Will provide files not needed on final image, small
> >     amount of extra space used.
> >
> >2) 1 package / conf file (${PN}-conf)
> >   - exactly what's needed will be installed
> >   - no needs for additional RPROVIDES
> >   - More packaging overhead, package data might be bigger than actual
> >contents!
>
> The status quo would suggest that Option 2 is more consistent with what
> people expect of the build system. However, if we were to do this, one
> might question why we should bother at all and not just leave it in the
> hands of MACHINE-specific overrides for the packages we're configuring, as
> is done today with alsa-state/asound.conf (for example).
>
> What was your idea here - to replace the MACHINE-specific config for these
> packages - or to augment it with an optional mega-config package?
>
> I think it would help to provide a bit of background/motivation regarding
> what exactly we're trying to accomplish with this. That would help me form
> an opinion on 1 vs 2 anyway.


A third option would be to create a class which examines a path or paths to
a directory structure containing just the config files, and injects a
custom function into PACKAGEFUNCS which overlays the bsp-specific default
configs into the original packages, obeying BBPATH to reflect layer
priorities. It'd essentially be the same as what's done today, just a
possibly more convenient way to do it, from a BSP maintainer perspective,
and wouldn't even be terribly complex, but it might be seen as too much
magic :)
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

[-- Attachment #2: Type: text/html, Size: 3611 bytes --]

Re: Creating a machine specific recipe for config file

[Stephen Arnold]

[-- Attachment #1: Type: text/plain, Size: 3288 bytes --]

Actually, that's not bad either.  As long as the magic is documented, that
sounds pretty good too.

Steve



On Tue, May 27, 2014 at 1:44 PM, Christopher Larson <clarson@kergoth.com>wrote:

>
> On Tue, May 27, 2014 at 1:39 PM, Darren Hart <dvhart@linux.intel.com>wrote:
>
>> On 5/27/14, 11:35, "Saul Wold" <sgw@linux.intel.com> wrote:
>>
>> >
>> >Folks,
>> >
>> >We have had an open enhancement in the form of bugzilla #4011
>> >(https://bugzilla.yoctoproject.org/show_bug.cgi?id=4011).
>> >
>> >I am currently working on this and want to get some feedback regarding
>> >the design, the below list of config files would move to one recipe in
>> >recipes-bsp, which will reduce the number of .bbappends that a BSP
>> >writer might need to create in order to customize the configuration of
>> >the BSP.
>> >
>> >Overall, my proposal is to move all the BSP related config files into
>> >one recipe directory tree. Create a recipe that can have a package or
>> >packages that are RRECOMMENDS on.
>> >
>> >We have 2 choices on the packaging side:
>> >
>> >1) 1 Package to rule them all (conffiles)
>> >   - RPROVIDES PN-conf
>> >   - conffile.bbclass
>> >       RRECOMMENDS = "${PN}-conf"
>> >       # Can be overriden in recipe
>> >       CONFFILES_conffiles ?= "${PN}.conf"
>> >   - Will provide files not needed on final image, small
>> >     amount of extra space used.
>> >
>> >2) 1 package / conf file (${PN}-conf)
>> >   - exactly what's needed will be installed
>> >   - no needs for additional RPROVIDES
>> >   - More packaging overhead, package data might be bigger than actual
>> >contents!
>>
>> The status quo would suggest that Option 2 is more consistent with what
>> people expect of the build system. However, if we were to do this, one
>> might question why we should bother at all and not just leave it in the
>> hands of MACHINE-specific overrides for the packages we're configuring, as
>> is done today with alsa-state/asound.conf (for example).
>>
>> What was your idea here - to replace the MACHINE-specific config for these
>> packages - or to augment it with an optional mega-config package?
>>
>> I think it would help to provide a bit of background/motivation regarding
>> what exactly we're trying to accomplish with this. That would help me form
>> an opinion on 1 vs 2 anyway.
>
>
> A third option would be to create a class which examines a path or paths
> to a directory structure containing just the config files, and injects a
> custom function into PACKAGEFUNCS which overlays the bsp-specific default
> configs into the original packages, obeying BBPATH to reflect layer
> priorities. It'd essentially be the same as what's done today, just a
> possibly more convenient way to do it, from a BSP maintainer perspective,
> and wouldn't even be terribly complex, but it might be seen as too much
> magic :)
> --
> Christopher Larson
> clarson at kergoth dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Maintainer - Tslib
> Senior Software Engineer, Mentor Graphics
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>

[-- Attachment #2: Type: text/html, Size: 4647 bytes --]

Re: Creating a machine specific recipe for config file

[Mark Hatle]

On 5/27/14, 3:39 PM, Darren Hart wrote:
> On 5/27/14, 11:35, "Saul Wold" <sgw@linux.intel.com> wrote:
>
>>
>> Folks,
>>
>> We have had an open enhancement in the form of bugzilla #4011
>> (https://bugzilla.yoctoproject.org/show_bug.cgi?id=4011).
>>
>> I am currently working on this and want to get some feedback regarding
>> the design, the below list of config files would move to one recipe in
>> recipes-bsp, which will reduce the number of .bbappends that a BSP
>> writer might need to create in order to customize the configuration of
>> the BSP.
>>
>> Overall, my proposal is to move all the BSP related config files into
>> one recipe directory tree. Create a recipe that can have a package or
>> packages that are RRECOMMENDS on.
>>
>> We have 2 choices on the packaging side:
>>
>> 1) 1 Package to rule them all (conffiles)
>>    - RPROVIDES PN-conf
>>    - conffile.bbclass
>>        RRECOMMENDS = "${PN}-conf"
>>        # Can be overriden in recipe
>>        CONFFILES_conffiles ?= "${PN}.conf"
>>    - Will provide files not needed on final image, small
>>      amount of extra space used.
>>
>> 2) 1 package / conf file (${PN}-conf)
>>    - exactly what's needed will be installed
>>    - no needs for additional RPROVIDES
>>    - More packaging overhead, package data might be bigger than actual
>> contents!
>
> The status quo would suggest that Option 2 is more consistent with what
> people expect of the build system. However, if we were to do this, one
> might question why we should bother at all and not just leave it in the
> hands of MACHINE-specific overrides for the packages we're configuring, as
> is done today with alsa-state/asound.conf (for example).
>
> What was your idea here - to replace the MACHINE-specific config for these
> packages - or to augment it with an optional mega-config package?

The reason to get away from MACHINE-specific config changes to the regular 
package is from a re-use standpoint.

If BSP_A and BSP_B both need different configurations of the FOO recipe, the 
"right" way today is for two machine specific versions of the FOO recipe/package 
to be generated.

foo-ver-rel.BSP_A.rpm
foo-ver-rel.BSP_B.rpm

This eliminates a lot of potential re-use, and if it's a large package could add 
a lot of unnecessary space (and build time) to the system.

Instead what we want is:

foo-ver-rel.armv7.rpm
foo-conf-ver-rel.armv7.rpm
foo-conf-ver-rel.BSP_A.rpm
foo-conf-ver-rel.BSP_B.rpm

So the package management system will select the best package to meet the 
requirement automatically.  You get to re-use the one foo package on all 
compatible system.  And then you can choose from a default (not-configured), or 
a BSP configuration.  Much quicker to package, install and takes (potentially) 
less space.  (On a trivial hello-world example, it'll actually take more space, 
but get outside the trivial and it will be helpful.)

> I think it would help to provide a bit of background/motivation regarding
> what exactly we're trying to accomplish with this. That would help me form
> an opinion on 1 vs 2 anyway.

So when talking with Saul I suggested that we do either #1 or #2.. and the base 
recipe (not configure) had a require or recommend of the configure file.

The more I think about this, the more I think multiple small configuration 
files/packages makes sense.. due to various system configuration possibilities 
-- but using appropriate RPROVIDES we shouldn't prevent the system from allowing 
a single monolitic configuration for a BSP.

--Mark