[PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

This allows:

1. Showing users where and how to define these settings correctly when setting up
a build from templates in poky (meta-poky/conf/site.conf.sample has commented
out examples and was previously unused).

2. Distributing site-specific settings with template configurations in other layers,
so there's no need to set them up separately.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 scripts/oe-setup-builddir | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/oe-setup-builddir b/scripts/oe-setup-builddir
index 54048e62ec..5ad6dd4138 100755
--- a/scripts/oe-setup-builddir
+++ b/scripts/oe-setup-builddir
@@ -64,6 +64,7 @@ if [ -n "$TEMPLATECONF" ]; then
     fi
     OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
     OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
+    OECORESITECONF="$TEMPLATECONF/site.conf.sample"
     OECORENOTESCONF="$TEMPLATECONF/conf-notes.txt"
 fi
 
@@ -77,9 +78,11 @@ You had no conf/local.conf file. This configuration file has therefore been
 created for you with some default values. You may wish to edit it to, for
 example, select a different MACHINE (target hardware). See conf/local.conf
 for more information as common configuration options are commented.
-
+Also check conf/site.conf for site specific settings such as proxies and
+download cache locations.
 EOM
     cp -f "$OECORELOCALCONF" "$BUILDDIR/conf/local.conf"
+    cp -f "$OECORESITECONF" "$BUILDDIR/conf/site.conf" || true
     SHOWYPDOC=yes
 fi
 
@@ -107,6 +110,7 @@ fi
 # Prevent disturbing a new GIT clone in same console
 unset OECORELOCALCONF
 unset OECORELAYERCONF
+unset OECORESITECONF
 
 # Ending the first-time run message. Show the YP Documentation banner.
 if [ ! -z "$SHOWYPDOC" ]; then
-- 
2.30.2

[PATCH 2/2] bitbake-layers: add a command to save the active build configuration as a template into a layer

[Alexander Kanavin]

This is the reverse of setting up a build by pointing TEMPLATECONF to a directory
with a template and running '. oe-init-build-env': this takes the config files from build/conf,
replaces site-specific paths in bblayers.conf with ##OECORE##-relative paths, and copies
the config files into a specified layer under a specified template name.

In many or perhaps most cases such static prefabricated configurations (that require no
further editing) are just enough, and I believe they should be offered by the
official configuration management. On the other hand, generating build configurations with a
sufficiently versatile tool is a far more complex problem, and one we should try to tackle
once we see where and how static configs fall short.

Tooling to discover and select these templates when setting up a build will be provided later on.

How to use:

alex@Zen2:/srv/work/alex/poky/build-layersetup$ bitbake-layers save-build-conf ../../meta-alex/ test-1
NOTE: Starting bitbake server...
NOTE: Configuration template placed into /srv/work/alex/meta-alex/conf/templates/test-1
Please review the files in there, and particularly provide a configuration description in /srv/work/alex/meta-alex/conf/templates/test-1/conf-notes.txt
You can try out the configuration with
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
alex@Zen2:/srv/work/alex/poky/build-layersetup$

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
 meta/lib/bblayers/buildconf.py           | 90 ++++++++++++++++++++++++
 meta/lib/oeqa/selftest/cases/bblayers.py |  5 ++
 2 files changed, 95 insertions(+)
 create mode 100644 meta/lib/bblayers/buildconf.py

diff --git a/meta/lib/bblayers/buildconf.py b/meta/lib/bblayers/buildconf.py
new file mode 100644
index 0000000000..0af04f9531
--- /dev/null
+++ b/meta/lib/bblayers/buildconf.py
@@ -0,0 +1,90 @@
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import logging
+import os
+import stat
+import sys
+import shutil
+import json
+
+import bb.utils
+import bb.process
+
+from bblayers.common import LayerPlugin
+
+logger = logging.getLogger('bitbake-layers')
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
+
+import oe.buildcfg
+
+def plugin_init(plugins):
+    return BuildConfPlugin()
+
+class BuildConfPlugin(LayerPlugin):
+    notes_fixme = """FIXME: Please place here the description of this build configuration.
+It will be shown to the users when they set up their builds via TEMPLATECONF.
+"""
+
+    def _save_conf(self, templatename, templatepath, oecorepath, relpaths_to_oecore):
+        confdir = os.path.join(os.environ["BBPATH"], "conf")
+        destdir = os.path.join(templatepath, "conf", "templates", templatename)
+        os.makedirs(destdir, exist_ok=True)
+
+        with open(os.path.join(confdir, "local.conf")) as src:
+            with open(os.path.join(destdir, "local.conf.sample"), 'w') as dest:
+                dest.write(src.read())
+
+        with open(os.path.join(confdir, "bblayers.conf")) as src:
+            with open(os.path.join(destdir, "bblayers.conf.sample"), 'w') as dest:
+                bblayers_data = src.read()
+
+                for (abspath, relpath) in relpaths_to_oecore:
+                    bblayers_data = bblayers_data.replace(abspath, "##OEROOT##/" + relpath)
+                dest.write(bblayers_data)
+
+        with open(os.path.join(destdir, "conf-notes.txt"), 'w') as dest:
+            dest.write(self.notes_fixme)
+
+        try:
+            with open(os.path.join(confdir, "site.conf")) as src:
+                with open(os.path.join(destdir, "site.conf.sample"), 'w') as dest:
+                    dest.write(src.read())
+        except FileNotFoundError:
+            pass
+
+        logger.info("""Configuration template placed into {}
+Please review the files in there, and particularly provide a configuration description in {}
+You can try out the configuration with
+TEMPLATECONF={} . {}/oe-init-build-env build-try-{}"""
+.format(destdir, os.path.join(destdir, "conf-notes.txt"), destdir, oecorepath, templatename))
+
+    def do_save_build_conf(self, args):
+        """ Save the currently active build configuration (conf/local.conf, conf/bblayers.conf) as a template into a layer.\n This template can later be used for setting up builds via TEMPLATECONF. """
+        repos = {}
+        layers = oe.buildcfg.get_layer_revisions(self.tinfoil.config_data)
+        targetlayer = None
+        oecore = None
+
+        for l in layers:
+            if l[0] == os.path.abspath(args.layerpath):
+                targetlayer = l[0]
+            if l[1] == 'meta':
+                oecore = os.path.dirname(l[0])
+
+        if not targetlayer:
+            logger.error("Layer {} not in one of the currently enabled layers:\n{}".format(args.layerpath, "\n".join([l[0] for l in layers])))
+        elif not oecore:
+            logger.error("Openembedded-core not in one of the currently enabled layers:\n{}".format("\n".join([l[0] for l in layers])))
+        else:
+            relpaths_to_oecore = [(l[0], os.path.relpath(l[0], start=oecore)) for l in layers]
+            self._save_conf(args.templatename, targetlayer, oecore, relpaths_to_oecore)
+
+    def register_commands(self, sp):
+        parser_build_conf = self.add_command(sp, 'save-build-conf', self.do_save_build_conf, parserecipes=False)
+        parser_build_conf.add_argument('layerpath',
+            help='The path to the layer where the configuration template should be saved.')
+        parser_build_conf.add_argument('templatename',
+            help='The name of the configuration template.')
diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py b/meta/lib/oeqa/selftest/cases/bblayers.py
index 7d74833f61..8aa45e432e 100644
--- a/meta/lib/oeqa/selftest/cases/bblayers.py
+++ b/meta/lib/oeqa/selftest/cases/bblayers.py
@@ -111,6 +111,11 @@ class BitbakeLayers(OESelftestTestCase):
 
         self.assertEqual(bb_vars['BBFILE_PRIORITY_%s' % layername], str(priority), 'BBFILE_PRIORITY_%s != %d' % (layername, priority))
 
+        result = runCmd('bitbake-layers save-build-conf {} {}'.format(layerpath, "buildconf-1"))
+        for f in ('local.conf.sample', 'bblayers.conf.sample', 'conf-notes.txt'):
+            fullpath = os.path.join(layerpath, "conf", "templates", "buildconf-1", f)
+            self.assertTrue(os.path.exists(fullpath), "Template configuration file {} not found".format(fullpath))
+
     def get_recipe_basename(self, recipe):
         recipe_file = ""
         result = runCmd("bitbake-layers show-recipes -f %s" % recipe)
-- 
2.30.2

RE: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Alexander Kanavin
> Sent: den 6 juli 2022 20:23
> To: openembedded-core@lists.openembedded.org
> Cc: Alexander Kanavin <alex@linutronix.de>
> Subject: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy
> site.conf.sample out of template directories (if it exists)
> 
> This allows:
> 
> 1. Showing users where and how to define these settings correctly when setting up
> a build from templates in poky (meta-poky/conf/site.conf.sample has commented
> out examples and was previously unused).
> 
> 2. Distributing site-specific settings with template configurations in other layers,
> so there's no need to set them up separately.
> 
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>  scripts/oe-setup-builddir | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/oe-setup-builddir b/scripts/oe-setup-builddir
> index 54048e62ec..5ad6dd4138 100755
> --- a/scripts/oe-setup-builddir
> +++ b/scripts/oe-setup-builddir
> @@ -64,6 +64,7 @@ if [ -n "$TEMPLATECONF" ]; then
>      fi
>      OECORELAYERCONF="$TEMPLATECONF/bblayers.conf.sample"
>      OECORELOCALCONF="$TEMPLATECONF/local.conf.sample"
> +    OECORESITECONF="$TEMPLATECONF/site.conf.sample"
>      OECORENOTESCONF="$TEMPLATECONF/conf-notes.txt"
>  fi
> 
> @@ -77,9 +78,11 @@ You had no conf/local.conf file. This configuration
> file has therefore been
>  created for you with some default values. You may wish to edit it to, for
>  example, select a different MACHINE (target hardware). See conf/local.conf
>  for more information as common configuration options are commented.
> -
> +Also check conf/site.conf for site specific settings such as proxies and
> +download cache locations.
>  EOM
>      cp -f "$OECORELOCALCONF" "$BUILDDIR/conf/local.conf"
> +    cp -f "$OECORESITECONF" "$BUILDDIR/conf/site.conf" || true
>      SHOWYPDOC=yes
>  fi
> 
> @@ -107,6 +110,7 @@ fi
>  # Prevent disturbing a new GIT clone in same console
>  unset OECORELOCALCONF
>  unset OECORELAYERCONF
> +unset OECORESITECONF
> 
>  # Ending the first-time run message. Show the YP Documentation banner.
>  if [ ! -z "$SHOWYPDOC" ]; then
> --
> 2.30.2

While I can't speak for others, this change will break our build environment. 
We have a global site.conf file in an NFS directory that everyone use. It 
contains all proxy settings and similar that are required to build inside the 
corporate environment. Copying a dummy site.conf file into the local build 
directory will prevent the global file from being used.

//Peter

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Thu, 7 Jul 2022 at 15:26, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:

> While I can't speak for others, this change will break our build environment.
> We have a global site.conf file in an NFS directory that everyone use. It
> contains all proxy settings and similar that are required to build inside the
> corporate environment. Copying a dummy site.conf file into the local build
> directory will prevent the global file from being used.

Then you need to create a configuration template that is specific to
your company first, and there's a convenient command to do that
provided in patch 2/2. Setting up a build directory from standard poky
or oe-core templates (which I presume is how you set up builds) does
not hold a promise that what is being copied will not change in
potentially breaking ways.

Alex

RE: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Peter Kjellerstedt]

> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: den 7 juli 2022 15:36
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin
> <alex@linutronix.de>
> Subject: Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy
> site.conf.sample out of template directories (if it exists)
> 
> On Thu, 7 Jul 2022 at 15:26, Peter Kjellerstedt
> <peter.kjellerstedt@axis.com> wrote:
> 
> > While I can't speak for others, this change will break our build environment.
> > We have a global site.conf file in an NFS directory that everyone use. It
> > contains all proxy settings and similar that are required to build inside the
> > corporate environment. Copying a dummy site.conf file into the local build
> > directory will prevent the global file from being used.
> 
> Then you need to create a configuration template that is specific to
> your company first, and there's a convenient command to do that
> provided in patch 2/2.

While I haven't actually tried your new command, based on what you and 
others have written, it does not at all seem compatible with how our 
environment is setup and what it expects from poky. :(

> Setting up a build directory from standard poky
> or oe-core templates (which I presume is how you set up builds) does
> not hold a promise that what is being copied will not change in
> potentially breaking ways.

While I may be able to fix the environment for builds that use some of 
our layers, setting up pure poky builds will no longer work without 
manual intervention.

> Alex

//Peter

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Thu, 7 Jul 2022 at 15:45, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:

> > Setting up a build directory from standard poky
> > or oe-core templates (which I presume is how you set up builds) does
> > not hold a promise that what is being copied will not change in
> > potentially breaking ways.
>
> While I may be able to fix the environment for builds that use some of
> our layers, setting up pure poky builds will no longer work without
> manual intervention.

Wait. How are you setting up pure poky builds right now, so that your
magic site.conf is correctly inserted into builds?

Alex

RE: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Alexander Kanavin
> Sent: den 7 juli 2022 15:56
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin
> <alex@linutronix.de>
> Subject: Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy
> site.conf.sample out of template directories (if it exists)
> 
> On Thu, 7 Jul 2022 at 15:45, Peter Kjellerstedt
> <peter.kjellerstedt@axis.com> wrote:
> 
> > > Setting up a build directory from standard poky
> > > or oe-core templates (which I presume is how you set up builds) does
> > > not hold a promise that what is being copied will not change in
> > > potentially breaking ways.
> >
> > While I may be able to fix the environment for builds that use some of
> > our layers, setting up pure poky builds will no longer work without
> > manual intervention.
> 
> Wait. How are you setting up pure poky builds right now, so that your
> magic site.conf is correctly inserted into builds?
> 
> Alex

Actually, you can forget what I said. I had a look at our wrapper for 
oe-init-build-env, which is responsible for creating our templateconf 
directory, and it will of course not be affected by your change. (It was 
six years since I wrote that code, so my memory was obviously a bit 
fuzzy.)

And in case you wonder what that wrapper actually does, it creates the 
bblayers.conf.sample file based on all layers that are found in the top 
directory. It also creates a local.conf.sample file by combining fragment 
files in the layers (i.e., local.conf.sample.XX where XX is a number 
between 00 and 99). The local.conf.sample from meta-poky is considered 
to have XX == 50. The reason for this is that we quickly realized that 
the static templateconf directory setup was unusable for us when we 
want to be able to use layers in different combinations.

//Peter

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Thu, 7 Jul 2022 at 16:38, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:

> And in case you wonder what that wrapper actually does, it creates the
> bblayers.conf.sample file based on all layers that are found in the top
> directory. It also creates a local.conf.sample file by combining fragment
> files in the layers (i.e., local.conf.sample.XX where XX is a number
> between 00 and 99). The local.conf.sample from meta-poky is considered
> to have XX == 50. The reason for this is that we quickly realized that
> the static templateconf directory setup was unusable for us when we
> want to be able to use layers in different combinations.

Cheers. I'd say if you need to place lots of customizations into
local.conf that's not a recommended setup to begin with: as said
elsewhere it should have precisely two definitions: distro and
machine, and nothing else. What kind of things are in those fragments?
Should they be consolidated in machine and distro definitions?

Alex

RE: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Peter Kjellerstedt]

> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: den 7 juli 2022 16:48
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin
> <alex@linutronix.de>
> Subject: Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy
> site.conf.sample out of template directories (if it exists)
> 
> On Thu, 7 Jul 2022 at 16:38, Peter Kjellerstedt
> <peter.kjellerstedt@axis.com> wrote:
> 
> > And in case you wonder what that wrapper actually does, it creates the
> > bblayers.conf.sample file based on all layers that are found in the top
> > directory. It also creates a local.conf.sample file by combining
> fragment
> > files in the layers (i.e., local.conf.sample.XX where XX is a number
> > between 00 and 99). The local.conf.sample from meta-poky is considered
> > to have XX == 50. The reason for this is that we quickly realized that
> > the static templateconf directory setup was unusable for us when we
> > want to be able to use layers in different combinations.
> 
> Cheers. I'd say if you need to place lots of customizations into

There actually isn't lots of configuration in those fragments. 

> local.conf that's not a recommended setup to begin with: as said
> elsewhere it should have precisely two definitions: distro and
> machine, and nothing else. What kind of things are in those fragments?

In our BSP layer there is some configuration we want even if our 
distro layer is not used (e.g., SSTATE_DIR). It also sets a default 
MACHINE that matches one from our BSPs. In the distro layer, it sets 
DISTRO. It also adds some extra, commented variables that our users 
may want to enable in their local.conf files, much like the examples 
in the local.conf.sample from meta-poky.

Part of this is of course related to that we use repo to fetch the 
layers we need and thus we do not have the luxury of bitbake specific 
tools like kas and whisk that can add bitbake configuration in addition 
to the actual layers. On the other hand, our environment is very similar 
to the default experience with poky from a user's perspective.

> Should they be consolidated in machine and distro definitions?
> 
> Alex

//Peter

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Richard Purdie]

On Wed, 2022-07-06 at 20:23 +0200, Alexander Kanavin wrote:
> This allows:
> 
> 1. Showing users where and how to define these settings correctly when setting up
> a build from templates in poky (meta-poky/conf/site.conf.sample has commented
> out examples and was previously unused).
> 
> 2. Distributing site-specific settings with template configurations in other layers,
> so there's no need to set them up separately.
> 
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
>  scripts/oe-setup-builddir | 6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)

We'd always intended that site.conf was more something the user would
setup once for their environment and not need to change. It was
therefore deliberately left out of the auto-setup piece.

I hadn't remembered we have an sample version of it. I think a sample
is fine as an example of how a site might setup a shared downloads
directory or common proxy information but I don't think the layer
should be installing it, particularly if it can trample an existing
file which I think the patch could as it stands.

As such I don't think this patch is the right thing to do.

Cheers,

Richard

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Fri, 15 Jul 2022 at 16:42, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> We'd always intended that site.conf was more something the user would
> setup once for their environment and not need to change. It was
> therefore deliberately left out of the auto-setup piece.
> I hadn't remembered we have an sample version of it. I think a sample
> is fine as an example of how a site might setup a shared downloads
> directory or common proxy information but I don't think the layer
> should be installing it, particularly if it can trample an existing
> file which I think the patch could as it stands.

Ah I see where the problem is: the sample file is installed subject to
existence of local.conf in the destination build directory, which is
ok most of the time, but not all of the time. It should check for
existence of site.conf.

The idea is to serve two use cases:
- when setting up a build from the poky template, there's a sample
file installed. Same as local.conf.sample, we give users something to
play with via commented out examples. They'll never find it in
meta-poky/conf, even you had no idea it's there :)
- when setting up a build from an organizational template, the right
servers get configured without any extra steps. Ross pointed out that
can be achieved with 'meta-product/conf/site.conf', but that can't be
edited when needed, and can't be used to configure different settings
for different builds (e.g. where the caches are - such things can be
under access control restrictions).

I think this is worth reworking and making the commit message clearer.
Or we should come up with a different way to provide 'official' way to
set up site.conf, because otherwise it's back to 'write a custom
script'.

Alex

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Fri, 15 Jul 2022 at 16:42, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> We'd always intended that site.conf was more something the user would
> setup once for their environment and not need to change. It was
> therefore deliberately left out of the auto-setup piece.

I don't think I understand this point in particular. If site.conf is
copied once into a newly created, empty build directory, why is that
problematic?

Alex

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Richard Purdie]

On Fri, 2022-07-15 at 18:15 +0200, Alexander Kanavin wrote:
> On Fri, 15 Jul 2022 at 16:42, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > We'd always intended that site.conf was more something the user would
> > setup once for their environment and not need to change. It was
> > therefore deliberately left out of the auto-setup piece.
> 
> I don't think I understand this point in particular. If site.conf is
> copied once into a newly created, empty build directory, why is that
> problematic?

Originally, we actually wanted this to be something like from the
user's homedir, i.e. things they really would setup just once.

Doing that has downsides in that if not careful, things can "leak" into
a build unexpectedly too so we never did end up enabling it from $HOME.
I do still think of it that way though.

In that sense, it was something a user would provide, not something
that would be provisioned by automated scripting. If there are settings
to be provisioned by a layer, we have local.conf for that? Why do we
need another file too?

Cheers,

Richard

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Fri, 15 Jul 2022 at 18:24, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> In that sense, it was something a user would provide, not something
> that would be provisioned by automated scripting. If there are settings
> to be provisioned by a layer, we have local.conf for that? Why do we
> need another file too?

Right, I'm ok with organizational proxies and servers going into
local.conf. Then site.conf.sample and local.conf.extended.sample
should be promoted in the banner when setting up from the meta-poky
template.

Alex

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexandre Belloni]

On 15/07/2022 18:00:57+0200, Alexander Kanavin wrote:
> On Fri, 15 Jul 2022 at 16:42, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > We'd always intended that site.conf was more something the user would
> > setup once for their environment and not need to change. It was
> > therefore deliberately left out of the auto-setup piece.
> > I hadn't remembered we have an sample version of it. I think a sample
> > is fine as an example of how a site might setup a shared downloads
> > directory or common proxy information but I don't think the layer
> > should be installing it, particularly if it can trample an existing
> > file which I think the patch could as it stands.
> 
> Ah I see where the problem is: the sample file is installed subject to
> existence of local.conf in the destination build directory, which is
> ok most of the time, but not all of the time. It should check for
> existence of site.conf.
> 
> The idea is to serve two use cases:
> - when setting up a build from the poky template, there's a sample
> file installed. Same as local.conf.sample, we give users something to
> play with via commented out examples. They'll never find it in
> meta-poky/conf, even you had no idea it's there :)
> - when setting up a build from an organizational template, the right
> servers get configured without any extra steps. Ross pointed out that
> can be achieved with 'meta-product/conf/site.conf', but that can't be
> edited when needed, and can't be used to configure different settings
> for different builds (e.g. where the caches are - such things can be
> under access control restrictions).
> 
> I think this is worth reworking and making the commit message clearer.
> Or we should come up with a different way to provide 'official' way to
> set up site.conf, because otherwise it's back to 'write a custom
> script'.
> 

I don't think you need a custom script for site.conf, it is searched for
relative to BBPATH? I usually just put it directly as conf/site.conf in
the custom layer so it is in version control and not copied to BUILDDIR.

> Alex

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#168121): https://lists.openembedded.org/g/openembedded-core/message/168121
> Mute This Topic: https://lists.openembedded.org/mt/92212803/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH 1/2] scripts/oe-setup-builddir: copy site.conf.sample out of template directories (if it exists)

[Alexander Kanavin]

On Fri, 15 Jul 2022 at 23:57, Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
> I don't think you need a custom script for site.conf, it is searched for
> relative to BBPATH? I usually just put it directly as conf/site.conf in
> the custom layer so it is in version control and not copied to BUILDDIR.

I was thinking of situations where the settings that go there vary
between build configurations. But we settled with RP that then they
should be in local.conf.

Alex