* [PATCH 2/6] bitbake-layers: add a command to save the active build configuration as a template into a layer
2022-07-29 14:10 [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Alexander Kanavin
@ 2022-07-29 14:10 ` Alexander Kanavin
2022-07-29 14:10 ` [PATCH 3/6] meta/files: add layer setup JSON schema and example Alexander Kanavin
` (4 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-29 14:10 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
This is the reverse of setting up a build by pointing TEMPLATECONF to a directory
with a template and running '. oe-init-build-env': this takes the config files from build/conf,
replaces site-specific paths in bblayers.conf with ##OECORE##-relative paths, and copies
the config files into a specified layer under a specified template name.
In many or perhaps most cases such static prefabricated configurations (that require no
further editing) are just enough, and I believe they should be offered by the
official configuration management. On the other hand, generating build configurations with a
sufficiently versatile tool is a far more complex problem, and one we should try to tackle
once we see where and how static configs fall short.
Tooling to discover and select these templates when setting up a build will be provided later on.
How to use:
alex@Zen2:/srv/work/alex/poky/build-layersetup$ bitbake-layers save-build-conf ../../meta-alex/ test-1
NOTE: Starting bitbake server...
NOTE: Configuration template placed into /srv/work/alex/meta-alex/conf/templates/test-1
Please review the files in there, and particularly provide a configuration description in /srv/work/alex/meta-alex/conf/templates/test-1/conf-notes.txt
You can try out the configuration with
TEMPLATECONF=/srv/work/alex/meta-alex/conf/templates/test-1 . /srv/work/alex/poky/oe-init-build-env build-try-test-1
alex@Zen2:/srv/work/alex/poky/build-layersetup$
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
meta/lib/bblayers/buildconf.py | 83 ++++++++++++++++++++++++
meta/lib/oeqa/selftest/cases/bblayers.py | 5 ++
2 files changed, 88 insertions(+)
create mode 100644 meta/lib/bblayers/buildconf.py
diff --git a/meta/lib/bblayers/buildconf.py b/meta/lib/bblayers/buildconf.py
new file mode 100644
index 0000000000..ece8836d5a
--- /dev/null
+++ b/meta/lib/bblayers/buildconf.py
@@ -0,0 +1,83 @@
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import logging
+import os
+import stat
+import sys
+import shutil
+import json
+
+import bb.utils
+import bb.process
+
+from bblayers.common import LayerPlugin
+
+logger = logging.getLogger('bitbake-layers')
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
+
+import oe.buildcfg
+
+def plugin_init(plugins):
+ return BuildConfPlugin()
+
+class BuildConfPlugin(LayerPlugin):
+ notes_fixme = """FIXME: Please place here the description of this build configuration.
+It will be shown to the users when they set up their builds via TEMPLATECONF.
+"""
+
+ def _save_conf(self, templatename, templatepath, oecorepath, relpaths_to_oecore):
+ confdir = os.path.join(os.environ["BBPATH"], "conf")
+ destdir = os.path.join(templatepath, "conf", "templates", templatename)
+ os.makedirs(destdir, exist_ok=True)
+
+ with open(os.path.join(confdir, "local.conf")) as src:
+ with open(os.path.join(destdir, "local.conf.sample"), 'w') as dest:
+ dest.write(src.read())
+
+ with open(os.path.join(confdir, "bblayers.conf")) as src:
+ with open(os.path.join(destdir, "bblayers.conf.sample"), 'w') as dest:
+ bblayers_data = src.read()
+
+ for (abspath, relpath) in relpaths_to_oecore:
+ bblayers_data = bblayers_data.replace(abspath, "##OEROOT##/" + relpath)
+ dest.write(bblayers_data)
+
+ with open(os.path.join(destdir, "conf-notes.txt"), 'w') as dest:
+ dest.write(self.notes_fixme)
+
+ logger.info("""Configuration template placed into {}
+Please review the files in there, and particularly provide a configuration description in {}
+You can try out the configuration with
+TEMPLATECONF={} . {}/oe-init-build-env build-try-{}"""
+.format(destdir, os.path.join(destdir, "conf-notes.txt"), destdir, oecorepath, templatename))
+
+ def do_save_build_conf(self, args):
+ """ Save the currently active build configuration (conf/local.conf, conf/bblayers.conf) as a template into a layer.\n This template can later be used for setting up builds via TEMPLATECONF. """
+ repos = {}
+ layers = oe.buildcfg.get_layer_revisions(self.tinfoil.config_data)
+ targetlayer = None
+ oecore = None
+
+ for l in layers:
+ if l[0] == os.path.abspath(args.layerpath):
+ targetlayer = l[0]
+ if l[1] == 'meta':
+ oecore = os.path.dirname(l[0])
+
+ if not targetlayer:
+ logger.error("Layer {} not in one of the currently enabled layers:\n{}".format(args.layerpath, "\n".join([l[0] for l in layers])))
+ elif not oecore:
+ logger.error("Openembedded-core not in one of the currently enabled layers:\n{}".format("\n".join([l[0] for l in layers])))
+ else:
+ relpaths_to_oecore = [(l[0], os.path.relpath(l[0], start=oecore)) for l in layers]
+ self._save_conf(args.templatename, targetlayer, oecore, relpaths_to_oecore)
+
+ def register_commands(self, sp):
+ parser_build_conf = self.add_command(sp, 'save-build-conf', self.do_save_build_conf, parserecipes=False)
+ parser_build_conf.add_argument('layerpath',
+ help='The path to the layer where the configuration template should be saved.')
+ parser_build_conf.add_argument('templatename',
+ help='The name of the configuration template.')
diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py b/meta/lib/oeqa/selftest/cases/bblayers.py
index 7d74833f61..8aa45e432e 100644
--- a/meta/lib/oeqa/selftest/cases/bblayers.py
+++ b/meta/lib/oeqa/selftest/cases/bblayers.py
@@ -111,6 +111,11 @@ class BitbakeLayers(OESelftestTestCase):
self.assertEqual(bb_vars['BBFILE_PRIORITY_%s' % layername], str(priority), 'BBFILE_PRIORITY_%s != %d' % (layername, priority))
+ result = runCmd('bitbake-layers save-build-conf {} {}'.format(layerpath, "buildconf-1"))
+ for f in ('local.conf.sample', 'bblayers.conf.sample', 'conf-notes.txt'):
+ fullpath = os.path.join(layerpath, "conf", "templates", "buildconf-1", f)
+ self.assertTrue(os.path.exists(fullpath), "Template configuration file {} not found".format(fullpath))
+
def get_recipe_basename(self, recipe):
recipe_file = ""
result = runCmd("bitbake-layers show-recipes -f %s" % recipe)
--
2.30.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 3/6] meta/files: add layer setup JSON schema and example
2022-07-29 14:10 [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Alexander Kanavin
2022-07-29 14:10 ` [PATCH 2/6] bitbake-layers: add a command to save the active build configuration as a template into a layer Alexander Kanavin
@ 2022-07-29 14:10 ` Alexander Kanavin
2022-07-29 14:10 ` [PATCH 4/6] bitbake-layers: add ability to save current layer repository configuration into a json file Alexander Kanavin
` (3 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-29 14:10 UTC (permalink / raw)
To: openembedded-core; +Cc: Joshua Watt, Alexander Kanavin
From: Joshua Watt <JPEWhacker@gmail.com>
Defines a common schema for layer setup that can be consumed by tools to
know how to fetch and assemble layers for end users. Also includes an
example of the layer setup that constructs poky/meta-intel/imaginary product layer
for reference.
The schema can be used to validate a layer setup file with the commands:
$ python3 -m pip install jsonschema
$ jsonschema -i meta/files/layers.example.json meta/files/layers.schema.json
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Alex: I made the following modifications to Joshua's original commit:
- moved the files from meta/lib to meta/files
- the example json showcases a multi-repo, multi-layer setup with additional
configurations and machines, instead of just poky - closer to a typical product
- added oe-selftest that validates the example json against the schema using python3-jsonschema-native
- the schema is modified so that:
-- all lists (sources, layers, remotes) are replaced by objects keyed by 'name' properties of the list items.
This allows using them as dicts inside Python, and makes the json more compact and readable.
-- added 'contains_this_file' property to source object
-- added 'buildconfigs', 'machines' and 'distros' properties to layer objects.
-- replaced 'remote' property with a 'oneOf' definition for git with a specific
'git-remote' property. 'oneOf' is problematic when schema validation fails:
the diagnostic is only that none of oneOf variants matched, which is too non-specific.
-- added 'describe' property to 'git-remote' object.
-- removed description property for a layer source: it is not clear how to add that
when auto-generating the json
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
meta/files/layers.example.json | 115 +++++++++++++++++++++
meta/files/layers.schema.json | 121 +++++++++++++++++++++++
meta/lib/oeqa/selftest/cases/bblayers.py | 16 ++-
3 files changed, 251 insertions(+), 1 deletion(-)
create mode 100644 meta/files/layers.example.json
create mode 100644 meta/files/layers.schema.json
diff --git a/meta/files/layers.example.json b/meta/files/layers.example.json
new file mode 100644
index 0000000000..b6e35c615d
--- /dev/null
+++ b/meta/files/layers.example.json
@@ -0,0 +1,115 @@
+{
+ "sources": {
+ "meta-alex": {
+ "contains_this_file": true,
+ "git-remote": {
+ "branch": "master",
+ "describe": "",
+ "remotes": {
+ "remote-alex": {
+ "uri": "https://github.com/kanavin/meta-alex"
+ }
+ },
+ "rev": "05b25605fb8b2399e4706d7323828676bf0da0b5"
+ },
+ "layers": {
+ "meta-alex": {
+ "buildconfigs": {
+ "conf/templates/configuration-gadget": {},
+ "conf/templates/configuration-gizmo": {}
+ },
+ "subpath": ""
+ }
+ },
+ "path": "meta-alex"
+ },
+ "meta-intel": {
+ "git-remote": {
+ "branch": "master",
+ "describe": "15.0-hardknott-3.3-310-g0a96edae",
+ "remotes": {
+ "origin": {
+ "uri": "git://git.yoctoproject.org/meta-intel"
+ }
+ },
+ "rev": "0a96edae609a3f48befac36af82cf1eed6786b4a"
+ },
+ "layers": {
+ "meta-intel": {
+ "machines": {
+ "intel-core2-32": {},
+ "intel-corei7-64": {},
+ "intel-skylake-64": {}
+ },
+ "subpath": ""
+ }
+ },
+ "path": "meta-intel"
+ },
+ "poky": {
+ "git-remote": {
+ "branch": "akanavin/setup-layers",
+ "describe": "4.1_M1-374-g9dda719b2a",
+ "remotes": {
+ "origin": {
+ "uri": "git://git.yoctoproject.org/poky"
+ },
+ "poky-contrib": {
+ "uri": "ssh://git@push.yoctoproject.org/poky-contrib"
+ }
+ },
+ "rev": "9dda719b2a4727a4d43a6ab8d9e23f8ca68790ec"
+ },
+ "layers": {
+ "meta": {
+ "distros": {
+ "defaultsetup": {}
+ },
+ "machines": {
+ "qemuarm": {},
+ "qemuarm64": {},
+ "qemuarmv5": {},
+ "qemumips": {},
+ "qemumips64": {},
+ "qemuppc": {},
+ "qemuppc64": {},
+ "qemuriscv32": {},
+ "qemuriscv64": {},
+ "qemux86": {},
+ "qemux86-64": {}
+ },
+ "subpath": "meta"
+ },
+ "meta-poky": {
+ "buildconfigs": {
+ "conf": {}
+ },
+ "distros": {
+ "poky": {},
+ "poky-altcfg": {},
+ "poky-bleeding": {},
+ "poky-tiny": {}
+ },
+ "subpath": "meta-poky"
+ },
+ "meta-selftest": {
+ "machines": {
+ "qemux86copy": {}
+ },
+ "subpath": "meta-selftest"
+ },
+ "meta-yocto-bsp": {
+ "machines": {
+ "beaglebone-yocto": {},
+ "edgerouter": {},
+ "genericx86": {},
+ "genericx86-64": {}
+ },
+ "subpath": "meta-yocto-bsp"
+ }
+ },
+ "path": "poky"
+ }
+ },
+ "version": "1.0"
+}
diff --git a/meta/files/layers.schema.json b/meta/files/layers.schema.json
new file mode 100644
index 0000000000..2603fae8f7
--- /dev/null
+++ b/meta/files/layers.schema.json
@@ -0,0 +1,121 @@
+{
+ "description": "OpenEmbedder Layer Setup Manifest",
+ "type": "object",
+ "additionalProperties": false,
+ "required": [
+ "version"
+ ],
+ "properties": {
+ "version": {
+ "description": "The version of this document; currently '1.0'",
+ "enum": ["1.0"]
+ },
+ "sources": {
+ "description": "The dict of layer sources",
+ "type": "object",
+ "patternProperties": { ".*" : {
+ "type": "object",
+ "description": "The upstream source from which a set of layers may be fetched",
+ "additionalProperties": false,
+ "required": [
+ "path"
+ ],
+ "properties": {
+ "path": {
+ "description": "The path where this layer source will be placed when fetching",
+ "type": "string"
+ },
+ "contains_this_file": {
+ "description": "Whether the directory with the layer source also contains this json description. Tools may want to skip the checkout of the source then.",
+ "type": "boolean"
+ },
+ "layers": {
+ "description": "The dict of layers to be used from this upstream source",
+ "type": "object",
+ "patternProperties": { ".*" : {
+ "description": "A layer from the upstream source",
+ "type": "object",
+ "additionalProperties": false,
+ "properties": {
+ "subpath": {
+ "description": "The subpath (relative to the source root) for this layer. Omit if the source root is the layer path",
+ "type": "string"
+ },
+ "buildconfigs": {
+ "description": "A dict of template build configurations in the layer, keyed by relative path to the layer root",
+ "type": "object",
+ "patternProperties": { ".*" : {
+ "description": "Properties of a build confuguration template",
+ "type": "object",
+ "additionalProperties": false,
+ "properties":{}
+ }}
+ },
+ "machines": {
+ "description": "A dict of machine definitions in the layer",
+ "type": "object",
+ "patternProperties": { ".*" : {
+ "description": "Properties of a machine definition",
+ "type": "object",
+ "additionalProperties": false,
+ "properties":{}
+ }}
+ },
+ "distros": {
+ "description": "A dict of distro definitions in the layer",
+ "type": "object",
+ "patternProperties": { ".*" : {
+ "description": "Properties of a distro definition",
+ "type": "object",
+ "additionalProperties": false,
+ "properties":{}
+ }}
+ }
+ }
+ }}
+ },
+ "git-remote": {
+ "description": "A remote git source from which to fetch",
+ "type": "object",
+ "additionalProperties": false,
+ "required": [
+ "rev"
+ ],
+ "properties": {
+ "branch": {
+ "description": "The git branch to fetch (optional)",
+ "type": "string"
+ },
+ "rev": {
+ "description": "The git revision to checkout",
+ "type": "string"
+ },
+ "describe": {
+ "description": "The output of 'git describe' (human readable description of the revision using tags in revision history).",
+ "type": "string"
+ },
+ "remotes": {
+ "description": "The dict of git remotes to add to this repository",
+ "type": "object",
+ "patternProperties": { ".*" : {
+ "description": "A git remote",
+ "type": "object",
+ "addtionalProperties": false,
+ "required": [
+ "uri"
+ ],
+ "properties": {
+ "uri": {
+ "description": "The URI for the remote",
+ "type": "string"
+ }
+ }
+ }}
+ }
+ }
+ }
+ }
+ }
+ }}
+ }
+}
diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py b/meta/lib/oeqa/selftest/cases/bblayers.py
index 8aa45e432e..4f4952f3eb 100644
--- a/meta/lib/oeqa/selftest/cases/bblayers.py
+++ b/meta/lib/oeqa/selftest/cases/bblayers.py
@@ -6,12 +6,16 @@ import os
import re
import oeqa.utils.ftools as ftools
-from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars
+from oeqa.utils.commands import runCmd, get_bb_var, get_bb_vars, bitbake
from oeqa.selftest.case import OESelftestTestCase
class BitbakeLayers(OESelftestTestCase):
+ def setUpLocal(self):
+ bitbake("python3-jsonschema-native")
+ bitbake("-c addto_recipe_sysroot python3-jsonschema-native")
+
def test_bitbakelayers_layerindexshowdepends(self):
result = runCmd('bitbake-layers layerindex-show-depends meta-poky')
find_in_contents = re.search("openembedded-core", result.output)
@@ -126,3 +130,13 @@ class BitbakeLayers(OESelftestTestCase):
self.assertTrue(os.path.isfile(recipe_file), msg = "Can't find recipe file for %s" % recipe)
return os.path.basename(recipe_file)
+
+ def validate_layersjson(self, json):
+ python = os.path.join(get_bb_var('STAGING_BINDIR', 'python3-jsonschema-native'), 'nativepython3')
+ jsonvalidator = os.path.join(get_bb_var('STAGING_BINDIR', 'python3-jsonschema-native'), 'jsonschema')
+ jsonschema = os.path.join(get_bb_var('COREBASE'), 'meta/files/layers.schema.json')
+ result = runCmd("{} {} -i {} {}".format(python, jsonvalidator, json, jsonschema))
+
+ def test_validate_examplelayersjson(self):
+ json = os.path.join(get_bb_var('COREBASE'), "meta/files/layers.example.json")
+ self.validate_layersjson(json)
--
2.30.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 4/6] bitbake-layers: add ability to save current layer repository configuration into a json file
2022-07-29 14:10 [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Alexander Kanavin
2022-07-29 14:10 ` [PATCH 2/6] bitbake-layers: add a command to save the active build configuration as a template into a layer Alexander Kanavin
2022-07-29 14:10 ` [PATCH 3/6] meta/files: add layer setup JSON schema and example Alexander Kanavin
@ 2022-07-29 14:10 ` Alexander Kanavin
2022-07-29 14:10 ` [PATCH 5/6] scripts/oe-setup-layers: add a script that restores the layer configuration from " Alexander Kanavin
` (2 subsequent siblings)
5 siblings, 0 replies; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-29 14:10 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
This addresses a long standing gap in the core offering:
there is no tooling to capture the currently configured layers
with their revisions, or restore the layers from a configuration
file (without using external tools, some of which aren't particularly
suitable for the task). This plugin addresses the 'capture' part.
How to save a layer configuration:
a) Running with default choices:
NOTE: Starting bitbake server...
NOTE: Created /srv/work/alex/meta-alex/setup-layers.json
NOTE: Created /srv/work/alex/meta-alex/setup-layers
b) Command line options:
alex@Zen2:/srv/work/alex/poky/build-layersetup$ bitbake-layers create-layers-setup -h
NOTE: Starting bitbake server...
usage: bitbake-layers create-layers-setup [-h] [--output-prefix OUTPUT_PREFIX] [--json-only] destdir
Writes out a python script and a json config that replicate the directory structure and revisions of the layers in a current build.
positional arguments:
destdir Directory where to write the output
(if it is inside one of the layers, the layer becomes a bootstrap repository and thus will be excluded from fetching by the script).
optional arguments:
-h, --help show this help message and exit
--output-prefix OUTPUT_PREFIX, -o OUTPUT_PREFIX
File name prefix for the output files, if the default (setup-layers) is undesirable.
--json-only Write only the layer configuruation in json format. Otherwise, also a copy of poky/scripts/oe-setup-layers is provided, which is a self contained python script that fetches all the needed layers and sets them to correct revisions using the data from the json.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
meta/lib/bblayers/makesetup.py | 139 +++++++++++++++++++++++++++++++++
1 file changed, 139 insertions(+)
create mode 100644 meta/lib/bblayers/makesetup.py
diff --git a/meta/lib/bblayers/makesetup.py b/meta/lib/bblayers/makesetup.py
new file mode 100644
index 0000000000..b7f62900f4
--- /dev/null
+++ b/meta/lib/bblayers/makesetup.py
@@ -0,0 +1,139 @@
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import logging
+import os
+import stat
+import sys
+import shutil
+import json
+
+import bb.utils
+import bb.process
+
+from bblayers.common import LayerPlugin
+
+logger = logging.getLogger('bitbake-layers')
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
+
+import oe.buildcfg
+
+def plugin_init(plugins):
+ return MakeSetupPlugin()
+
+class MakeSetupPlugin(LayerPlugin):
+
+ def _write_python(self, input, output):
+ with open(input) as f:
+ script = f.read()
+ with open(output, 'w') as f:
+ f.write(script)
+ st = os.stat(output)
+ os.chmod(output, st.st_mode | stat.S_IEXEC | stat.S_IXGRP | stat.S_IXOTH)
+
+ def _write_json(self, repos, output):
+ with open(output, 'w') as f:
+ json.dump(repos, f, sort_keys=True, indent=4)
+
+ def _get_repo_path(self, layer_path):
+ repo_path, _ = bb.process.run('git rev-parse --show-toplevel', cwd=layer_path)
+ return repo_path.strip()
+
+ def _get_remotes(self, repo_path):
+ remotes = {}
+ remotes_list,_ = bb.process.run('git remote', cwd=repo_path)
+ for r in remotes_list.split():
+ uri,_ = bb.process.run('git remote get-url {r}'.format(r=r), cwd=repo_path)
+ remotes[r] = {'uri':uri.strip()}
+ return remotes
+
+ def _get_describe(self, repo_path):
+ try:
+ describe,_ = bb.process.run('git describe --tags', cwd=repo_path)
+ except bb.process.ExecutionError:
+ return ""
+ return describe.strip()
+
+ def _get_confs(self, conf_path):
+ try:
+ files = os.listdir(conf_path)
+ except:
+ return []
+ return {f.replace(".conf",""):{} for f in files if f.endswith(".conf")}
+
+ def _get_distros(self, layer_path):
+ return self._get_confs(os.path.join(layer_path, "conf/distro"))
+
+ def _get_machines(self, layer_path):
+ return self._get_confs(os.path.join(layer_path, "conf/machine"))
+
+ def _get_buildconfigs(self, layerpath):
+ return {os.path.relpath(dir, start=layerpath):{} for (dir, subdirs, files) in os.walk(layerpath) if 'local.conf.sample' in files and 'bblayers.conf.sample' in files}
+
+ def _make_repo_config(self, destdir):
+ repos = {}
+ layers = oe.buildcfg.get_layer_revisions(self.tinfoil.config_data)
+ try:
+ destdir_repo = self._get_repo_path(destdir)
+ except bb.process.ExecutionError:
+ destdir_repo = None
+
+ for (l_path, l_name, l_branch, l_rev, l_ismodified) in layers:
+ if l_name == 'workspace':
+ continue
+ if l_ismodified:
+ logger.error("Layer {name} in {path} has uncommitted modifications or is not in a git repository.".format(name=l_name,path=l_path))
+ return
+ repo_path = self._get_repo_path(l_path)
+ if repo_path not in repos.keys():
+ repos[repo_path] = {'path':os.path.basename(repo_path),'layers':{},'git-remote':{'rev':l_rev, 'branch':l_branch, 'remotes':self._get_remotes(repo_path), 'describe':self._get_describe(repo_path)}}
+ if repo_path == destdir_repo:
+ repos[repo_path]['contains_this_file'] = True
+ if not repos[repo_path]['git-remote']['remotes'] and not repos[repo_path]['contains_this_file']:
+ logger.error("Layer repository in {path} does not have any remotes configured. Please add at least one with 'git remote add'.".format(path=repo_path))
+ return
+ repos[repo_path]['layers'][l_name] = {'subpath':l_path.replace(repo_path,'')[1:]}
+ distros = self._get_distros(l_path)
+ machines = self._get_machines(l_path)
+ buildconfigs = self._get_buildconfigs(l_path)
+ if distros:
+ repos[repo_path]['layers'][l_name]['distros'] = distros
+ if machines:
+ repos[repo_path]['layers'][l_name]['machines'] = machines
+ if buildconfigs:
+ repos[repo_path]['layers'][l_name]['buildconfigs'] = buildconfigs
+
+ top_path = os.path.commonpath([os.path.dirname(r) for r in repos.keys()])
+
+ repos_nopaths = {}
+ for r in repos.keys():
+ r_nopath = os.path.basename(r)
+ repos_nopaths[r_nopath] = repos[r]
+ r_relpath = os.path.relpath(r, top_path)
+ repos_nopaths[r_nopath]['path'] = r_relpath
+ return repos_nopaths
+
+ def do_make_setup(self, args):
+ """ Writes out a python script and a json config that replicate the directory structure and revisions of the layers in a current build. """
+ repos = self._make_repo_config(args.destdir)
+ json = {"version":"1.0","sources":repos}
+ if not repos:
+ raise Exception("Could not determine layer sources")
+ output = args.output_prefix or "setup-layers"
+ output = os.path.join(os.path.abspath(args.destdir),output)
+ self._write_json(json, output + ".json")
+ logger.info('Created {}.json'.format(output))
+ if not args.json_only:
+ self._write_python(os.path.join(os.path.dirname(__file__),'../../../scripts/oe-setup-layers'), output)
+ logger.info('Created {}'.format(output))
+
+ def register_commands(self, sp):
+ parser_setup_layers = self.add_command(sp, 'create-layers-setup', self.do_make_setup, parserecipes=False)
+ parser_setup_layers.add_argument('destdir',
+ help='Directory where to write the output\n(if it is inside one of the layers, the layer becomes a bootstrap repository and thus will be excluded from fetching by the script).')
+ parser_setup_layers.add_argument('--output-prefix', '-o',
+ help='File name prefix for the output files, if the default (setup-layers) is undesirable.')
+ parser_setup_layers.add_argument('--json-only', action='store_true',
+ help='Write only the layer configuruation in json format. Otherwise, also a copy of poky/scripts/oe-setup-layers is provided, which is a self contained python script that fetches all the needed layers and sets them to correct revisions using the data from the json.')
--
2.30.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 5/6] scripts/oe-setup-layers: add a script that restores the layer configuration from a json file
2022-07-29 14:10 [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Alexander Kanavin
` (2 preceding siblings ...)
2022-07-29 14:10 ` [PATCH 4/6] bitbake-layers: add ability to save current layer repository configuration into a json file Alexander Kanavin
@ 2022-07-29 14:10 ` Alexander Kanavin
2022-08-02 13:12 ` [OE-core] " Luca Ceresoli
2022-07-29 14:10 ` [PATCH 6/6] selftest/bblayers: add a test for creating a layer setup and using it to restore the layers Alexander Kanavin
2022-07-29 19:55 ` [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Peter Kjellerstedt
5 siblings, 1 reply; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-29 14:10 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
This script can be used directly from poky, or can be copied directly into a
layer or any other repository - it is self-suffucient and requires only python3
and git on the host where it will run. It is also copied by the bitbake-layers
layers-setup plugin together with the json, unless requested otherwise.
1. How to restore the layers from the saved configuration:
a) Clone the bootstrap layer or some other repository to obtain the json config and the setup script that can use it.
(use 'bitbake-layers create-layer-setup' from the previous commit to create them)
b) Running with default options:
(note: this will work to update an existing checkout as well)
alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers
Note: not checking out source meta-alex, use --force-bootstraplayer-checkout to override.
Setting up source meta-intel, revision 15.0-hardknott-3.3-310-g0a96edae, branch master
Running 'git init -q /srv/work/alex/my-build/meta-intel'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/meta-intel' in /srv/work/alex/my-build/meta-intel
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/meta-intel
Running 'git checkout -q 0a96edae609a3f48befac36af82cf1eed6786b4a' in /srv/work/alex/my-build/meta-intel
Setting up source poky, revision 4.1_M1-372-g55483d28f2, branch akanavin/setup-layers
Running 'git init -q /srv/work/alex/my-build/poky'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/poky' in /srv/work/alex/my-build/poky
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/poky
Running 'git remote remove poky-contrib > /dev/null 2>&1; git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib' in /srv/work/alex/my-build/poky
Running 'git fetch -q poky-contrib || true' in /srv/work/alex/my-build/poky
Running 'git checkout -q 11db0390b02acac1324e0f827beb0e2e3d0d1d63' in /srv/work/alex/my-build/poky
Available build configurations:
/srv/work/alex/my-build/meta-alex/conf/templates/configuration-gadget
/srv/work/alex/my-build/meta-alex/conf/templates/configuration-gizmo
/srv/work/alex/my-build/poky/meta-poky/conf
You can set up a build with:
TEMPLATECONF=one/of/the/above . /srv/work/alex/my-build/poky/oe-init-build-env
2. Command line options:
alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers -h
usage: setup-layers [-h] [--force-bootstraplayer-checkout] [--destdir DESTDIR] [--jsondata JSONDATA]
A self contained python script that fetches all the needed layers and sets them to correct revisions
optional arguments:
-h, --help show this help message and exit
--force-bootstraplayer-checkout
Force the checkout of the layer containing this file (by default it is presumed that as this script is in it, the layer is already in place).
--destdir DESTDIR Where to check out the layers (default is /srv/work/alex/my-build).
--jsondata JSONDATA File containing the layer data in json format (default is /srv/work/alex/my-build/meta-alex/setup-layers.json).
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
scripts/oe-setup-layers | 85 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 85 insertions(+)
create mode 100755 scripts/oe-setup-layers
diff --git a/scripts/oe-setup-layers b/scripts/oe-setup-layers
new file mode 100755
index 0000000000..017c400ad6
--- /dev/null
+++ b/scripts/oe-setup-layers
@@ -0,0 +1,85 @@
+#!/usr/bin/env python3
+#
+# This file was copied from poky(or oe-core)/scripts/oe-setup-layers by running
+#
+# bitbake-layers create-layers-setup destdir
+#
+# It is recommended that you do not modify this file directly, but rather re-run the above command to get the freshest upstream copy.
+#
+
+import argparse
+import json
+import os
+import subprocess
+
+def _do_checkout(args, json):
+ layers = json['sources']
+ buildconfs = []
+ oecorepath = ""
+ for l_name in layers:
+ l_data = layers[l_name]
+ layerdir = os.path.abspath(os.path.join(args['destdir'], l_data['path']))
+
+ for ll_name in l_data['layers']:
+ if ll_name == 'meta':
+ oecorepath = layerdir
+ ll_data = l_data['layers'][ll_name]
+ if 'buildconfigs' in ll_data:
+ for c in ll_data['buildconfigs']:
+ buildconfs.append(os.path.join(layerdir, ll_data['subpath'], c))
+
+ if 'contains_this_file' in l_data.keys():
+ force_arg = 'force_bootstraplayer_checkout'
+ if not args[force_arg]:
+ print('Note: not checking out source {layer}, use {layerflag} to override.'.format(layer=l_name, layerflag='--force-bootstraplayer-checkout'))
+ continue
+ l_remote = l_data['git-remote']
+ rev = l_remote['rev']
+ desc = l_remote['describe']
+ if not desc:
+ desc = rev[:10]
+ branch = l_remote['branch']
+ remotes = l_remote['remotes']
+
+ print('\nSetting up source {}, revision {}, branch {}'.format(l_name, desc, branch))
+ cmd = 'git init -q {}'.format(layerdir)
+ print("Running '{}'".format(cmd))
+ subprocess.check_output(cmd, text=True, shell=True)
+
+ for remote in remotes:
+ cmd = "git remote remove {} > /dev/null 2>&1; git remote add {} {}".format(remote, remote, remotes[remote]['uri'])
+ print("Running '{}' in {}".format(cmd, layerdir))
+ subprocess.check_output(cmd, text=True, shell=True, cwd=layerdir)
+
+ cmd = "git fetch -q {} || true".format(remote)
+ print("Running '{}' in {}".format(cmd, layerdir))
+ subprocess.check_output(cmd, text=True, shell=True, cwd=layerdir)
+
+ cmd = 'git checkout -q {}'.format(rev)
+ print("Running '{}' in {}".format(cmd, layerdir))
+ subprocess.check_output(cmd, text=True, shell=True, cwd=layerdir)
+
+
+ print("\nAvailable build configurations:\n{}\n".format("\n".join(buildconfs)))
+ print("You can set up a build with:\nTEMPLATECONF=one/of/the/above . {}/oe-init-build-env".format(oecorepath))
+
+
+parser = argparse.ArgumentParser(description="A self contained python script that fetches all the needed layers and sets them to correct revisions using data in a json format from a separate file. The json data can be created from an active build directory with 'bitbake-layers create-layers-setup destdir' and there's a sample file and a schema in meta/files/")
+
+parser.add_argument('--force-bootstraplayer-checkout', action='store_true',
+ help='Force the checkout of the layer containing this file (by default it is presumed that as this script is in it, the layer is already in place).')
+
+try:
+ defaultdest = os.path.dirname(subprocess.check_output('git rev-parse --show-toplevel', text=True, shell=True, cwd=os.path.dirname(__file__)))
+except subprocess.CalledProcessError as e:
+ defaultdest = os.path.abspath(".")
+
+parser.add_argument('--destdir', default=defaultdest, help='Where to check out the layers (default is {defaultdest}).'.format(defaultdest=defaultdest))
+parser.add_argument('--jsondata', default=__file__+".json", help='File containing the layer data in json format (default is {defaultjson}).'.format(defaultjson=__file__+".json"))
+
+args = parser.parse_args()
+
+with open(args.jsondata) as f:
+ json = json.load(f)
+
+_do_checkout(vars(args), json)
--
2.30.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [OE-core] [PATCH 5/6] scripts/oe-setup-layers: add a script that restores the layer configuration from a json file
2022-07-29 14:10 ` [PATCH 5/6] scripts/oe-setup-layers: add a script that restores the layer configuration from " Alexander Kanavin
@ 2022-08-02 13:12 ` Luca Ceresoli
2022-08-02 14:31 ` Alexander Kanavin
0 siblings, 1 reply; 15+ messages in thread
From: Luca Ceresoli @ 2022-08-02 13:12 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-core, Alexander Kanavin
Hi Alex,
On Fri, 29 Jul 2022 16:10:48 +0200
"Alexander Kanavin" <alex.kanavin@gmail.com> wrote:
> This script can be used directly from poky, or can be copied directly into a
> layer or any other repository - it is self-suffucient and requires only python3
> and git on the host where it will run. It is also copied by the bitbake-layers
> layers-setup plugin together with the json, unless requested otherwise.
>
> 1. How to restore the layers from the saved configuration:
>
> a) Clone the bootstrap layer or some other repository to obtain the json config and the setup script that can use it.
> (use 'bitbake-layers create-layer-setup' from the previous commit to create them)
>
> b) Running with default options:
> (note: this will work to update an existing checkout as well)
>
> alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers
> Note: not checking out source meta-alex, use --force-bootstraplayer-checkout to override.
>
> Setting up source meta-intel, revision 15.0-hardknott-3.3-310-g0a96edae, branch master
> Running 'git init -q /srv/work/alex/my-build/meta-intel'
> Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/meta-intel' in /srv/work/alex/my-build/meta-intel
> Running 'git fetch -q origin || true' in /srv/work/alex/my-build/meta-intel
> Running 'git checkout -q 0a96edae609a3f48befac36af82cf1eed6786b4a' in /srv/work/alex/my-build/meta-intel
>
> Setting up source poky, revision 4.1_M1-372-g55483d28f2, branch akanavin/setup-layers
> Running 'git init -q /srv/work/alex/my-build/poky'
> Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/poky' in /srv/work/alex/my-build/poky
> Running 'git fetch -q origin || true' in /srv/work/alex/my-build/poky
> Running 'git remote remove poky-contrib > /dev/null 2>&1; git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib' in /srv/work/alex/my-build/poky
> Running 'git fetch -q poky-contrib || true' in /srv/work/alex/my-build/poky
> Running 'git checkout -q 11db0390b02acac1324e0f827beb0e2e3d0d1d63' in /srv/work/alex/my-build/poky
>
> Available build configurations:
> /srv/work/alex/my-build/meta-alex/conf/templates/configuration-gadget
> /srv/work/alex/my-build/meta-alex/conf/templates/configuration-gizmo
> /srv/work/alex/my-build/poky/meta-poky/conf
>
> You can set up a build with:
> TEMPLATECONF=one/of/the/above . /srv/work/alex/my-build/poky/oe-init-build-env
>
> 2. Command line options:
>
> alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers -h
> usage: setup-layers [-h] [--force-bootstraplayer-checkout] [--destdir DESTDIR] [--jsondata JSONDATA]
>
> A self contained python script that fetches all the needed layers and sets them to correct revisions
>
> optional arguments:
> -h, --help show this help message and exit
> --force-bootstraplayer-checkout
> Force the checkout of the layer containing this file (by default it is presumed that as this script is in it, the layer is already in place).
> --destdir DESTDIR Where to check out the layers (default is /srv/work/alex/my-build).
> --jsondata JSONDATA File containing the layer data in json format (default is /srv/work/alex/my-build/meta-alex/setup-layers.json).
>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
This patch triggers an AB error:
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3916/steps/15/logs/stdio
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [OE-core] [PATCH 5/6] scripts/oe-setup-layers: add a script that restores the layer configuration from a json file
2022-08-02 13:12 ` [OE-core] " Luca Ceresoli
@ 2022-08-02 14:31 ` Alexander Kanavin
0 siblings, 0 replies; 15+ messages in thread
From: Alexander Kanavin @ 2022-08-02 14:31 UTC (permalink / raw)
To: Luca Ceresoli; +Cc: OE-core, Alexander Kanavin
That's python 3.6 giving an error about a 3.7 and later parameter. I
sent an adjusted patchset.
Alex
On Tue, 2 Aug 2022 at 15:12, Luca Ceresoli <luca.ceresoli@bootlin.com> wrote:
>
> Hi Alex,
>
> On Fri, 29 Jul 2022 16:10:48 +0200
> "Alexander Kanavin" <alex.kanavin@gmail.com> wrote:
>
> > This script can be used directly from poky, or can be copied directly into a
> > layer or any other repository - it is self-suffucient and requires only python3
> > and git on the host where it will run. It is also copied by the bitbake-layers
> > layers-setup plugin together with the json, unless requested otherwise.
> >
> > 1. How to restore the layers from the saved configuration:
> >
> > a) Clone the bootstrap layer or some other repository to obtain the json config and the setup script that can use it.
> > (use 'bitbake-layers create-layer-setup' from the previous commit to create them)
> >
> > b) Running with default options:
> > (note: this will work to update an existing checkout as well)
> >
> > alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers
> > Note: not checking out source meta-alex, use --force-bootstraplayer-checkout to override.
> >
> > Setting up source meta-intel, revision 15.0-hardknott-3.3-310-g0a96edae, branch master
> > Running 'git init -q /srv/work/alex/my-build/meta-intel'
> > Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/meta-intel' in /srv/work/alex/my-build/meta-intel
> > Running 'git fetch -q origin || true' in /srv/work/alex/my-build/meta-intel
> > Running 'git checkout -q 0a96edae609a3f48befac36af82cf1eed6786b4a' in /srv/work/alex/my-build/meta-intel
> >
> > Setting up source poky, revision 4.1_M1-372-g55483d28f2, branch akanavin/setup-layers
> > Running 'git init -q /srv/work/alex/my-build/poky'
> > Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/poky' in /srv/work/alex/my-build/poky
> > Running 'git fetch -q origin || true' in /srv/work/alex/my-build/poky
> > Running 'git remote remove poky-contrib > /dev/null 2>&1; git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib' in /srv/work/alex/my-build/poky
> > Running 'git fetch -q poky-contrib || true' in /srv/work/alex/my-build/poky
> > Running 'git checkout -q 11db0390b02acac1324e0f827beb0e2e3d0d1d63' in /srv/work/alex/my-build/poky
> >
> > Available build configurations:
> > /srv/work/alex/my-build/meta-alex/conf/templates/configuration-gadget
> > /srv/work/alex/my-build/meta-alex/conf/templates/configuration-gizmo
> > /srv/work/alex/my-build/poky/meta-poky/conf
> >
> > You can set up a build with:
> > TEMPLATECONF=one/of/the/above . /srv/work/alex/my-build/poky/oe-init-build-env
> >
> > 2. Command line options:
> >
> > alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers -h
> > usage: setup-layers [-h] [--force-bootstraplayer-checkout] [--destdir DESTDIR] [--jsondata JSONDATA]
> >
> > A self contained python script that fetches all the needed layers and sets them to correct revisions
> >
> > optional arguments:
> > -h, --help show this help message and exit
> > --force-bootstraplayer-checkout
> > Force the checkout of the layer containing this file (by default it is presumed that as this script is in it, the layer is already in place).
> > --destdir DESTDIR Where to check out the layers (default is /srv/work/alex/my-build).
> > --jsondata JSONDATA File containing the layer data in json format (default is /srv/work/alex/my-build/meta-alex/setup-layers.json).
> >
> > Signed-off-by: Alexander Kanavin <alex@linutronix.de>
>
> This patch triggers an AB error:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/3916/steps/15/logs/stdio
>
> --
> Luca Ceresoli, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 6/6] selftest/bblayers: add a test for creating a layer setup and using it to restore the layers
2022-07-29 14:10 [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Alexander Kanavin
` (3 preceding siblings ...)
2022-07-29 14:10 ` [PATCH 5/6] scripts/oe-setup-layers: add a script that restores the layer configuration from " Alexander Kanavin
@ 2022-07-29 14:10 ` Alexander Kanavin
2022-08-16 20:29 ` [OE-core] " Richard Purdie
2022-07-29 19:55 ` [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Peter Kjellerstedt
5 siblings, 1 reply; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-29 14:10 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
This does a basic run-through of the bitbake-layers plugin, and the resulting json layer config
and the layer setup script that uses it. Only poky is actually fetched by the script.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
meta/lib/oeqa/selftest/cases/bblayers.py | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py b/meta/lib/oeqa/selftest/cases/bblayers.py
index 4f4952f3eb..1f5c576356 100644
--- a/meta/lib/oeqa/selftest/cases/bblayers.py
+++ b/meta/lib/oeqa/selftest/cases/bblayers.py
@@ -140,3 +140,15 @@ class BitbakeLayers(OESelftestTestCase):
def test_validate_examplelayersjson(self):
json = os.path.join(get_bb_var('COREBASE'), "meta/files/layers.example.json")
self.validate_layersjson(json)
+
+ def test_bitbakelayers_setup(self):
+ result = runCmd('bitbake-layers create-layers-setup {}'.format(self.testlayer_path))
+ jsonfile = os.path.join(self.testlayer_path, "setup-layers.json")
+ self.validate_layersjson(jsonfile)
+
+ testcheckoutdir = os.path.join(self.builddir, 'test-layer-checkout')
+ result = runCmd('{}/setup-layers --destdir {}'.format(self.testlayer_path, testcheckoutdir))
+ # May not necessarily be named 'poky'
+ pokydir = os.listdir(testcheckoutdir)[0]
+ testcheckoutfile = os.path.join(testcheckoutdir, pokydir, "oe-init-build-env")
+ self.assertTrue(os.path.exists(testcheckoutfile), "File {} not found in test layer checkout".format(testcheckoutfile))
--
2.30.2
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [OE-core] [PATCH 6/6] selftest/bblayers: add a test for creating a layer setup and using it to restore the layers
2022-07-29 14:10 ` [PATCH 6/6] selftest/bblayers: add a test for creating a layer setup and using it to restore the layers Alexander Kanavin
@ 2022-08-16 20:29 ` Richard Purdie
2022-08-16 20:41 ` Alexander Kanavin
0 siblings, 1 reply; 15+ messages in thread
From: Richard Purdie @ 2022-08-16 20:29 UTC (permalink / raw)
To: Alexander Kanavin, openembedded-core; +Cc: Alexander Kanavin
On Fri, 2022-07-29 at 16:10 +0200, Alexander Kanavin wrote:
> This does a basic run-through of the bitbake-layers plugin, and the resulting json layer config
> and the layer setup script that uses it. Only poky is actually fetched by the script.
>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
> meta/lib/oeqa/selftest/cases/bblayers.py | 12 ++++++++++++
> 1 file changed, 12 insertions(+)
>
> diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py b/meta/lib/oeqa/selftest/cases/bblayers.py
> index 4f4952f3eb..1f5c576356 100644
> --- a/meta/lib/oeqa/selftest/cases/bblayers.py
> +++ b/meta/lib/oeqa/selftest/cases/bblayers.py
> @@ -140,3 +140,15 @@ class BitbakeLayers(OESelftestTestCase):
> def test_validate_examplelayersjson(self):
> json = os.path.join(get_bb_var('COREBASE'), "meta/files/layers.example.json")
> self.validate_layersjson(json)
> +
> + def test_bitbakelayers_setup(self):
> + result = runCmd('bitbake-layers create-layers-setup {}'.format(self.testlayer_path))
> + jsonfile = os.path.join(self.testlayer_path, "setup-layers.json")
> + self.validate_layersjson(jsonfile)
> +
> + testcheckoutdir = os.path.join(self.builddir, 'test-layer-checkout')
> + result = runCmd('{}/setup-layers --destdir {}'.format(self.testlayer_path, testcheckoutdir))
> + # May not necessarily be named 'poky'
> + pokydir = os.listdir(testcheckoutdir)[0]
> + testcheckoutfile = os.path.join(testcheckoutdir, pokydir, "oe-init-build-env")
> + self.assertTrue(os.path.exists(testcheckoutfile), "File {} not found in test layer checkout".format(testcheckoutfile))
https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/4001/steps/14/logs/stdio
(and other similar selftest failures)
I suspect this was due to master-next being force pushed whilst the
build was running.
Cheers,
Richard
^ permalink raw reply [flat|nested] 15+ messages in thread
* RE: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from
2022-07-29 14:10 [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Alexander Kanavin
` (4 preceding siblings ...)
2022-07-29 14:10 ` [PATCH 6/6] selftest/bblayers: add a test for creating a layer setup and using it to restore the layers Alexander Kanavin
@ 2022-07-29 19:55 ` Peter Kjellerstedt
2022-07-29 20:11 ` Alexander Kanavin
5 siblings, 1 reply; 15+ messages in thread
From: Peter Kjellerstedt @ 2022-07-29 19:55 UTC (permalink / raw)
To: Alexander Kanavin, openembedded-core@lists.openembedded.org
Cc: Alexander Kanavin
> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <openembedded-
> core@lists.openembedded.org> On Behalf Of Alexander Kanavin
> Sent: den 29 juli 2022 16:11
> To: openembedded-core@lists.openembedded.org
> Cc: Alexander Kanavin <alex@linutronix.de>
> Subject: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known
> where configurations come from
>
> "with some default values" isn't clear; if the user wants to change or
> inspect the default values and their history, we should help them find
> where they are. This becomes especially important when using template
> configs other than poky's.
>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
> scripts/oe-setup-builddir | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/scripts/oe-setup-builddir b/scripts/oe-setup-builddir
> index 54048e62ec..b4fec4d120 100755
> --- a/scripts/oe-setup-builddir
> +++ b/scripts/oe-setup-builddir
> @@ -74,9 +74,9 @@ fi
> if [ ! -r "$BUILDDIR/conf/local.conf" ]; then
> cat <<EOM
> You had no conf/local.conf file. This configuration file has therefore been
> -created for you with some default values. You may wish to edit it to, for
> -example, select a different MACHINE (target hardware). See conf/local.conf
> -for more information as common configuration options are commented.
> +created for you from $OECORELOCALCONF
> +You may wish to edit it to, for example, select a different MACHINE (target hardware).
> +See conf/local.conf for more information as common configuration options are commented.
Please continue to limit the text to within 80 characters (as it was):
You may wish to edit it to, for example, select a different MACHINE (target
hardware). See conf/local.conf for more information as common configuration
options are commented.
or:
You may wish to edit it to, e.g., select a different MACHINE (target hardware).
See conf/local.conf for more information as common configuration options are
commented.
>
> EOM
> cp -f "$OECORELOCALCONF" "$BUILDDIR/conf/local.conf"
> @@ -89,8 +89,9 @@ fi
> if [ ! -r "$BUILDDIR/conf/bblayers.conf" ]; then
> cat <<EOM
> You had no conf/bblayers.conf file. This configuration file has therefore been
> -created for you with some default values. To add additional metadata layers
> -into your configuration please add entries to conf/bblayers.conf.
> +created for you from $OECORELAYERCONF
> +To add additional metadata layers into your configuration please add entries
> +to conf/bblayers.conf.
>
> EOM
>
> --
> 2.30.2
//Peter
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from
2022-07-29 19:55 ` [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from Peter Kjellerstedt
@ 2022-07-29 20:11 ` Alexander Kanavin
2022-07-30 9:54 ` Peter Kjellerstedt
2022-07-30 18:15 ` Chuck Wolber
0 siblings, 2 replies; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-29 20:11 UTC (permalink / raw)
To: Peter Kjellerstedt
Cc: openembedded-core@lists.openembedded.org, Alexander Kanavin
On Fri, 29 Jul 2022 at 21:55, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
> Please continue to limit the text to within 80 characters (as it was):
>
> You may wish to edit it to, for example, select a different MACHINE (target
> hardware). See conf/local.conf for more information as common configuration
> options are commented.
>
> or:
>
> You may wish to edit it to, e.g., select a different MACHINE (target hardware).
> See conf/local.conf for more information as common configuration options are
> commented.
I can do that, but I'm curious about this 80 character limit. Who
cares, in 2022, and why?
Alex
^ permalink raw reply [flat|nested] 15+ messages in thread
* RE: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from
2022-07-29 20:11 ` Alexander Kanavin
@ 2022-07-30 9:54 ` Peter Kjellerstedt
2022-07-30 17:31 ` Alexander Kanavin
2022-07-30 18:15 ` Chuck Wolber
1 sibling, 1 reply; 15+ messages in thread
From: Peter Kjellerstedt @ 2022-07-30 9:54 UTC (permalink / raw)
To: Alexander Kanavin
Cc: openembedded-core@lists.openembedded.org, Alexander Kanavin
> -----Original Message-----
> From: Alexander Kanavin <alex.kanavin@gmail.com>
> Sent: den 29 juli 2022 22:12
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Cc: openembedded-core@lists.openembedded.org; Alexander Kanavin
> <alex@linutronix.de>
> Subject: Re: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it
> known where configurations come from
>
> On Fri, 29 Jul 2022 at 21:55, Peter Kjellerstedt <peter.kjellerstedt@axis.com> wrote:
> > Please continue to limit the text to within 80 characters (as it was):
> >
> > You may wish to edit it to, for example, select a different MACHINE (target
> > hardware). See conf/local.conf for more information as common configuration
> > options are commented.
> >
> > or:
> >
> > You may wish to edit it to, e.g., select a different MACHINE (target hardware).
> > See conf/local.conf for more information as common configuration options are
> > commented.
>
> I can do that, but I'm curious about this 80 character limit. Who
> cares, in 2022, and why?
>
> Alex
Well, I care since all my windows happen to be 80 characters wide (and I have
a lot of them on my 4K monitor). And since I care and most don't, keeping the
80 character width should hopefully not be too much of a problem. ;)
//Peter
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from
2022-07-30 9:54 ` Peter Kjellerstedt
@ 2022-07-30 17:31 ` Alexander Kanavin
0 siblings, 0 replies; 15+ messages in thread
From: Alexander Kanavin @ 2022-07-30 17:31 UTC (permalink / raw)
To: Peter Kjellerstedt
Cc: openembedded-core@lists.openembedded.org, Alexander Kanavin
On Sat, 30 Jul 2022 at 11:55, Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
> Well, I care since all my windows happen to be 80 characters wide (and I have
> a lot of them on my 4K monitor). And since I care and most don't, keeping the
> 80 character width should hopefully not be too much of a problem. ;)
On the other hand, I have a single fullscreen terminal window (with
tabs) which offers 212 char x 53 lines of text. So that I don't need
to be irritated by other people's indentation and line breaking
preferences (or lack of care for them), and everything just fits and
is perfectly readable, always :)
Alex
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [OE-core] [PATCH 1/6] scripts/oe-setup-builddir: make it known where configurations come from
2022-07-29 20:11 ` Alexander Kanavin
2022-07-30 9:54 ` Peter Kjellerstedt
@ 2022-07-30 18:15 ` Chuck Wolber
1 sibling, 0 replies; 15+ messages in thread
From: Chuck Wolber @ 2022-07-30 18:15 UTC (permalink / raw)
To: Alexander Kanavin
Cc: Alexander Kanavin, Peter Kjellerstedt,
openembedded-core@lists.openembedded.org
[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]
On Fri, Jul 29, 2022 at 13:12 Alexander Kanavin <alex.kanavin@gmail.com>
wrote:
I can do that, but I'm curious about this 80 character limit. Who
> cares, in 2022, and why?
I care :)
Why? It is an expectation built into a lot of tools for one thing.
The git commit log is one notable example. Wrapping commit messages at 72
characters and keeping code width to (generally) 80 characters has
non-trivial benefits that are (hopefully) reasonably obvious to anyone who
has had a lot of windows up at one time.
We also have to have *some* sort of limit and it *should* be consistently
applied. The human ability to mentally “carriage return” drops off as lines
get longer and minds get fatigued.
Finally, looking back at my own learning curve and having mentored new
developers, line length guidelines force developers to write code that is
more readable in the long run.
..Ch:W..
> --
*"Perfection must be reached by degrees; she requires the slow hand of
time." - Voltaire*
[-- Attachment #2: Type: text/html, Size: 2413 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread