* [PATCH 1/4] classes/populate_sdk_ext.bbclass: handle if local.conf doesn't end with a newline
2016-01-12 18:47 [PATCH 0/4] Extensible SDK fixes Paul Eggleton
@ 2016-01-12 18:47 ` Paul Eggleton
2016-01-12 18:47 ` [PATCH 2/4] classes/populate_sdk_ext: support auto.conf Paul Eggleton
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Paul Eggleton @ 2016-01-12 18:47 UTC (permalink / raw)
To: openembedded-core
If there is no newline at the end of local.conf, appending
INHERIT = "uninative" won't work, it will corrupt the line and the
installed eSDK will build things, making the "Preparing build system..."
step take an age.
Fixes [YOCTO #8897].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/populate_sdk_ext.bbclass | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index 87fb767..984f538 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -142,6 +142,8 @@ python copy_buildsystem () {
for line in newlines:
if line.strip() and not line.startswith('#'):
f.write(line)
+ # Write a newline just in case there's none at the end of the original
+ f.write('\n')
f.write('INHERIT += "%s"\n\n' % 'uninative')
f.write('CONF_VERSION = "%s"\n\n' % d.getVar('CONF_VERSION', False))
--
2.5.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/4] classes/populate_sdk_ext: support auto.conf
2016-01-12 18:47 [PATCH 0/4] Extensible SDK fixes Paul Eggleton
2016-01-12 18:47 ` [PATCH 1/4] classes/populate_sdk_ext.bbclass: handle if local.conf doesn't end with a newline Paul Eggleton
@ 2016-01-12 18:47 ` Paul Eggleton
2016-01-12 18:47 ` [PATCH 3/4] classes/buildhistory: save auto.conf and bblayers.conf for extensible SDK Paul Eggleton
2016-01-12 18:47 ` [PATCH 4/4] classes/populate_sdk_ext: check that extensible SDK prepared correctly Paul Eggleton
3 siblings, 0 replies; 6+ messages in thread
From: Paul Eggleton @ 2016-01-12 18:47 UTC (permalink / raw)
To: openembedded-core
If auto.conf exists in the user's configuration we need to also run it
through the same filter and write the result into the ext SDK, or we
risk missing configuration applied on an autobuilder.
Fixes [YOCTO #8904].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/populate_sdk_ext.bbclass | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index 984f538..69e13ab 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -169,6 +169,19 @@ python copy_buildsystem () {
f.write('require conf/locked-sigs.inc\n')
+ if os.path.exists(builddir + '/conf/auto.conf'):
+ with open(builddir + '/conf/auto.conf', 'r') as f:
+ oldlines = f.readlines()
+ (updated, newlines) = bb.utils.edit_metadata(oldlines, varlist, handle_var)
+ with open(baseoutpath + '/conf/auto.conf', 'w') as f:
+ f.write('# WARNING: this configuration has been automatically generated and in\n')
+ f.write('# most cases should not be edited. If you need more flexibility than\n')
+ f.write('# this configuration provides, it is strongly suggested that you set\n')
+ f.write('# up a proper instance of the full build system and use that instead.\n\n')
+ for line in newlines:
+ if line.strip() and not line.startswith('#'):
+ f.write(line)
+
sigfile = d.getVar('WORKDIR', True) + '/locked-sigs.inc'
oe.copy_buildsystem.generate_locked_sigs(sigfile, d)
--
2.5.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/4] classes/buildhistory: save auto.conf and bblayers.conf for extensible SDK
2016-01-12 18:47 [PATCH 0/4] Extensible SDK fixes Paul Eggleton
2016-01-12 18:47 ` [PATCH 1/4] classes/populate_sdk_ext.bbclass: handle if local.conf doesn't end with a newline Paul Eggleton
2016-01-12 18:47 ` [PATCH 2/4] classes/populate_sdk_ext: support auto.conf Paul Eggleton
@ 2016-01-12 18:47 ` Paul Eggleton
2016-01-12 18:47 ` [PATCH 4/4] classes/populate_sdk_ext: check that extensible SDK prepared correctly Paul Eggleton
3 siblings, 0 replies; 6+ messages in thread
From: Paul Eggleton @ 2016-01-12 18:47 UTC (permalink / raw)
To: openembedded-core
These form part of the configuration for the extensible SDK, we should
really be recording what goes into them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/buildhistory.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/buildhistory.bbclass b/meta/classes/buildhistory.bbclass
index 4153e58..2c144ab 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -13,7 +13,7 @@ BUILDHISTORY_DIR_IMAGE = "${BUILDHISTORY_DIR}/images/${MACHINE_ARCH}/${TCLIBC}/$
BUILDHISTORY_DIR_PACKAGE = "${BUILDHISTORY_DIR}/packages/${MULTIMACH_TARGET_SYS}/${PN}"
BUILDHISTORY_DIR_SDK = "${BUILDHISTORY_DIR}/sdk/${SDK_NAME}${SDK_EXT}/${IMAGE_BASENAME}"
BUILDHISTORY_IMAGE_FILES ?= "/etc/passwd /etc/group"
-BUILDHISTORY_SDK_FILES ?= "conf/local.conf conf/locked-sigs.inc conf/devtool.conf"
+BUILDHISTORY_SDK_FILES ?= "conf/local.conf conf/bblayers.conf conf/auto.conf conf/locked-sigs.inc conf/devtool.conf"
BUILDHISTORY_COMMIT ?= "0"
BUILDHISTORY_COMMIT_AUTHOR ?= "buildhistory <buildhistory@${DISTRO}>"
BUILDHISTORY_PUSH_REPO ?= ""
--
2.5.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 4/4] classes/populate_sdk_ext: check that extensible SDK prepared correctly
2016-01-12 18:47 [PATCH 0/4] Extensible SDK fixes Paul Eggleton
` (2 preceding siblings ...)
2016-01-12 18:47 ` [PATCH 3/4] classes/buildhistory: save auto.conf and bblayers.conf for extensible SDK Paul Eggleton
@ 2016-01-12 18:47 ` Paul Eggleton
3 siblings, 0 replies; 6+ messages in thread
From: Paul Eggleton @ 2016-01-12 18:47 UTC (permalink / raw)
To: openembedded-core
After the change to use --setscene-only when running bitbake to prepare
the SDK at the end of installation, add a check that the SDK got
prepared correctly by doing a dry-run and looking at the output for any
real tasks that we don't expect. In order to make this easier, the
preparation shell script was rewritten in python.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
meta/classes/populate_sdk_ext.bbclass | 5 +-
meta/files/ext-sdk-prepare.py | 92 +++++++++++++++++++++++++++++++++++
meta/files/ext-sdk-prepare.sh | 8 ---
3 files changed, 94 insertions(+), 11 deletions(-)
create mode 100644 meta/files/ext-sdk-prepare.py
delete mode 100644 meta/files/ext-sdk-prepare.sh
diff --git a/meta/classes/populate_sdk_ext.bbclass b/meta/classes/populate_sdk_ext.bbclass
index 69e13ab..4d8d2a6 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -224,7 +224,7 @@ install_tools() {
install ${SDK_DEPLOY}/${BUILD_ARCH}-nativesdk-libc.tar.bz2 ${SDK_OUTPUT}/${SDKPATH}
- install -m 0755 ${COREBASE}/meta/files/ext-sdk-prepare.sh ${SDK_OUTPUT}/${SDKPATH}
+ install -m 0644 ${COREBASE}/meta/files/ext-sdk-prepare.py ${SDK_OUTPUT}/${SDKPATH}
}
# Since bitbake won't run as root it doesn't make sense to try and install
@@ -270,9 +270,8 @@ sdk_ext_postinst() {
# current working directory when first ran, nor will it set $1 when
# sourcing a script. That is why this has to look so ugly.
LOGFILE="$target_sdk_dir/preparing_build_system.log"
- sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && $target_sdk_dir/ext-sdk-prepare.sh $target_sdk_dir '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "ERROR: SDK preparation failed: see $LOGFILE"; echo "printf 'ERROR: this SDK was not fully installed and needs reinstalling\n'" >> $env_setup_script ; exit 1 ; }
+ sh -c ". buildtools/environment-setup* > $LOGFILE && cd $target_sdk_dir/`dirname ${oe_init_build_env_path}` && set $target_sdk_dir && . $target_sdk_dir/${oe_init_build_env_path} $target_sdk_dir >> $LOGFILE && python $target_sdk_dir/ext-sdk-prepare.py '${SDK_TARGETS}' >> $LOGFILE 2>&1" || { echo "ERROR: SDK preparation failed: see $LOGFILE"; echo "printf 'ERROR: this SDK was not fully installed and needs reinstalling\n'" >> $env_setup_script ; exit 1 ; }
fi
- rm -f $target_sdk_dir/ext-sdk-prepare.sh
echo done
}
diff --git a/meta/files/ext-sdk-prepare.py b/meta/files/ext-sdk-prepare.py
new file mode 100644
index 0000000..143e0fe
--- /dev/null
+++ b/meta/files/ext-sdk-prepare.py
@@ -0,0 +1,92 @@
+#!/usr/bin/env python
+
+# Prepare the build system within the extensible SDK
+
+import sys
+import os
+import subprocess
+
+def exec_watch(cmd, **options):
+ """Run program with stdout shown on sys.stdout"""
+ if isinstance(cmd, basestring) and not "shell" in options:
+ options["shell"] = True
+
+ process = subprocess.Popen(
+ cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **options
+ )
+
+ buf = ''
+ while True:
+ out = process.stdout.read(1)
+ if out:
+ sys.stdout.write(out)
+ sys.stdout.flush()
+ buf += out
+ elif out == '' and process.poll() != None:
+ break
+
+ return process.returncode, buf
+
+
+def main():
+ if len(sys.argv) < 2:
+ print('Please specify target to prepare with')
+ return 1
+
+ sdk_targets = ' '.join(sys.argv[1:]).split()
+ print('Preparing SDK for %s...' % ', '.join(sdk_targets))
+
+ ret, out = exec_watch('bitbake %s --setscene-only' % ' '.join(sdk_targets))
+ if ret:
+ return ret
+
+ targetlist = []
+ for target in sdk_targets:
+ if ':' in target:
+ target = target.split(':')[0]
+ if not target in targetlist:
+ targetlist.append(target)
+
+ recipes = []
+ for target in targetlist:
+ try:
+ out = subprocess.check_output(('bitbake -e %s' % target).split(), stderr=subprocess.STDOUT)
+ for line in out.splitlines():
+ if line.startswith('FILE='):
+ splitval = line.rstrip().split('=')
+ if len(splitval) > 1:
+ recipes.append(splitval[1].strip('"'))
+ break
+ except subprocess.CalledProcessError as e:
+ print('ERROR: Failed to get recipe for target %s:\n%s' % (target, e.output))
+ return 1
+
+ try:
+ out = subprocess.check_output('bitbake %s -n' % ' '.join(sdk_targets), stderr=subprocess.STDOUT, shell=True)
+ unexpected = []
+ for line in out.splitlines():
+ if 'Running task' in line:
+ for recipe in recipes:
+ if recipe in line:
+ break
+ else:
+ line = line.split('Running', 1)[-1]
+ unexpected.append(line.rstrip())
+ except subprocess.CalledProcessError as e:
+ print('ERROR: Failed to execute dry-run:\n%s' % e.output)
+ return 1
+
+ if unexpected:
+ print('ERROR: Unexpected tasks left over to be executed:')
+ for line in unexpected:
+ print(' ' + line)
+ return 1
+
+if __name__ == "__main__":
+ try:
+ ret = main()
+ except Exception:
+ ret = 1
+ import traceback
+ traceback.print_exc(5)
+ sys.exit(ret)
diff --git a/meta/files/ext-sdk-prepare.sh b/meta/files/ext-sdk-prepare.sh
deleted file mode 100644
index b3f5d93..0000000
--- a/meta/files/ext-sdk-prepare.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/sh
-
-# Prepare the build system within the extensible SDK
-
-target_sdk_dir="$1"
-sdk_targets="$2"
-
-bitbake $sdk_targets --setscene-only || exit 1
--
2.5.0
^ permalink raw reply related [flat|nested] 6+ messages in thread