[PATCH 0/4] Fixes for shell message changes (OE-Core side)

[PATCH 0/4] Fixes for shell message changes (OE-Core side)

[Paul Eggleton]

Unfortunately the shell message changes caused a few minor regressions;
this is a perfect example of a situation where what seems like a simple
and logical change can have unintended consequences that you'll miss if
you're not careful. More testing all round would probably be advisable
in future. Having said that, it has had the effect of highlighting
something that we need to be mindful of in future - don't call bbfatal()
where you expect the rest of the error log to be printed - use die() or
bbfatal_log() for that, or alternatively make your message meaningful
enough that you don't need the log.

NOTE: the fix for die() and bbfatal_log() requires the BitBake change
that implements bbfatal_log internally which has just sent to the
bitbake-devel list. Having said that, logging.bbclass now works as it
used to for older BitBake versions that didn't have the message support
at all.


The following changes since commit 6be698b7270f73f40d38713ecf13f12aec0ced61:

  dpkg: Fix for Fedora22 and new versions of tar (2015-07-13 13:46:45 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib paule/oecore-shell-logging-fixes
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/oecore-shell-logging-fixes

Paul Eggleton (4):
  classes/useradd: don't read bbnote/bbwarn/bbfatal values
  classes/base: fix die() to print the full log
  Use die() or bbfatal_log() where the log should definitely be printed
  classes/logging: allow shell message functions to work in devshell

 meta/classes/autotools.bbclass              |  4 +--
 meta/classes/base.bbclass                   |  2 +-
 meta/classes/kernel-yocto.bbclass           | 16 +++++-----
 meta/classes/logging.bbclass                | 48 +++++++++++++++++++++++++----
 meta/classes/scons.bbclass                  |  4 +--
 meta/classes/useradd.bbclass                |  6 ++--
 meta/recipes-extended/groff/groff_1.22.3.bb |  2 +-
 7 files changed, 59 insertions(+), 23 deletions(-)

-- 
2.1.0

[PATCH 1/4] classes/useradd: don't read bbnote/bbwarn/bbfatal values

[Paul Eggleton]

Since these functions now log through BitBake's message logging system
we must have standalone implementations here.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/useradd.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/useradd.bbclass b/meta/classes/useradd.bbclass
index aae038f..4577e56 100644
--- a/meta/classes/useradd.bbclass
+++ b/meta/classes/useradd.bbclass
@@ -191,9 +191,9 @@ fakeroot python populate_packages_prepend () {
         preinst = d.getVar('pkg_preinst_%s' % pkg, True) or d.getVar('pkg_preinst', True)
         if not preinst:
             preinst = '#!/bin/sh\n'
-        preinst += 'bbnote () {\n%s}\n' % d.getVar('bbnote', True)
-        preinst += 'bbwarn () {\n%s}\n' % d.getVar('bbwarn', True)
-        preinst += 'bbfatal () {\n%s}\n' % d.getVar('bbfatal', True)
+        preinst += 'bbnote () {\n\techo "NOTE: $*"\n}\n'
+        preinst += 'bbwarn () {\n\techo "WARNING: $*"\n}\n'
+        preinst += 'bbfatal () {\n\techo "ERROR: $*"\n\texit 1\n}\n'
         preinst += 'perform_groupadd () {\n%s}\n' % d.getVar('perform_groupadd', True)
         preinst += 'perform_useradd () {\n%s}\n' % d.getVar('perform_useradd', True)
         preinst += 'perform_groupmems () {\n%s}\n' % d.getVar('perform_groupmems', True)
-- 
2.1.0

[PATCH 2/4] classes/base: fix die() to print the full log

[Paul Eggleton]

The recent change to connect through the shell logging functions caused
a regression - bb.error() and bb.fatal() cause a flag to be set
internally such that BitBake's UI will not print the full task log on
failure; unfortunately we have in a lot of places called die() or
bbfatal() within shell functions with a very terse message as a means of
exiting out, where we still want to see the full log (and we were
previously). We do still want to have fatal errors with proper messages
that don't result in the full log being printed, however we can't ignore
the typical usage of die(). Having added a mechanism to BitBake to log an
error and reset the flag, create a bbfatal_log() function that uses this
and call it from die() to restore the previous behaviour.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/base.bbclass    | 2 +-
 meta/classes/logging.bbclass | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index fe803f1..6676078 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -47,7 +47,7 @@ def lsb_distro_identifier(d):
     return oe.lsb.distro_identifier(adjust_func)
 
 die() {
-	bbfatal "$*"
+	bbfatal_log "$*"
 }
 
 oe_runmake_call() {
diff --git a/meta/classes/logging.bbclass b/meta/classes/logging.bbclass
index f19eddd..6b24839 100644
--- a/meta/classes/logging.bbclass
+++ b/meta/classes/logging.bbclass
@@ -40,6 +40,14 @@ bbfatal() {
 	exit 1
 }
 
+# Like bbfatal, except prevents the suppression of the error log by
+# bitbake's UI.
+# Output: logs console
+bbfatal_log() {
+	printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+	exit 1
+}
+
 # Print debug messages. These are appropriate for progress checkpoint
 # messages to the logs. Depending on the debug log level, they may also
 # go to the console.
-- 
2.1.0

[PATCH 3/4] Use die() or bbfatal_log() where the log should definitely be printed

[Paul Eggleton]

Change calls to bbfatal() to either die() or bbfatal_log() where we know
we want the full log to be printed by the UI (calling bberror or bbfatal
would otherwise suppress it since the change to connect these functions
through to the UI.) bbfatal() is still fine to use where there is enough
context information in the message such that the log isn't needed.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/autotools.bbclass              |  4 ++--
 meta/classes/kernel-yocto.bbclass           | 16 ++++++++--------
 meta/classes/scons.bbclass                  |  4 ++--
 meta/recipes-extended/groff/groff_1.22.3.bb |  2 +-
 4 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/meta/classes/autotools.bbclass b/meta/classes/autotools.bbclass
index 454dcb6..6f51429 100644
--- a/meta/classes/autotools.bbclass
+++ b/meta/classes/autotools.bbclass
@@ -87,7 +87,7 @@ oe_runconf () {
 		if [ "$?" != "0" ]; then
 			echo "Configure failed. The contents of all config.log files follows to aid debugging"
 			find ${S} -ignore_readdir_race -name config.log -print -exec cat {} \;
-			bbfatal "oe_runconf failed"
+			die "oe_runconf failed"
 		fi
 		set -e
 	else
@@ -287,7 +287,7 @@ autotools_do_configure() {
 			intltoolize --copy --force --automake
 		fi
 		bbnote Executing ACLOCAL=\"$ACLOCAL\" autoreconf --verbose --install --force ${EXTRA_AUTORECONF} $acpaths
-		ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || bbfatal "autoreconf execution failed."
+		ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || die "autoreconf execution failed."
 		cd $olddir
 	fi
 	if [ -e ${S}/configure ]; then
diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 6fd025e..878b554 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -120,7 +120,7 @@ do_kernel_metadata() {
 
 	createme -v -v ${createme_flags} ${ARCH} ${machine_branch}
 	if [ $? -ne 0 ]; then
-		bbfatal "Could not create ${machine_branch}"
+		bbfatal_log "Could not create ${machine_branch}"
 	fi
 
 	sccs="$sccs ${@" ".join(find_sccs(d))}"
@@ -152,7 +152,7 @@ do_kernel_metadata() {
 	updateme ${updateme_flags} -DKDESC=${KMACHINE}:${LINUX_KERNEL_TYPE} \
                          ${includes} ${addon_features} ${ARCH} ${KMACHINE} ${sccs} ${patches}
 	if [ $? -ne 0 ]; then
-		bbfatal "Could not update ${machine_branch}"
+		bbfatal_log "Could not update ${machine_branch}"
 	fi
 }
 
@@ -163,7 +163,7 @@ do_patch() {
 	patchme ${KMACHINE}
 	if [ $? -ne 0 ]; then
 		bberror "Could not apply patches for ${KMACHINE}."
-		bbfatal "Patch failures can be resolved in the linux source directory ${S})"
+		bbfatal_log "Patch failures can be resolved in the linux source directory ${S})"
 	fi
 
 	# check to see if the specified SRCREV is reachable from the final branch.
@@ -243,7 +243,7 @@ do_kernel_checkout() {
 		if [ $? -eq 1 ]; then
 			bberror "The branch '${KMETA}' is required and was not found"
 			bberror "Ensure that the SRC_URI points to a valid linux-yocto"
-			bbfatal "kernel repository"
+			bbfatal_log "kernel repository"
 		fi
 	fi
 	
@@ -283,7 +283,7 @@ do_kernel_configme() {
 	PATH=${PATH}:${S}/scripts/util
 	configme ${configmeflags} --reconfig --output ${B} ${LINUX_KERNEL_TYPE} ${KMACHINE}
 	if [ $? -ne 0 ]; then
-		bbfatal "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
+		bbfatal_log "Could not configure ${KMACHINE}-${LINUX_KERNEL_TYPE}"
 	fi
 	
 	echo "# Global settings from linux recipe" >> ${B}/.config
@@ -362,7 +362,7 @@ do_validate_branches() {
 		git cat-file -t ${machine_srcrev} > /dev/null
 		if [ $? -ne 0 ]; then
 			bberror "${machine_srcrev} is not a valid commit ID."
-			bbfatal "The kernel source tree may be out of sync"
+			bbfatal_log "The kernel source tree may be out of sync"
 		fi
 		force_srcrev=${machine_srcrev}
 	fi
@@ -377,14 +377,14 @@ do_validate_branches() {
 		git cat-file -t ${target_meta_head} > /dev/null
 		if [ $? -ne 0 ]; then
 			bberror "${target_meta_head} is not a valid commit ID"
-			bbfatal "The kernel source tree may be out of sync"
+			bbfatal_log "The kernel source tree may be out of sync"
 		fi
 		if [ "$meta_head" != "$target_meta_head" ]; then
 			bbnote "Setting branch ${KMETA} to ${target_meta_head}"
 			git branch -m ${KMETA} ${KMETA}-orig
 			git checkout -q -b ${KMETA} ${target_meta_head}
 			if [ $? -ne 0 ];then
-				bbfatal "Could not checkout ${KMETA} branch from known hash ${target_meta_head}"
+				bbfatal_log "Could not checkout ${KMETA} branch from known hash ${target_meta_head}"
 			fi
 		fi
 	fi
diff --git a/meta/classes/scons.bbclass b/meta/classes/scons.bbclass
index fc0f26b..b8de822 100644
--- a/meta/classes/scons.bbclass
+++ b/meta/classes/scons.bbclass
@@ -4,12 +4,12 @@ EXTRA_OESCONS ?= ""
 
 scons_do_compile() {
         ${STAGING_BINDIR_NATIVE}/scons ${PARALLEL_MAKE} PREFIX=${prefix} prefix=${prefix} ${EXTRA_OESCONS} || \
-        bbfatal "scons build execution failed."
+        die "scons build execution failed."
 }
 
 scons_do_install() {
         ${STAGING_BINDIR_NATIVE}/scons PREFIX=${D}${prefix} prefix=${D}${prefix} install ${EXTRA_OESCONS}|| \
-        bbfatal "scons install execution failed."
+        die "scons install execution failed."
 }
 
 EXPORT_FUNCTIONS do_compile do_install
diff --git a/meta/recipes-extended/groff/groff_1.22.3.bb b/meta/recipes-extended/groff/groff_1.22.3.bb
index e8cf524..4bffa81 100644
--- a/meta/recipes-extended/groff/groff_1.22.3.bb
+++ b/meta/recipes-extended/groff/groff_1.22.3.bb
@@ -41,7 +41,7 @@ do_configure_append() {
     # generate gnulib configure script
     olddir=`pwd`
     cd ${S}/src/libs/gnulib/
-    ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || bbfatal "autoreconf execution failed."
+    ACLOCAL="$ACLOCAL" autoreconf -Wcross --verbose --install --force ${EXTRA_AUTORECONF} $acpaths || die "autoreconf execution failed."
     cd ${olddir}
 }
 
-- 
2.1.0

[PATCH 4/4] classes/logging: allow shell message functions to work in devshell

[Paul Eggleton]

Fix a regression caused by the shell message changes - if you run a
shell task's runfile, the task isn't actually running in BitBake and
thus the message FIFO won't exist - so we should just fall back to
printing the message with echo as we did before so that the run files
are still useful.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
---
 meta/classes/logging.bbclass | 42 +++++++++++++++++++++++++++++++++++-------
 1 file changed, 35 insertions(+), 7 deletions(-)

diff --git a/meta/classes/logging.bbclass b/meta/classes/logging.bbclass
index 6b24839..06c7c31 100644
--- a/meta/classes/logging.bbclass
+++ b/meta/classes/logging.bbclass
@@ -9,34 +9,54 @@ LOGFIFO = "${T}/fifo.${@os.getpid()}"
 # tasks that should be seen on the console. Use sparingly.
 # Output: logs console
 bbplain() {
-	printf "%b\0" "bbplain $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbplain $*" > ${LOGFIFO}
+	else
+		echo "$*"
+	fi
 }
 
 # Notify the user of a noteworthy condition. 
 # Output: logs
 bbnote() {
-	printf "%b\0" "bbnote $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbnote $*" > ${LOGFIFO}
+	else
+		echo "NOTE: $*"
+	fi
 }
 
 # Print a warning to the log. Warnings are non-fatal, and do not
 # indicate a build failure.
 # Output: logs console
 bbwarn() {
-	printf "%b\0" "bbwarn $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbwarn $*" > ${LOGFIFO}
+	else
+		echo "WARNING: $*"
+	fi
 }
 
 # Print an error to the log. Errors are non-fatal in that the build can
 # continue, but they do indicate a build failure.
 # Output: logs console
 bberror() {
-	printf "%b\0" "bberror $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bberror $*" > ${LOGFIFO}
+	else
+		echo "ERROR: $*"
+	fi
 }
 
 # Print a fatal error to the log. Fatal errors indicate build failure
 # and halt the build, exiting with an error code.
 # Output: logs console
 bbfatal() {
-	printf "%b\0" "bbfatal $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbfatal $*" > ${LOGFIFO}
+	else
+		echo "ERROR: $*"
+	fi
 	exit 1
 }
 
@@ -44,7 +64,11 @@ bbfatal() {
 # bitbake's UI.
 # Output: logs console
 bbfatal_log() {
-	printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbfatal_log $*" > ${LOGFIFO}
+	else
+		echo "ERROR: $*"
+	fi
 	exit 1
 }
 
@@ -68,6 +92,10 @@ bbdebug() {
 	fi
 
 	# All debug output is printed to the logs
-	printf "%b\0" "bbdebug $DBGLVL $*" > ${LOGFIFO}
+	if [ -p ${LOGFIFO} ] ; then
+		printf "%b\0" "bbdebug $DBGLVL $*" > ${LOGFIFO}
+	else
+		echo "DEBUG: $*"
+	fi
 }
 
-- 
2.1.0