[PATCH 1/5] multilib_global: Fix KERNEL_VERSION expansion problems

[PATCH 1/5] multilib_global: Fix KERNEL_VERSION expansion problems

[Richard Purdie]

KERNEL_VERSION gets expanded at runtime to contain the real kernel
version. There is code to ensure the signatures are determinisic but
the multilib expansion code breaks this.

Exclude the variable from the datastore used for expansion to avoid this.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/multilib_global.bbclass | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/meta/classes/multilib_global.bbclass b/meta/classes/multilib_global.bbclass
index 11ac5b04579..98f65c8aae2 100644
--- a/meta/classes/multilib_global.bbclass
+++ b/meta/classes/multilib_global.bbclass
@@ -172,21 +172,27 @@ python multilib_virtclass_handler_global () {
     if bb.data.inherits_class('kernel', e.data) or \
             bb.data.inherits_class('module-base', e.data) or \
             d.getVar('BPN') in non_ml_recipes:
+
+            # We need to avoid expanding KERNEL_VERSION which we can do by deleting it
+            # from a copy of the datastore
+            localdata = bb.data.createCopy(d)
+            localdata.delVar("KERNEL_VERSION")
+
             variants = (e.data.getVar("MULTILIB_VARIANTS") or "").split()
 
             import oe.classextend
             clsextends = []
             for variant in variants:
-                clsextends.append(oe.classextend.ClassExtender(variant, e.data))
+                clsextends.append(oe.classextend.ClassExtender(variant, localdata))
 
             # Process PROVIDES
-            origprovs = provs = e.data.getVar("PROVIDES") or ""
+            origprovs = provs = localdata.getVar("PROVIDES") or ""
             for clsextend in clsextends:
                 provs = provs + " " + clsextend.map_variable("PROVIDES", setvar=False)
             e.data.setVar("PROVIDES", provs)
 
             # Process RPROVIDES
-            origrprovs = rprovs = e.data.getVar("RPROVIDES") or ""
+            origrprovs = rprovs = localdata.getVar("RPROVIDES") or ""
             for clsextend in clsextends:
                 rprovs = rprovs + " " + clsextend.map_variable("RPROVIDES", setvar=False)
             if rprovs.strip():
@@ -194,7 +200,7 @@ python multilib_virtclass_handler_global () {
 
             # Process RPROVIDES_${PN}...
             for pkg in (e.data.getVar("PACKAGES") or "").split():
-                origrprovs = rprovs = e.data.getVar("RPROVIDES_%s" % pkg) or ""
+                origrprovs = rprovs = localdata.getVar("RPROVIDES_%s" % pkg) or ""
                 for clsextend in clsextends:
                     rprovs = rprovs + " " + clsextend.map_variable("RPROVIDES_%s" % pkg, setvar=False)
                     rprovs = rprovs + " " + clsextend.extname + "-" + pkg
-- 
2.20.1

[PATCH 2/5] oeqa/runtime/syslog: Improve test debug messages

[Richard Purdie]

Its useful to test whether the restart command returned an error code and
exit early from the test if so.

Also add different messages to tell if the syslog processes didn't
die or didn't restart.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oeqa/runtime/cases/oe_syslog.py | 32 +++++++++++++++---------
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/oe_syslog.py b/meta/lib/oeqa/runtime/cases/oe_syslog.py
index 0f79e5a0f38..f987dccfb1b 100644
--- a/meta/lib/oeqa/runtime/cases/oe_syslog.py
+++ b/meta/lib/oeqa/runtime/cases/oe_syslog.py
@@ -43,31 +43,38 @@ class SyslogTestConfig(OERuntimeTestCase):
     def restart_sanity(self, names, restart_cmd):
         status, original_pids = self.verify_running(names)
         if status:
-            return 1
+            return False
 
         status, output = self.target.run(restart_cmd)
 
+        msg = ('Could not restart %s service. Status and output: %s and %s' % (names, status, output))
+        self.assertEqual(status, 0, msg)
+
         # Always check for an error, most likely a race between shutting down and starting up
         timeout = time.time() + 30
 
+        restarted = False
+        status = ""
         while time.time() < timeout:
             # Verify the previous ones are no longer running
             status = self.verif_not_running(original_pids)
             if status:
+                status = "Original syslog processes still running"
                 continue
 
             status, pids = self.verify_running(names)
             if status:
+                status = "New syslog processes not running"
                 continue
 
             # Everything is fine now, so exit to continue the test
-            status = 0
+            restarted = True
             break
 
-        msg = ('Could not restart %s service. Status and output: %s and %s'
-               %(names, status, output))
-        self.assertEqual(status, 0, msg)
+        msg = ('%s didn\'t appear to restart: %s' % (names, status))
+        self.assertTrue(restarted, msg)
 
+        return True
 
     @OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
     def test_syslog_logger(self):
@@ -88,13 +95,14 @@ class SyslogTestConfig(OERuntimeTestCase):
 
     @OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
     def test_syslog_restart(self):
-        status = self.restart_sanity(['systemd-journald'], 'systemctl restart syslog.service')
-        if status:
-            status = self.restart_sanity(['rsyslogd'], '/etc/init.d/rsyslog restart')
-            if status:
-                status = self.restart_sanity(['syslogd', 'klogd'], '/etc/init.d/syslog restart')
-            else:
-                self.logger.info("No syslog found to restart, ignoring")
+        if self.restart_sanity(['systemd-journald'], 'systemctl restart syslog.service'):
+            pass
+        elif self.restart_sanity(['rsyslogd'], '/etc/init.d/rsyslog restart'):
+            pass
+        elif self.restart_sanity(['syslogd', 'klogd'], '/etc/init.d/syslog restart'):
+            pass
+        else:
+            self.logger.info("No syslog found to restart, ignoring")
 
 
     @OETestDepends(['oe_syslog.SyslogTestConfig.test_syslog_logger'])
-- 
2.20.1

[PATCH 3/5] busybox: Improve syslog restart handling

[Richard Purdie]

We're seeing races on the autobuilder where syslogd fails to shut down
fast enough to be restarted leading to failures.

Add some checks to ensure when restarting that processes exit before
being restarted.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/recipes-core/busybox/files/syslog | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/busybox/files/syslog b/meta/recipes-core/busybox/files/syslog
index 89c4d12e9cd..49033c1755a 100644
--- a/meta/recipes-core/busybox/files/syslog
+++ b/meta/recipes-core/busybox/files/syslog
@@ -51,6 +51,22 @@ else
 	SYSLOG_ARGS="-C"
 fi
 
+waitpid ()
+{
+  pid=$1
+  # Give pid a chance to exit before we restart with a 5s timeout in 1s intervals
+  if [ -z "$pid" ]; then
+    return
+  fi
+  timeout=5;
+  while [ $timeout -gt 0 ]
+  do
+    timeout=$(( $timeout-1 ))
+    kill -0 $pid 2> /dev/null || break
+    sleep 1
+  done
+}
+
 case "$1" in
   start)
 	echo -n "Starting syslogd/klogd: "
@@ -65,7 +81,11 @@ case "$1" in
 	echo "done"
 	;;
   restart)
-  	$0 stop
+	pid1=`pidof syslogd`
+	pid2=`pidof syslogd`
+	$0 stop
+	waitpid $pid1
+	waitpid $pid2
 	$0 start
 	;;
   *)
-- 
2.20.1

[PATCH 4/5] oeqa/runtime/oesyslog: systemd syslog restart doesn't change pid

[Richard Purdie]

The systemd-journald process doesn't restart/change the way syslog
does, don't test/error in this case.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oeqa/runtime/cases/oe_syslog.py | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/runtime/cases/oe_syslog.py b/meta/lib/oeqa/runtime/cases/oe_syslog.py
index f987dccfb1b..449df233d13 100644
--- a/meta/lib/oeqa/runtime/cases/oe_syslog.py
+++ b/meta/lib/oeqa/runtime/cases/oe_syslog.py
@@ -40,7 +40,7 @@ class SyslogTestConfig(OERuntimeTestCase):
         return 0, pids
 
 
-    def restart_sanity(self, names, restart_cmd):
+    def restart_sanity(self, names, restart_cmd, pidchange=True):
         status, original_pids = self.verify_running(names)
         if status:
             return False
@@ -50,6 +50,9 @@ class SyslogTestConfig(OERuntimeTestCase):
         msg = ('Could not restart %s service. Status and output: %s and %s' % (names, status, output))
         self.assertEqual(status, 0, msg)
 
+        if not pidchange:
+            return True
+
         # Always check for an error, most likely a race between shutting down and starting up
         timeout = time.time() + 30
 
@@ -95,7 +98,7 @@ class SyslogTestConfig(OERuntimeTestCase):
 
     @OETestDepends(['oe_syslog.SyslogTest.test_syslog_running'])
     def test_syslog_restart(self):
-        if self.restart_sanity(['systemd-journald'], 'systemctl restart syslog.service'):
+        if self.restart_sanity(['systemd-journald'], 'systemctl restart syslog.service', pidchange=False):
             pass
         elif self.restart_sanity(['rsyslogd'], '/etc/init.d/rsyslog restart'):
             pass
-- 
2.20.1

[PATCH 5/5] oeqa/runtime/syslog: Add delay to test to avoid failures

[Richard Purdie]

On a loaded builder we've seen the log message not make it to the log file
before the ssh command completes. Add a short delay to try and ensure
this does happen. There is unforunately no way to flush syslog in all
cases we test.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/lib/oeqa/runtime/cases/oe_syslog.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/lib/oeqa/runtime/cases/oe_syslog.py b/meta/lib/oeqa/runtime/cases/oe_syslog.py
index 449df233d13..3a8271a53a8 100644
--- a/meta/lib/oeqa/runtime/cases/oe_syslog.py
+++ b/meta/lib/oeqa/runtime/cases/oe_syslog.py
@@ -85,6 +85,9 @@ class SyslogTestConfig(OERuntimeTestCase):
         msg = "Can't log into syslog. Output: %s " % output
         self.assertEqual(status, 0, msg=msg)
 
+        # There is no way to flush the logger to disk in all cases
+        time.sleep(1)
+
         status, output = self.target.run('grep foobar /var/log/messages')
         if status != 0:
             if self.tc.td.get("VIRTUAL-RUNTIME_init_manager") == "systemd":
-- 
2.20.1

✗ patchtest: failure for "multilib_global: Fix KERNEL_VE..." and 4 more

[Patchwork]

== Series Details ==

Series: "multilib_global: Fix KERNEL_VE..." and 4 more
Revision: 1
URL   : https://patchwork.openembedded.org/series/18362/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 4bb3e8f98e)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe