[PATCH 1/2] oeqa/selftest: Add test for conflicting sysroot provider

[PATCH 1/2] oeqa/selftest: Add test for conflicting sysroot provider

[Richard Purdie]

sysroot-test depends on virtual/sysroot-test which we build for one machine,
switch machine, switch provider of virtual/sysroot-test and check that the
sysroot is correctly cleaned up. The files in the two providers overlap
so can cause errors if the sysroot code doesn't function correctly.

Yes, sysroot-test should be machine specific really to avoid this, however
the sysroot cleanup should also work.

This adds a test for bug:

[YOCTO #13702]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 .../sysroot-test/sysroot-test-arch1_1.0.bb    | 12 ++++++
 .../sysroot-test/sysroot-test-arch2_1.0.bb    | 12 ++++++
 .../sysroot-test/sysroot-test_1.0.bb          |  4 ++
 meta/lib/oeqa/selftest/cases/sysroot.py       | 37 +++++++++++++++++++
 4 files changed, 65 insertions(+)
 create mode 100644 meta-selftest/recipes-test/sysroot-test/sysroot-test-arch1_1.0.bb
 create mode 100644 meta-selftest/recipes-test/sysroot-test/sysroot-test-arch2_1.0.bb
 create mode 100644 meta-selftest/recipes-test/sysroot-test/sysroot-test_1.0.bb
 create mode 100644 meta/lib/oeqa/selftest/cases/sysroot.py

diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-test-arch1_1.0.bb b/meta-selftest/recipes-test/sysroot-test/sysroot-test-arch1_1.0.bb
new file mode 100644
index 00000000000..0b4ed2ef052
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-test-arch1_1.0.bb
@@ -0,0 +1,12 @@
+LICENSE = "CLOSED"
+
+PROVIDES = "virtual/sysroot-test"
+INHIBIT_DEFAULT_DEPS = "1"
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+TESTSTRING ?= "1"
+
+do_install() {
+	install -d ${D}${includedir}
+	echo "# test ${TESTSTRING}" > ${D}${includedir}/sysroot-test.h
+}
\ No newline at end of file
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-test-arch2_1.0.bb b/meta-selftest/recipes-test/sysroot-test/sysroot-test-arch2_1.0.bb
new file mode 100644
index 00000000000..f8b50acda2c
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-test-arch2_1.0.bb
@@ -0,0 +1,12 @@
+LICENSE = "CLOSED"
+
+PROVIDES = "virtual/sysroot-test"
+INHIBIT_DEFAULT_DEPS = "1"
+PACKAGE_ARCH = "${MACHINE_ARCH}"
+
+TESTSTRING ?= "2"
+
+do_install() {
+	install -d ${D}${includedir}
+	echo "# test ${TESTSTRING}" > ${D}${includedir}/sysroot-test.h
+}
\ No newline at end of file
diff --git a/meta-selftest/recipes-test/sysroot-test/sysroot-test_1.0.bb b/meta-selftest/recipes-test/sysroot-test/sysroot-test_1.0.bb
new file mode 100644
index 00000000000..bec0eecb980
--- /dev/null
+++ b/meta-selftest/recipes-test/sysroot-test/sysroot-test_1.0.bb
@@ -0,0 +1,4 @@
+SUMMARY = "Virtual provider sysroot test"
+LICENSE = "CLOSED"
+INHIBIT_DEFAULT_DEPS = "1"
+DEPENDS = "virtual/sysroot-test"
diff --git a/meta/lib/oeqa/selftest/cases/sysroot.py b/meta/lib/oeqa/selftest/cases/sysroot.py
new file mode 100644
index 00000000000..6e34927c909
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/sysroot.py
@@ -0,0 +1,37 @@
+#
+# SPDX-License-Identifier: MIT
+#
+
+import uuid
+
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import  bitbake
+
+class SysrootTests(OESelftestTestCase):
+    def test_sysroot_cleanup(self):
+        """
+        Build sysroot test which depends on virtual/sysroot-test for one machine,
+        switch machine, switch provider of virtual/sysroot-test and check that the
+        sysroot is correctly cleaned up. The files in the two providers overlap
+        so can cause errors if the sysroot code doesn't function correctly.
+        Yes, sysroot-test should be machine specific really to avoid this, however
+        the sysroot cleanup should also work [YOCTO #13702]. 
+        """
+
+        uuid1 = uuid.uuid4()
+        uuid2 = uuid.uuid4()
+
+        self.write_config("""
+PREFERRED_PROVIDER_virtual/sysroot-test = "sysroot-test-arch1"
+MACHINE = "qemux86"
+TESTSTRING_pn-sysroot-test-arch1 = "%s"
+TESTSTRING_pn-sysroot-test-arch2 = "%s"
+""" % (uuid1, uuid2))
+        bitbake("sysroot-test")
+        self.write_config("""
+PREFERRED_PROVIDER_virtual/sysroot-test = "sysroot-test-arch2"
+MACHINE = "qemux86copy"
+TESTSTRING_pn-sysroot-test-arch1 = "%s"
+TESTSTRING_pn-sysroot-test-arch2 = "%s"
+""" % (uuid1, uuid2))
+        bitbake("sysroot-test")
-- 
2.25.1

[PATCH 2/2] staging: Fix overlapping file failures

[Richard Purdie]

If there are different providers of a file and they are swiched when the
recipe isn't machine specific, we can get tracebacks due to the overlapping
files. The issue is that the previous provider isn't uninstalled since
the system can't tell whether some later task needs them.

By tracking which tasks we depend upon, the code can now choose to
uninstall more things since a later task can reinstall if/as needed.

The code here was to protect against code with two different tasks
running in parallel which is still protected agaisnt.

[YOCTO #13702]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/staging.bbclass | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass
index 5dab42745f7..5b04f88b2d8 100644
--- a/meta/classes/staging.bbclass
+++ b/meta/classes/staging.bbclass
@@ -277,11 +277,13 @@ python extend_recipe_sysroot() {
 
     start = None
     configuredeps = []
+    owntaskdeps = []
     for dep in taskdepdata:
         data = taskdepdata[dep]
         if data[1] == mytaskname and data[0] == pn:
             start = dep
-            break
+        elif data[0] == pn:
+            owntaskdeps.append(data[1])
     if start is None:
         bb.fatal("Couldn't find ourself in BB_TASKDEPDATA?")
 
@@ -427,7 +429,7 @@ python extend_recipe_sysroot() {
                         # Was likely already uninstalled
                         continue
                     potential.append(l)
-        # We need to ensure not other task needs this dependency. We hold the sysroot
+        # We need to ensure no other task needs this dependency. We hold the sysroot
         # lock so we ca search the indexes to check
         if potential:
             for i in glob.glob(depdir + "/index.*"):
@@ -435,6 +437,11 @@ python extend_recipe_sysroot() {
                     continue
                 with open(i, "r") as f:
                     for l in f:
+                        if l.startswith("TaskDeps:"):
+                            prevtasks = l.split()[1:]
+                            if mytaskname in prevtasks:
+                                # We're a dependency of this task so we can clear items out the sysroot
+                                break
                         l = l.strip()
                         if l in potential:
                             potential.remove(l)
@@ -588,6 +595,7 @@ python extend_recipe_sysroot() {
         os.symlink(manifests[dep], depdir + "/" + c + ".complete")
 
     with open(taskindex, "w") as f:
+        f.write("TaskDeps: " + " ".join(owntaskdeps) + "\n")
         for l in sorted(installed):
             f.write(l + "\n")
 
-- 
2.25.1

✗ patchtest: failure for "oeqa/selftest: Add test for co..." and 1 more

[Patchwork]

== Series Details ==

Series: "oeqa/selftest: Add test for co..." and 1 more
Revision: 1
URL   : https://patchwork.openembedded.org/series/23824/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Errors in your Python code were encountered [test_pylint] 
  Suggested fix    Correct the lines introduced by your patch
  Output           Please, fix the listed issues:
                   meta/lib/oeqa/selftest/cases/sysroot.py does not exist



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe