[PATCH] oeqa: reproducible: Do two clean builds

[PATCH] oeqa: reproducible: Do two clean builds

[Joshua Watt]

Perform two clean builds without sstate instead of one partial rebuild
with sstate and one clean build without. There are some classes of
reproducibility issues that this solves, and while we would like to
resolve them in the long term the direction to do so is not currently
clear.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 48 ++++++++++++--------
 1 file changed, 28 insertions(+), 20 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index c6cc0b7d0eb..b3bbb70158b 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -123,45 +123,53 @@ class ReproducibleTests(OESelftestTestCase):
     def test_reproducible_builds(self):
         capture_vars = ['DEPLOY_DIR_' + c.upper() for c in self.package_classes]
 
+        # Build native utilities
+        self.write_config('')
+        bitbake("diffutils-native -c addto_recipe_sysroot")
+        diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native")
+
+        # Reproducible builds should not pull from sstate or mirrors, but
+        # sharing DL_DIR is fine
         common_config = textwrap.dedent('''\
             INHERIT += "reproducible_build"
             PACKAGE_CLASSES = "%s"
+            SSTATE_DIR = "${TMPDIR}/sstate"
+            SSTATE_MIRROR = ""
             ''') % (' '.join('package_%s' % c for c in self.package_classes))
 
-        # Do an initial build. It's acceptable for this build to use sstate
-        self.write_config(common_config)
-        vars_reference = get_bb_vars(capture_vars)
-        bitbake(' '.join(self.images))
+        # Perform a build.
+        reproducibleA_tmp = os.path.join(self.topdir, 'reproducibleA', 'tmp')
+        if os.path.exists(reproducibleA_tmp):
+            bb.utils.remove(reproducibleA_tmp, recurse=True)
 
-        # Build native utilities
-        bitbake("diffutils-native -c addto_recipe_sysroot")
-        diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native")
+        self.write_config((textwrap.dedent('''\
+            TMPDIR = "%s"
+            ''') % reproducibleA_tmp) + common_config)
+        vars_A = get_bb_vars(capture_vars)
+        bitbake(' '.join(self.images))
 
-        reproducible_tmp = os.path.join(self.topdir, 'reproducible', 'tmp')
-        if os.path.exists(reproducible_tmp):
-            bb.utils.remove(reproducible_tmp, recurse=True)
+        # Perform another build.
+        reproducibleB_tmp = os.path.join(self.topdir, 'reproducibleB', 'tmp')
+        if os.path.exists(reproducibleB_tmp):
+            bb.utils.remove(reproducibleB_tmp, recurse=True)
 
-        # Perform another build. This build should *not* share sstate or pull
-        # from any mirrors, but sharing a DL_DIR is fine
         self.write_config((textwrap.dedent('''\
             TMPDIR = "%s"
-            SSTATE_DIR = "${TMPDIR}/sstate"
-            SSTATE_MIRROR = ""
-            ''') % reproducible_tmp) + common_config)
-        vars_test = get_bb_vars(capture_vars)
+            ''') % reproducibleB_tmp) + common_config)
+        vars_B = get_bb_vars(capture_vars)
         bitbake(' '.join(self.images))
 
-        # NOTE: The temp directory from the reproducible build is purposely
+        # NOTE: The temp directories from the reproducible build are purposely
         # kept after the build so it can be diffed for debugging.
 
         for c in self.package_classes:
             with self.subTest(package_class=c):
                 package_class = 'package_' + c
 
-                deploy_reference = vars_reference['DEPLOY_DIR_' + c.upper()]
-                deploy_test = vars_test['DEPLOY_DIR_' + c.upper()]
+                deploy_A = vars_A['DEPLOY_DIR_' + c.upper()]
+                deploy_B = vars_B['DEPLOY_DIR_' + c.upper()]
 
-                result = self.compare_packages(deploy_reference, deploy_test, diffutils_sysroot)
+                result = self.compare_packages(deploy_A, deploy_B, diffutils_sysroot)
 
                 self.logger.info('Reproducibility summary for %s: %s' % (c, result))
 
-- 
2.21.0

✗ patchtest: failure for oeqa: reproducible: Do two clean builds

[Patchwork]

== Series Details ==

Series: oeqa: reproducible: Do two clean builds
Revision: 1
URL   : https://patchwork.openembedded.org/series/19569/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at fb064356af)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Guidelines:     https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe