[PATCH 0/3] selftest: reproducible: Various improvements

Openembedded Core Discussions
 help / color / mirror / Atom feed

* [PATCH 0/3] selftest: reproducible: Various improvements
@ 2020-01-19 18:59 Joshua Watt
  2020-01-19 18:59 ` [PATCH 1/3] oeqa: reproducible: Add flag for building from sstate Joshua Watt
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Joshua Watt @ 2020-01-19 18:59 UTC (permalink / raw)
  To: openembedded-core

Adds improvements and bug fixes to reproducible builds to make it easier
to diagnose reproducibility problems.

Joshua Watt (3):
  oeqa: reproducible: Add flag for building from sstate
  oeqa: reproducible: Fix extra data reporting
  oeqa: reproducible: Do not strip packages

 meta/lib/oeqa/selftest/cases/reproducible.py | 77 +++++++++++---------
 1 file changed, 41 insertions(+), 36 deletions(-)

-- 
2.24.1



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH 1/3] oeqa: reproducible: Add flag for building from sstate
  2020-01-19 18:59 [PATCH 0/3] selftest: reproducible: Various improvements Joshua Watt
@ 2020-01-19 18:59 ` Joshua Watt
  2020-01-19 18:59 ` [PATCH 2/3] oeqa: reproducible: Fix extra data reporting Joshua Watt
  2020-01-19 18:59 ` [PATCH 3/3] oeqa: reproducible: Do not strip packages Joshua Watt
  2 siblings, 0 replies; 8+ messages in thread
From: Joshua Watt @ 2020-01-19 18:59 UTC (permalink / raw)
  To: openembedded-core

Adds a flag to control if the reproducible QA test should allow building
from sstate or not. Building from sstate may not be reproducible
depending on how the sstate is populated.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 66 +++++++++++---------
 1 file changed, 35 insertions(+), 31 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index db538a4f891..b7484f3e10e 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -80,6 +80,13 @@ class ReproducibleTests(OESelftestTestCase):
     package_classes = ['deb', 'ipk']
     images = ['core-image-minimal', 'core-image-sato', 'core-image-full-cmdline']
     save_results = False
+    # This variable controls if one of the test builds is allowed to pull from
+    # an sstate cache/mirror. The other build is always done clean as a point of
+    # comparison.
+    # If you know that your sstate archives are reproducible, enabling this
+    # will test that and also make the test run faster. If your sstate is not
+    # reproducible, disable this in your derived test class
+    build_from_sstate = True
 
     def setUpLocal(self):
         super().setUpLocal()
@@ -127,7 +134,7 @@ class ReproducibleTests(OESelftestTestCase):
         bb.utils.mkdirhier(os.path.dirname(dest))
         shutil.copyfile(source, dest)
 
-    def test_reproducible_builds(self):
+    def do_test_build(self, name, use_sstate):
         capture_vars = ['DEPLOY_DIR_' + c.upper() for c in self.package_classes]
 
         if self.save_results:
@@ -135,41 +142,38 @@ class ReproducibleTests(OESelftestTestCase):
             os.chmod(save_dir, stat.S_IRWXU | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH)
             self.logger.info('Non-reproducible packages will be copied to %s', save_dir)
 
-        # Build native utilities
-        self.write_config('')
-        bitbake("diffutils-native -c addto_recipe_sysroot")
-        diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native")
+        tmpdir = os.path.join(self.topdir, name, 'tmp')
+        if os.path.exists(tmpdir):
+            bb.utils.remove(tmpdir, recurse=True)
 
-        # Reproducible builds should not pull from sstate or mirrors, but
-        # sharing DL_DIR is fine
-        common_config = textwrap.dedent('''\
+        config = textwrap.dedent('''\
             INHERIT += "reproducible_build"
-            PACKAGE_CLASSES = "%s"
-            SSTATE_DIR = "${TMPDIR}/sstate"
-            ''') % (' '.join('package_%s' % c for c in self.package_classes))
-
-        # Perform a build.
-        reproducibleA_tmp = os.path.join(self.topdir, 'reproducibleA', 'tmp')
-        if os.path.exists(reproducibleA_tmp):
-            bb.utils.remove(reproducibleA_tmp, recurse=True)
-
-        self.write_config((textwrap.dedent('''\
-            TMPDIR = "%s"
-            ''') % reproducibleA_tmp) + common_config)
-        vars_A = get_bb_vars(capture_vars)
+            PACKAGE_CLASSES = "{package_classes}"
+            TMPDIR = "{tmpdir}"
+            ''').format(package_classes=' '.join('package_%s' % c for c in self.package_classes),
+                        tmpdir=tmpdir)
+
+        if not use_sstate:
+            # This config fragment will disable using shared and the sstate
+            # mirror, forcing a complete build from scratch
+            config += textwrap.dedent('''\
+                SSTATE_DIR = "${TMPDIR}/sstate"
+                SSTATE_MIRROR = ""
+                ''')
+
+        self.write_config(config)
+        d = get_bb_vars(capture_vars)
         bitbake(' '.join(self.images))
+        return d
 
-        # Perform another build.
-        reproducibleB_tmp = os.path.join(self.topdir, 'reproducibleB', 'tmp')
-        if os.path.exists(reproducibleB_tmp):
-            bb.utils.remove(reproducibleB_tmp, recurse=True)
+    def test_reproducible_builds(self):
+        # Build native utilities
+        self.write_config('')
+        bitbake("diffutils-native -c addto_recipe_sysroot")
+        diffutils_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "diffutils-native")
 
-        self.write_config((textwrap.dedent('''\
-            SSTATE_MIRROR = ""
-            TMPDIR = "%s"
-            ''') % reproducibleB_tmp) + common_config)
-        vars_B = get_bb_vars(capture_vars)
-        bitbake(' '.join(self.images))
+        vars_A = self.do_test_build('reproducibleA', self.build_from_sstate)
+        vars_B = self.do_test_build('reproducibleB', False)
 
         # NOTE: The temp directories from the reproducible build are purposely
         # kept after the build so it can be diffed for debugging.
-- 
2.24.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 2/3] oeqa: reproducible: Fix extra data reporting
  2020-01-19 18:59 [PATCH 0/3] selftest: reproducible: Various improvements Joshua Watt
  2020-01-19 18:59 ` [PATCH 1/3] oeqa: reproducible: Add flag for building from sstate Joshua Watt
@ 2020-01-19 18:59 ` Joshua Watt
  2020-01-19 18:59 ` [PATCH 3/3] oeqa: reproducible: Do not strip packages Joshua Watt
  2 siblings, 0 replies; 8+ messages in thread
From: Joshua Watt @ 2020-01-19 18:59 UTC (permalink / raw)
  To: openembedded-core

A typo was preventing the extra data about the reproducible build from
being reported in the test results

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index b7484f3e10e..c00b92d2e90 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -95,12 +95,12 @@ class ReproducibleTests(OESelftestTestCase):
         for v in needed_vars:
             setattr(self, v.lower(), bb_vars[v])
 
-        self.extrasresults = {}
-        self.extrasresults.setdefault('reproducible.rawlogs', {})['log'] = ''
-        self.extrasresults.setdefault('reproducible', {}).setdefault('files', {})
+        self.extraresults = {}
+        self.extraresults.setdefault('reproducible.rawlogs', {})['log'] = ''
+        self.extraresults.setdefault('reproducible', {}).setdefault('files', {})
 
     def append_to_log(self, msg):
-        self.extrasresults['reproducible.rawlogs']['log'] += msg
+        self.extraresults['reproducible.rawlogs']['log'] += msg
 
     def compare_packages(self, reference_dir, test_dir, diffutils_sysroot):
         result = PackageCompareResults()
@@ -127,7 +127,7 @@ class ReproducibleTests(OESelftestTestCase):
         return result
 
     def write_package_list(self, package_class, name, packages):
-        self.extrasresults['reproducible']['files'].setdefault(package_class, {})[name] = [
+        self.extraresults['reproducible']['files'].setdefault(package_class, {})[name] = [
                 {'reference': p.reference, 'test': p.test} for p in packages]
 
     def copy_file(self, source, dest):
-- 
2.24.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* [PATCH 3/3] oeqa: reproducible: Do not strip packages
  2020-01-19 18:59 [PATCH 0/3] selftest: reproducible: Various improvements Joshua Watt
  2020-01-19 18:59 ` [PATCH 1/3] oeqa: reproducible: Add flag for building from sstate Joshua Watt
  2020-01-19 18:59 ` [PATCH 2/3] oeqa: reproducible: Fix extra data reporting Joshua Watt
@ 2020-01-19 18:59 ` Joshua Watt
  2020-01-22 15:53   ` Richard Purdie
  2 siblings, 1 reply; 8+ messages in thread
From: Joshua Watt @ 2020-01-19 18:59 UTC (permalink / raw)
  To: openembedded-core

Do not strip packages when testing reproducible builds. In some cases,
stripped data differs between builds, but then gets removed. However,
the contents affect the generation of the GCC build-id, which then
differs in the resulting ELF files, even though the data that caused
this is no longer there. Inhibit stripping so that their causes can be
evaluated.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index c00b92d2e90..34462a0b3a4 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -149,6 +149,7 @@ class ReproducibleTests(OESelftestTestCase):
         config = textwrap.dedent('''\
             INHERIT += "reproducible_build"
             PACKAGE_CLASSES = "{package_classes}"
+            INHIBIT_PACKAGE_STRIP = "1"
             TMPDIR = "{tmpdir}"
             ''').format(package_classes=' '.join('package_%s' % c for c in self.package_classes),
                         tmpdir=tmpdir)
-- 
2.24.1



^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH 3/3] oeqa: reproducible: Do not strip packages
  2020-01-19 18:59 ` [PATCH 3/3] oeqa: reproducible: Do not strip packages Joshua Watt
@ 2020-01-22 15:53   ` Richard Purdie
  2020-01-22 16:02     ` Joshua Watt
  0 siblings, 1 reply; 8+ messages in thread
From: Richard Purdie @ 2020-01-22 15:53 UTC (permalink / raw)
  To: Joshua Watt, openembedded-core

On Sun, 2020-01-19 at 12:59 -0600, Joshua Watt wrote:
> Do not strip packages when testing reproducible builds. In some cases,
> stripped data differs between builds, but then gets removed. However,
> the contents affect the generation of the GCC build-id, which then
> differs in the resulting ELF files, even though the data that caused
> this is no longer there. Inhibit stripping so that their causes can be
> evaluated.
> 
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>  meta/lib/oeqa/selftest/cases/reproducible.py | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
> index c00b92d2e90..34462a0b3a4 100644
> --- a/meta/lib/oeqa/selftest/cases/reproducible.py
> +++ b/meta/lib/oeqa/selftest/cases/reproducible.py
> @@ -149,6 +149,7 @@ class ReproducibleTests(OESelftestTestCase):
>          config = textwrap.dedent('''\
>              INHERIT += "reproducible_build"
>              PACKAGE_CLASSES = "{package_classes}"
> +            INHIBIT_PACKAGE_STRIP = "1"
>              TMPDIR = "{tmpdir}"
>              ''').format(package_classes=' '.join('package_%s' % c for c in self.package_classes),
>                          tmpdir=tmpdir)

This generates quite the failure list:

 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dbg_20190110-r0_all.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dev_20190110-r0_all.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-doc_20190110-r0_all.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates_20190110-r0_all.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/acl-ptest_2.2.53-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/attr-ptest_2.4.48-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/bc-doc_1.07.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/binutils-doc_2.33.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/busybox-ptest_1.31.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/coreutils-doc_8.31-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/e2fsprogs-doc_1.45.4-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/elfutils-ptest_0.178-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ethtool-ptest_5.3-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/flex-ptest_2.6.4-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-dbg_3.24.13-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-src_3.24.13-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/icu-dev_64.2-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libevdev-src_1.8.0-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt-dbg_1.8.5-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt_1.8.5-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgtk-3.0_3.24.13-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libidn2-dev_2.3.0-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput-dbg_1.14.3-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput10_1.14.3-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libx11-locale_1.6.9-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxml2-ptest_2.9.10-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence-dbg_1.3-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence1_1.3-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/m4-ptest_1.4.18-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-doc_4.8.23-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-helpers-perl_4.8.23-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ncurses-doc_6.1+20190803-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-doc_1.1.1d-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-ptest_1.1.1d-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/opkg-utils-doc_0.4.2-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dbg_5.30.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dev_5.30.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-config-heavy_5.30.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-data-dumper_5.30.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-ptest_5.30.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl_5.30.1-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-dbg_1.8.29-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-doc_1.8.29-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo_1.8.29-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar-dbg_1.32-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar_1.32-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tcp-wrappers-src_7.6-r10_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/time-doc_1.9-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg-dbg_1.20.6-r0_amd64.deb
 /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg_1.20.6-r0_amd64.deb

(I lined wrapped it)

https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/629

Cheers,

Richard





^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH 3/3] oeqa: reproducible: Do not strip packages
  2020-01-22 15:53   ` Richard Purdie
@ 2020-01-22 16:02     ` Joshua Watt
  2020-01-22 16:06       ` Richard Purdie
  0 siblings, 1 reply; 8+ messages in thread
From: Joshua Watt @ 2020-01-22 16:02 UTC (permalink / raw)
  To: Richard Purdie, openembedded-core


On 1/22/20 9:53 AM, Richard Purdie wrote:
> On Sun, 2020-01-19 at 12:59 -0600, Joshua Watt wrote:
>> Do not strip packages when testing reproducible builds. In some cases,
>> stripped data differs between builds, but then gets removed. However,
>> the contents affect the generation of the GCC build-id, which then
>> differs in the resulting ELF files, even though the data that caused
>> this is no longer there. Inhibit stripping so that their causes can be
>> evaluated.
>>
>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>> ---
>>   meta/lib/oeqa/selftest/cases/reproducible.py | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
>> index c00b92d2e90..34462a0b3a4 100644
>> --- a/meta/lib/oeqa/selftest/cases/reproducible.py
>> +++ b/meta/lib/oeqa/selftest/cases/reproducible.py
>> @@ -149,6 +149,7 @@ class ReproducibleTests(OESelftestTestCase):
>>           config = textwrap.dedent('''\
>>               INHERIT += "reproducible_build"
>>               PACKAGE_CLASSES = "{package_classes}"
>> +            INHIBIT_PACKAGE_STRIP = "1"
>>               TMPDIR = "{tmpdir}"
>>               ''').format(package_classes=' '.join('package_%s' % c for c in self.package_classes),
>>                           tmpdir=tmpdir)
> This generates quite the failure list:
>
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dbg_20190110-r0_all.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dev_20190110-r0_all.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-doc_20190110-r0_all.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates_20190110-r0_all.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/acl-ptest_2.2.53-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/attr-ptest_2.4.48-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/bc-doc_1.07.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/binutils-doc_2.33.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/busybox-ptest_1.31.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/coreutils-doc_8.31-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/e2fsprogs-doc_1.45.4-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/elfutils-ptest_0.178-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ethtool-ptest_5.3-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/flex-ptest_2.6.4-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-dbg_3.24.13-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-src_3.24.13-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/icu-dev_64.2-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libevdev-src_1.8.0-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt-dbg_1.8.5-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt_1.8.5-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgtk-3.0_3.24.13-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libidn2-dev_2.3.0-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput-dbg_1.14.3-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput10_1.14.3-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libx11-locale_1.6.9-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxml2-ptest_2.9.10-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence-dbg_1.3-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence1_1.3-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/m4-ptest_1.4.18-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-doc_4.8.23-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-helpers-perl_4.8.23-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ncurses-doc_6.1+20190803-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-doc_1.1.1d-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-ptest_1.1.1d-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/opkg-utils-doc_0.4.2-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dbg_5.30.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dev_5.30.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-config-heavy_5.30.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-data-dumper_5.30.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-ptest_5.30.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl_5.30.1-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-dbg_1.8.29-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-doc_1.8.29-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo_1.8.29-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar-dbg_1.32-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar_1.32-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tcp-wrappers-src_7.6-r10_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/time-doc_1.9-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg-dbg_1.20.6-r0_amd64.deb
>   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg_1.20.6-r0_amd64.deb
>
> (I lined wrapped it)
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/629


Ok, thanks. Did the offending packages get captured somewhere so I can 
take a look at them?


>
> Cheers,
>
> Richard
>
>
>


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH 3/3] oeqa: reproducible: Do not strip packages
  2020-01-22 16:02     ` Joshua Watt
@ 2020-01-22 16:06       ` Richard Purdie
  2020-01-22 18:04         ` Joshua Watt
  0 siblings, 1 reply; 8+ messages in thread
From: Richard Purdie @ 2020-01-22 16:06 UTC (permalink / raw)
  To: Joshua Watt, openembedded-core

On Wed, 2020-01-22 at 10:02 -0600, Joshua Watt wrote:
> On 1/22/20 9:53 AM, Richard Purdie wrote:
> > On Sun, 2020-01-19 at 12:59 -0600, Joshua Watt wrote:
> > > Do not strip packages when testing reproducible builds. In some cases,
> > > stripped data differs between builds, but then gets removed. However,
> > > the contents affect the generation of the GCC build-id, which then
> > > differs in the resulting ELF files, even though the data that caused
> > > this is no longer there. Inhibit stripping so that their causes can be
> > > evaluated.
> > > 
> > > Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> > > ---
> > >   meta/lib/oeqa/selftest/cases/reproducible.py | 1 +
> > >   1 file changed, 1 insertion(+)
> > > 
> > > diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
> > > index c00b92d2e90..34462a0b3a4 100644
> > > --- a/meta/lib/oeqa/selftest/cases/reproducible.py
> > > +++ b/meta/lib/oeqa/selftest/cases/reproducible.py
> > > @@ -149,6 +149,7 @@ class ReproducibleTests(OESelftestTestCase):
> > >           config = textwrap.dedent('''\
> > >               INHERIT += "reproducible_build"
> > >               PACKAGE_CLASSES = "{package_classes}"
> > > +            INHIBIT_PACKAGE_STRIP = "1"
> > >               TMPDIR = "{tmpdir}"
> > >               ''').format(package_classes=' '.join('package_%s' % c for c in self.package_classes),
> > >                           tmpdir=tmpdir)
> > This generates quite the failure list:
> > 
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dbg_20190110-r0_all.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dev_20190110-r0_all.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-doc_20190110-r0_all.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates_20190110-r0_all.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/acl-ptest_2.2.53-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/attr-ptest_2.4.48-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/bc-doc_1.07.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/binutils-doc_2.33.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/busybox-ptest_1.31.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/coreutils-doc_8.31-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/e2fsprogs-doc_1.45.4-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/elfutils-ptest_0.178-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ethtool-ptest_5.3-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/flex-ptest_2.6.4-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-dbg_3.24.13-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-src_3.24.13-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/icu-dev_64.2-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libevdev-src_1.8.0-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt-dbg_1.8.5-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt_1.8.5-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgtk-3.0_3.24.13-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libidn2-dev_2.3.0-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput-dbg_1.14.3-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput10_1.14.3-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libx11-locale_1.6.9-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxml2-ptest_2.9.10-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence-dbg_1.3-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence1_1.3-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/m4-ptest_1.4.18-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-doc_4.8.23-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-helpers-perl_4.8.23-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ncurses-doc_6.1+20190803-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-doc_1.1.1d-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-ptest_1.1.1d-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/opkg-utils-doc_0.4.2-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dbg_5.30.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dev_5.30.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-config-heavy_5.30.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-data-dumper_5.30.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-ptest_5.30.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl_5.30.1-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-dbg_1.8.29-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-doc_1.8.29-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo_1.8.29-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar-dbg_1.32-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar_1.32-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tcp-wrappers-src_7.6-r10_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/time-doc_1.9-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg-dbg_1.20.6-r0_amd64.deb
> >   /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg_1.20.6-r0_amd64.deb
> > 
> > (I lined wrapped it)
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/629
> 
> Ok, thanks. Did the offending packages get captured somewhere so I can 
> take a look at them?

I've placed them here:

https://autobuilder.yocto.io/pub/repro-fail/build-st-12270/

which is after I fixed that code:

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=9f225cf966b8a96775d74f41423206d157bce2e3

Perhaps we could make this a cofiguration parameter I could pass in and
make it clean up empty directories if nothing is saved. I might then be
able to make it "auto publish" these failures.

Cheers,

Richard





^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH 3/3] oeqa: reproducible: Do not strip packages
  2020-01-22 16:06       ` Richard Purdie
@ 2020-01-22 18:04         ` Joshua Watt
  0 siblings, 0 replies; 8+ messages in thread
From: Joshua Watt @ 2020-01-22 18:04 UTC (permalink / raw)
  To: Richard Purdie, openembedded-core


On 1/22/20 10:06 AM, Richard Purdie wrote:
> On Wed, 2020-01-22 at 10:02 -0600, Joshua Watt wrote:
>> On 1/22/20 9:53 AM, Richard Purdie wrote:
>>> On Sun, 2020-01-19 at 12:59 -0600, Joshua Watt wrote:
>>>> Do not strip packages when testing reproducible builds. In some cases,
>>>> stripped data differs between builds, but then gets removed. However,
>>>> the contents affect the generation of the GCC build-id, which then
>>>> differs in the resulting ELF files, even though the data that caused
>>>> this is no longer there. Inhibit stripping so that their causes can be
>>>> evaluated.
>>>>
>>>> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
>>>> ---
>>>>    meta/lib/oeqa/selftest/cases/reproducible.py | 1 +
>>>>    1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
>>>> index c00b92d2e90..34462a0b3a4 100644
>>>> --- a/meta/lib/oeqa/selftest/cases/reproducible.py
>>>> +++ b/meta/lib/oeqa/selftest/cases/reproducible.py
>>>> @@ -149,6 +149,7 @@ class ReproducibleTests(OESelftestTestCase):
>>>>            config = textwrap.dedent('''\
>>>>                INHERIT += "reproducible_build"
>>>>                PACKAGE_CLASSES = "{package_classes}"
>>>> +            INHIBIT_PACKAGE_STRIP = "1"
>>>>                TMPDIR = "{tmpdir}"
>>>>                ''').format(package_classes=' '.join('package_%s' % c for c in self.package_classes),
>>>>                            tmpdir=tmpdir)
>>> This generates quite the failure list:
>>>
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dbg_20190110-r0_all.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-dev_20190110-r0_all.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates-doc_20190110-r0_all.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./all/ca-certificates_20190110-r0_all.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/acl-ptest_2.2.53-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/attr-ptest_2.4.48-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/bc-doc_1.07.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/binutils-doc_2.33.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/busybox-ptest_1.31.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/coreutils-doc_8.31-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/e2fsprogs-doc_1.45.4-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/elfutils-ptest_0.178-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ethtool-ptest_5.3-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/flex-ptest_2.6.4-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-dbg_3.24.13-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/gtk+3-src_3.24.13-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/icu-dev_64.2-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libevdev-src_1.8.0-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt-dbg_1.8.5-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgcrypt_1.8.5-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libgtk-3.0_3.24.13-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libidn2-dev_2.3.0-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput-dbg_1.14.3-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libinput10_1.14.3-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libx11-locale_1.6.9-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxml2-ptest_2.9.10-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence-dbg_1.3-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/libxshmfence1_1.3-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/m4-ptest_1.4.18-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-doc_4.8.23-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/mc-helpers-perl_4.8.23-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/ncurses-doc_6.1+20190803-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-doc_1.1.1d-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/openssl-ptest_1.1.1d-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/opkg-utils-doc_0.4.2-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dbg_5.30.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-dev_5.30.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-config-heavy_5.30.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-module-data-dumper_5.30.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl-ptest_5.30.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/perl_5.30.1-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-dbg_1.8.29-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo-doc_1.8.29-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/sudo_1.8.29-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar-dbg_1.32-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tar_1.32-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/tcp-wrappers-src_7.6-r10_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/time-doc_1.9-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg-dbg_1.20.6-r0_amd64.deb
>>>    /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-12270/reproducibleB/tmp/deploy/deb/./core2-64/xserver-xorg_1.20.6-r0_amd64.deb
>>>
>>> (I lined wrapped it)
>>>
>>> https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/629
>> Ok, thanks. Did the offending packages get captured somewhere so I can
>> take a look at them?
> I've placed them here:
>
> https://autobuilder.yocto.io/pub/repro-fail/build-st-12270/

OK. Thanks. I took a look at these, and FWIW, very few (if any) appear 
to be because of the addition of INHIBIT_PACKAGE_STRIP, and are instead 
due to actual reproducibility issues. I'm not sure why that patch caused 
them all to show up; perhaps because it made all the do_package tasks rerun?

>
> which is after I fixed that code:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=9f225cf966b8a96775d74f41423206d157bce2e3
>
> Perhaps we could make this a cofiguration parameter I could pass in and
> make it clean up empty directories if nothing is saved. I might then be
> able to make it "auto publish" these failures.

Ya, that would be nice. Were you thinking of making them configurable in 
the AB json config files?

>
> Cheers,
>
> Richard
>
>
>


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2020-01-22 18:04 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-01-19 18:59 [PATCH 0/3] selftest: reproducible: Various improvements Joshua Watt
2020-01-19 18:59 ` [PATCH 1/3] oeqa: reproducible: Add flag for building from sstate Joshua Watt
2020-01-19 18:59 ` [PATCH 2/3] oeqa: reproducible: Fix extra data reporting Joshua Watt
2020-01-19 18:59 ` [PATCH 3/3] oeqa: reproducible: Do not strip packages Joshua Watt
2020-01-22 15:53   ` Richard Purdie
2020-01-22 16:02     ` Joshua Watt
2020-01-22 16:06       ` Richard Purdie
2020-01-22 18:04         ` Joshua Watt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox