All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Mathieu Dubois-Briand" <mathieu.dubois-briand@bootlin.com>
To: <alhe@linux.microsoft.com>, <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [PATCH 4/4] bitbake.conf, rust-common.bbclass: include RUST_BUILD_SYS in the task hash
Date: Thu, 06 Aug 2026 07:58:41 +0200	[thread overview]
Message-ID: <DKHMPRMDWGEP.3NF324RFG6OHF@bootlin.com> (raw)
In-Reply-To: <20260804220456.2342710-4-alhe@linux.microsoft.com>

On Wed Aug 5, 2026 at 12:04 AM CEST, Alejandro Hernandez Samaniego via lists.openembedded.org wrote:
> RUST_BUILD_SYS is currently listed in BB_BASEHASH_IGNORE_VARS, which
> tells bitbake that rust recipes should hash-match regardless of the
> build machine's triplet. In practice that is not true:
>
> rustc computes each crate's Strict Version Hash (SVH) using inputs
> that include the *stage0/stage1 bootstrap compiler* fingerprint, which
> in turn depends on the build host arch. That seed cascades through
> every dependent crate's mangled symbols and the packing order of
> rodata sections, so an sstate blob populated on one worker arch and
> reused on a differently-arched worker produces byte-different (but
> semantically identical) artifacts and fails the reproducibility
> selftest causing autobuilder intermittent issues when the sstate
> matches for the incorrect architecture.
>
> Remove RUST_BUILD_SYS from BB_BASEHASH_IGNORE_VARS so the task hash
> tracks the build triplet and mixed-arch autobuilder pools get an
> sstate miss instead of a silently-wrong hit. RUST_HOST_SYS and
> RUST_TARGET_SYS stay excluded because they're already covered by the
> target/host arch hash inputs.
>
> While this is not ideal, it should unblock the reproducible test case,
> another solution would be to patch rust sources manually and attempt
> to upstream that change.
>
> This also has the side-effect that multiple variants of the conflicting
> rust recipes sstate artifacts are created, but they'll be correctly used
> now in each architecture.
>
> Also add an explanatory comment next to the RUST_*_SYS[vardepvalue]
> declarations in rust-common.bbclass so future readers don't re-add
> the exclusion.
>
> Verified locally: Tier 1 sighash test toggling BUILD_ARCH now produces
> different task hashes for rust/libstd-rs/rpm-sequoia/python3-crypto-
> graphy/cargo/librsvg, where previously the hashes matched despite the
> build machine change.
>
> [YOCTO #15554]
>
> Assisted-by: AI - OpenAI
> Signed-off-by: Alejandro Hernandez <alhe@linux.microsoft.com>
> ---

Hi Alejandro,

It looks like this is breaking some selftests:

2026-08-05 10:11:05,494 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_sdk_arch_same_hash (subunit.RemotedTestCase)
2026-08-05 10:11:05,494 - oe-selftest - INFO -  ... FAIL
...

2026-08-05 10:11:05,495 - oe-selftest - INFO - 6: 27/69 496/763 (108.24s) (2 failed) (sstatetests.SStateHashSameSigs.test_sstate_sdk_arch_same_hash)
2026-08-05 10:11:05,495 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 416, in test_sstate_sdk_arch_same_hash
    self.sstate_hashtest("aarch64")
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
  File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/layers/openembedded-core/meta/lib/oeqa/selftest/cases/sstatetests.py", line 401, in sstate_hashtest
    self.assertCountEqual(files1, files2)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/unittest/case.py", line 1238, in assertCountEqual
    self.fail(msg)
    ~~~~~~~~~^^^^^
  File "/usr/lib/python3.13/unittest/case.py", line 732, in fail
    raise self.failureException(msg)
AssertionError: Element counts were not equal:
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_create_spdx.sigdata.469683d7e676bd4feadb3e363efaa9a55c47eb06559fcda0d3f5cf26473919c5'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_populate_sysroot.sigdata.c29ff418823da4b52189a4cf7395c310a3c14fed57eb24562cd0e387fe2235a3'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/gtk+3-native/3.24.52.do_create_package_spdx.sigdata.ede62d86abb84a871e239607d0a7173c6a9c45027aafffc624b37e2e1bb5111f'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_create_spdx.sigdata.76ed6652ba8e8eba4e8750240fa7a048fcc90e94b45e753068a44560ef2e29f0'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_create_package_spdx.sigdata.4234a33aff5f4b52703a74c1caec1893206476f908c70b32c78e46dcc3e4fb38'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_compile.sigdata.67ed98cf74ac06f829d6432f4fd904779b2f6e540f588baeab3dcf0ac35a9c75'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_populate_sysroot.sigdata.3203f6ea1fd6bb92ed523ba0e1ef0464166f1e5392207ea002c35e0007d435f9'
First has 1, Second has 0:  '/srv/pokybuild/yocto-worker/oe-selftest-debian/build/build-st-3278758/tmp-sstatesamehash/stamps/x86_64-linux/rust-native/1.96.1.do_configure.sigdata.df7cc5717b4cbc2ccb19bdb823fb55193cc2f95049ac7e07d440bc2da5590c96'
...

And same for sstatetests.SStateHashSameSigs.test_sstate_32_64_same_hash:
2026-08-05 10:07:29,974 - oe-selftest - INFO - sstatetests.SStateHashSameSigs.test_sstate_32_64_same_hash (subunit.RemotedTestCase)
2026-08-05 10:07:29,975 - oe-selftest - INFO -  ... FAIL
...

https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/4472
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/4290

Can you have a look at the issue?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com



  reply	other threads:[~2026-08-06  5:58 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04 22:04 [PATCH 1/4] libtool: strip build system triplet from installed libtool script Alejandro Hernandez
2026-08-04 22:04 ` [PATCH 2/4] ptest.bbclass: strip build machine triplet from installed ptest Makefiles Alejandro Hernandez
2026-08-04 22:04 ` [PATCH 3/4] python_pep517.bbclass: always export _PYTHON_HOST_PLATFORM for wheel tag Alejandro Hernandez
2026-08-06 12:26   ` [OE-core] " Richard Purdie
2026-08-06 15:51     ` Alejandro Hernandez
2026-08-06 16:09       ` Richard Purdie
2026-08-06 17:28         ` Alejandro Hernandez
2026-08-04 22:04 ` [PATCH 4/4] bitbake.conf, rust-common.bbclass: include RUST_BUILD_SYS in the task hash Alejandro Hernandez
2026-08-06  5:58   ` Mathieu Dubois-Briand [this message]
2026-08-06 15:53     ` [OE-core] " Alejandro Hernandez
2026-08-06 12:53   ` Richard Purdie
2026-08-06 15:50     ` Alejandro Hernandez
2026-08-06 16:12       ` Richard Purdie
     [not found] ` <18C8B9671621C916.3417489@lists.openembedded.org>
2026-08-05  4:16   ` Alejandro Hernandez

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DKHMPRMDWGEP.3NF324RFG6OHF@bootlin.com \
    --to=mathieu.dubois-briand@bootlin.com \
    --cc=alhe@linux.microsoft.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.