* [PATCH 2/4] rust: add reproducibility patch to eliminate host leakage
2024-05-16 11:21 [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexander Kanavin
@ 2024-05-16 11:21 ` Alexander Kanavin
2024-05-16 11:21 ` [PATCH 3/4] rust: use rust-snapshot binaries only in rust-native Alexander Kanavin
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2024-05-16 11:21 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
From: Alexander Kanavin <alex@linutronix.de>
[YOCTO #15185]
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
...te-host-information-into-compilation.patch | 51 +++++++++++++++++++
meta/recipes-devtools/rust/rust-source.inc | 1 +
2 files changed, 52 insertions(+)
create mode 100644 meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch
diff --git a/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch b/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch
new file mode 100644
index 00000000000..a6ee8676058
--- /dev/null
+++ b/meta/recipes-devtools/rust/files/0001-cargo-do-not-write-host-information-into-compilation.patch
@@ -0,0 +1,51 @@
+From 065d7c263091118437465d714d8a29dbb6296921 Mon Sep 17 00:00:00 2001
+From: Alexander Kanavin <alex@linutronix.de>
+Date: Mon, 13 May 2024 14:57:54 +0200
+Subject: [PATCH] cargo: do not write host information into compilation unit
+ hashes
+
+This breaks reproducibility in cross-builds where the cross-target
+can be the same, but build hosts are different, as seen with
+"rustc --version -v":
+...
+host: x86_64-unknown-linux-gnu
+
+vs.
+
+host: aarch64-unknown-linux-gnu
+
+This can possibly be improved by only hashing host info if the build
+is a native one (e.g. there's no --target option passed to cargo
+invocation) but I'm not sure how.
+
+Upstream-Status: Inappropriate [reported at https://github.com/rust-lang/cargo/issues/13922]
+Signed-off-by: Alexander Kanavin <alex@linutronix.de>
+---
+ .../src/cargo/core/compiler/context/compilation_files.rs | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs b/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs
+index d83dbf10c..b2ad8d9f3 100644
+--- a/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs
++++ b/src/tools/cargo/src/cargo/core/compiler/context/compilation_files.rs
+@@ -652,7 +652,7 @@ fn hash_rustc_version(bcx: &BuildContext<'_, '_>, hasher: &mut StableHasher) {
+ if vers.pre.is_empty() || bcx.config.cli_unstable().separate_nightlies {
+ // For stable, keep the artifacts separate. This helps if someone is
+ // testing multiple versions, to avoid recompiles.
+- bcx.rustc().verbose_version.hash(hasher);
++ //bcx.rustc().verbose_version.hash(hasher);
+ return;
+ }
+ // On "nightly"/"beta"/"dev"/etc, keep each "channel" separate. Don't hash
+@@ -665,7 +665,7 @@ fn hash_rustc_version(bcx: &BuildContext<'_, '_>, hasher: &mut StableHasher) {
+ // Keep "host" since some people switch hosts to implicitly change
+ // targets, (like gnu vs musl or gnu vs msvc). In the future, we may want
+ // to consider hashing `unit.kind.short_name()` instead.
+- bcx.rustc().host.hash(hasher);
++ //bcx.rustc().host.hash(hasher);
+ // None of the other lines are important. Currently they are:
+ // binary: rustc <-- or "rustdoc"
+ // commit-hash: 38114ff16e7856f98b2b4be7ab4cd29b38bed59a
+--
+2.39.2
+
diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc
index c83c8ec3a39..20ef5e82bc4 100644
--- a/meta/recipes-devtools/rust/rust-source.inc
+++ b/meta/recipes-devtools/rust/rust-source.inc
@@ -12,6 +12,7 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
file://target-build-value.patch;patchdir=${RUSTSRC} \
file://0001-Handle-vendored-sources-when-remapping-paths.patch;patchdir=${RUSTSRC} \
file://repro-issue-fix-with-v175.patch;patchdir=${RUSTSRC} \
+ file://0001-cargo-do-not-write-host-information-into-compilation.patch;patchdir=${RUSTSRC} \
"
SRC_URI[rust.sha256sum] = "4526f786d673e4859ff2afa0bab2ba13c918b796519a25c1acce06dba9542340"
--
2.39.2
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3/4] rust: use rust-snapshot binaries only in rust-native
2024-05-16 11:21 [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexander Kanavin
2024-05-16 11:21 ` [PATCH 2/4] rust: add reproducibility patch to eliminate host leakage Alexander Kanavin
@ 2024-05-16 11:21 ` Alexander Kanavin
2024-05-16 11:21 ` [PATCH 4/4] rust: build the default set of tools Alexander Kanavin
2024-05-18 21:55 ` [OE-core] [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexandre Belloni
3 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2024-05-16 11:21 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
From: Alexander Kanavin <alex@linutronix.de>
Otherwise, use rust-native and cargo-native binaries as that allows
our native tweaks in them to be used for target/nativesdk rust -
same as for everything else written in rust.
In particular, this allows building target rust with
cargo-native that includes important reproducibility tweaks.
Unfortunately, this also breaks rust selftest, and that
is partially addressed by the following commit.
[YOCTO #15185]
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
meta/recipes-devtools/rust/rust_1.75.0.bb | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-devtools/rust/rust_1.75.0.bb b/meta/recipes-devtools/rust/rust_1.75.0.bb
index 8ef838ee90c..e82a7395e44 100644
--- a/meta/recipes-devtools/rust/rust_1.75.0.bb
+++ b/meta/recipes-devtools/rust/rust_1.75.0.bb
@@ -11,6 +11,11 @@ DEPENDS += "file-native python3-native"
DEPENDS:append:class-native = " rust-llvm-native"
DEPENDS:append:class-nativesdk = " nativesdk-rust-llvm"
+# native rust uses cargo/rustc from binary snapshots to bootstrap
+# but everything else should use our native builds
+DEPENDS:append:class-target = " cargo-native rust-native"
+DEPENDS:append:class-nativesdk = " cargo-native rust-native"
+
DEPENDS += "rust-llvm (=${PV})"
RDEPENDS:${PN}:append:class-target = " gcc g++ binutils"
@@ -68,6 +73,11 @@ addtask do_test_compile after do_configure do_rust_gen_targets
do_rust_setup_snapshot[dirs] += "${WORKDIR}/rust-snapshot"
do_rust_setup_snapshot[vardepsexclude] += "UNINATIVE_LOADER"
+RUSTC_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/rustc"
+CARGO_BOOTSTRAP = "${STAGING_BINDIR_NATIVE}/cargo"
+RUSTC_BOOTSTRAP:class-native = "${WORKDIR}/rust-snapshot/bin/rustc"
+CARGO_BOOTSTRAP:class-native = "${WORKDIR}/rust-snapshot/bin/cargo"
+
python do_configure() {
import json
import configparser
@@ -141,10 +151,10 @@ python do_configure() {
config.set("build", "docs", e(False))
config.set("build", "tools", ["rust-demangler",])
- rustc = d.expand("${WORKDIR}/rust-snapshot/bin/rustc")
+ rustc = d.getVar('RUSTC_BOOTSTRAP')
config.set("build", "rustc", e(rustc))
- cargo = d.expand("${WORKDIR}/rust-snapshot/bin/cargo")
+ cargo = d.getVar('CARGO_BOOTSTRAP')
config.set("build", "cargo", e(cargo))
config.set("build", "vendor", e(True))
--
2.39.2
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 4/4] rust: build the default set of tools
2024-05-16 11:21 [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexander Kanavin
2024-05-16 11:21 ` [PATCH 2/4] rust: add reproducibility patch to eliminate host leakage Alexander Kanavin
2024-05-16 11:21 ` [PATCH 3/4] rust: use rust-snapshot binaries only in rust-native Alexander Kanavin
@ 2024-05-16 11:21 ` Alexander Kanavin
2024-05-18 21:55 ` [OE-core] [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexandre Belloni
3 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2024-05-16 11:21 UTC (permalink / raw)
To: openembedded-core; +Cc: Alexander Kanavin
From: Alexander Kanavin <alex@linutronix.de>
Setting it explicitly replaces rust's default choice which is rustdoc
(needed for example in selftests and otherwise expected to be present
in typical rust installations):
https://github.com/rust-lang/rust/blob/master/config.example.toml#L320
This addresses some of the rust selftest failures but not all. Help
is appreciate to restore the selftest.
Unfortunately, this also breaks rust reproducibility (or rather exposes
that it was never properly fixed, as explained here:
https://lists.openembedded.org/g/openembedded-core/message/199288
)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
---
meta/lib/oeqa/selftest/cases/reproducible.py | 2 ++
meta/recipes-devtools/rust/rust_1.75.0.bb | 1 -
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index 80e830136f7..97a9c3da908 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -16,6 +16,8 @@ import os
import datetime
exclude_packages = [
+ 'rust-rustdoc',
+ 'rust-dbg'
]
def is_excluded(package):
diff --git a/meta/recipes-devtools/rust/rust_1.75.0.bb b/meta/recipes-devtools/rust/rust_1.75.0.bb
index e82a7395e44..b041a5f8e4c 100644
--- a/meta/recipes-devtools/rust/rust_1.75.0.bb
+++ b/meta/recipes-devtools/rust/rust_1.75.0.bb
@@ -149,7 +149,6 @@ python do_configure() {
config.add_section("build")
config.set("build", "submodules", e(False))
config.set("build", "docs", e(False))
- config.set("build", "tools", ["rust-demangler",])
rustc = d.getVar('RUSTC_BOOTSTRAP')
config.set("build", "rustc", e(rustc))
--
2.39.2
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [OE-core] [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0
2024-05-16 11:21 [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexander Kanavin
` (2 preceding siblings ...)
2024-05-16 11:21 ` [PATCH 4/4] rust: build the default set of tools Alexander Kanavin
@ 2024-05-18 21:55 ` Alexandre Belloni
2024-05-19 0:27 ` Alexander Kanavin
3 siblings, 1 reply; 7+ messages in thread
From: Alexandre Belloni @ 2024-05-18 21:55 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-core, Alexander Kanavin
ello,
The series broke the -tc builds as
rust.RustSelfTestSystemEmulated.test_rust is failing:
https://autobuilder.yoctoproject.org/typhoon/#/builders/151/builds/1723/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/154/builds/1715/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/1711/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/145/builds/1700/steps/12/logs/stdio
On 16/05/2024 13:21:08+0200, Alexander Kanavin wrote:
> From: Alexander Kanavin <alex@linutronix.de>
>
> This does not seem to be used in regular builds, but is beneficial
> in rust selftest, where it allows dropping a custom patch
> that is unsuitable for upstream (and was rejected by them).
>
> Also remove an obsolete comment that seems related to the code
> but describes something that was resolved long time ago.
>
> I have confirmed that the rust selftest continues to pass with just
> this one commit on top of master (as the following changes do break
> the selftest).
>
> Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> ---
> .../rust/files/cargo-path.patch | 37 -------------------
> meta/recipes-devtools/rust/rust-source.inc | 1 -
> meta/recipes-devtools/rust/rust_1.75.0.bb | 6 +--
> 3 files changed, 2 insertions(+), 42 deletions(-)
> delete mode 100644 meta/recipes-devtools/rust/files/cargo-path.patch
>
> diff --git a/meta/recipes-devtools/rust/files/cargo-path.patch b/meta/recipes-devtools/rust/files/cargo-path.patch
> deleted file mode 100644
> index 9a50c402201..00000000000
> --- a/meta/recipes-devtools/rust/files/cargo-path.patch
> +++ /dev/null
> @@ -1,37 +0,0 @@
> -Fix the cargo binary path error and ensure that it is fetched
> -during rustc bootstrap in rust oe-selftest.
> -
> -======================================================================
> -ERROR: test_cargoflags (bootstrap_test.BuildBootstrap)
> -----------------------------------------------------------------------
> -Traceback (most recent call last):
> - File "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap_test.py", line 157, in test_cargoflags
> - args, _ = self.build_args(env={"CARGOFLAGS": "--timings"})
> - File "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap_test.py", line 154, in build_args
> - return build.build_bootstrap_cmd(env), env
> - File "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap.py", line 960, in build_bootstrap_cmd
> - raise Exception("no cargo executable found at `{}`".format(
> -Exception: no cargo executable found at `/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo`
> -
> -Upstream-Status: Submitted [https://github.com/rust-lang/rust/pull/120125]
> -
> -Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
> ----
> -diff --git a/src/bootstrap/bootstrap.py b/src/bootstrap/bootstrap.py
> ---- a/src/bootstrap/bootstrap.py
> -+++ b/src/bootstrap/bootstrap.py
> -@@ -954,9 +954,11 @@
> - if "RUSTFLAGS_BOOTSTRAP" in env:
> - env["RUSTFLAGS"] += " " + env["RUSTFLAGS_BOOTSTRAP"]
> -
> -- env["PATH"] = os.path.join(self.bin_root(), "bin") + \
> -- os.pathsep + env["PATH"]
> -- if not os.path.isfile(self.cargo()):
> -+ cargo_bin_path = os.path.join(self.bin_root(), "bin", "cargo")
> -+ if not os.path.isfile(cargo_bin_path):
> -+ cargo_bin_path = os.getenv("RUST_TARGET_PATH") + "rust-snapshot/bin/cargo"
> -+ env["PATH"] = os.path.dirname(cargo_bin_path) + os.pathsep + env["PATH"]
> -+ else:
> - raise Exception("no cargo executable found at `{}`".format(
> - self.cargo()))
> - args = [self.cargo(), "build", "--manifest-path",
> diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc
> index b14221b6cb8..c83c8ec3a39 100644
> --- a/meta/recipes-devtools/rust/rust-source.inc
> +++ b/meta/recipes-devtools/rust/rust-source.inc
> @@ -7,7 +7,6 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
> file://rv32-missing-syscalls.patch;patchdir=${RUSTSRC} \
> file://rv32-rustix-libc-backend.patch;patchdir=${RUSTSRC} \
> file://rv32-cargo-rustix-0.38.19-fix.patch;patchdir=${RUSTSRC} \
> - file://cargo-path.patch;patchdir=${RUSTSRC} \
> file://custom-target-cfg.patch;patchdir=${RUSTSRC} \
> file://rustc-bootstrap.patch;patchdir=${RUSTSRC} \
> file://target-build-value.patch;patchdir=${RUSTSRC} \
> diff --git a/meta/recipes-devtools/rust/rust_1.75.0.bb b/meta/recipes-devtools/rust/rust_1.75.0.bb
> index 76e1fe2d84a..8ef838ee90c 100644
> --- a/meta/recipes-devtools/rust/rust_1.75.0.bb
> +++ b/meta/recipes-devtools/rust/rust_1.75.0.bb
> @@ -35,8 +35,6 @@ RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-co
> # own vendoring.
> CARGO_DISABLE_BITBAKE_VENDORING = "1"
>
> -# We can't use RUST_BUILD_SYS here because that may be "musl" if
> -# TCLIBC="musl". Snapshots are always -unknown-linux-gnu
> setup_cargo_environment () {
> # The first step is to build bootstrap and some early stage tools,
> # these are build for the same target as the snapshot, e.g.
> @@ -54,8 +52,8 @@ do_rust_setup_snapshot () {
>
> # Some versions of rust (e.g. 1.18.0) tries to find cargo in stage0/bin/cargo
> # and fail without it there.
> - mkdir -p ${RUSTSRC}/build/${BUILD_SYS}
> - ln -sf ${WORKDIR}/rust-snapshot/ ${RUSTSRC}/build/${BUILD_SYS}/stage0
> + mkdir -p ${RUSTSRC}/build/${RUST_BUILD_SYS}
> + ln -sf ${WORKDIR}/rust-snapshot/ ${RUSTSRC}/build/${RUST_BUILD_SYS}/stage0
>
> # Need to use uninative's loader if enabled/present since the library paths
> # are used internally by rust and result in symbol mismatches if we don't
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#199464): https://lists.openembedded.org/g/openembedded-core/message/199464
> Mute This Topic: https://lists.openembedded.org/mt/106132432/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [OE-core] [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0
2024-05-18 21:55 ` [OE-core] [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0 Alexandre Belloni
@ 2024-05-19 0:27 ` Alexander Kanavin
2024-05-19 0:28 ` Alexander Kanavin
0 siblings, 1 reply; 7+ messages in thread
From: Alexander Kanavin @ 2024-05-19 0:27 UTC (permalink / raw)
To: Alexandre Belloni
Cc: Alexander Kanavin, Randy MacLeod, Richard Purdie,
openembedded-core
[-- Attachment #1: Type: text/plain, Size: 7141 bytes --]
This is intentional and known. The expectation is that fixes by maintainers
of the test will come shortly.
Alex
On Sat 18. May 2024 at 23.55, Alexandre Belloni <
alexandre.belloni@bootlin.com> wrote:
> ello,
>
> The series broke the -tc builds as
> rust.RustSelfTestSystemEmulated.test_rust is failing:
>
>
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/151/builds/1723/steps/12/logs/stdio
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/154/builds/1715/steps/12/logs/stdio
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/1711/steps/12/logs/stdio
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/145/builds/1700/steps/12/logs/stdio
>
> On 16/05/2024 13:21:08+0200, Alexander Kanavin wrote:
> > From: Alexander Kanavin <alex@linutronix.de>
> >
> > This does not seem to be used in regular builds, but is beneficial
> > in rust selftest, where it allows dropping a custom patch
> > that is unsuitable for upstream (and was rejected by them).
> >
> > Also remove an obsolete comment that seems related to the code
> > but describes something that was resolved long time ago.
> >
> > I have confirmed that the rust selftest continues to pass with just
> > this one commit on top of master (as the following changes do break
> > the selftest).
> >
> > Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> > ---
> > .../rust/files/cargo-path.patch | 37 -------------------
> > meta/recipes-devtools/rust/rust-source.inc | 1 -
> > meta/recipes-devtools/rust/rust_1.75.0.bb | 6 +--
> > 3 files changed, 2 insertions(+), 42 deletions(-)
> > delete mode 100644 meta/recipes-devtools/rust/files/cargo-path.patch
> >
> > diff --git a/meta/recipes-devtools/rust/files/cargo-path.patch
> b/meta/recipes-devtools/rust/files/cargo-path.patch
> > deleted file mode 100644
> > index 9a50c402201..00000000000
> > --- a/meta/recipes-devtools/rust/files/cargo-path.patch
> > +++ /dev/null
> > @@ -1,37 +0,0 @@
> > -Fix the cargo binary path error and ensure that it is fetched
> > -during rustc bootstrap in rust oe-selftest.
> > -
> > -======================================================================
> > -ERROR: test_cargoflags (bootstrap_test.BuildBootstrap)
> > -----------------------------------------------------------------------
> > -Traceback (most recent call last):
> > - File
> "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap_test.py",
> line 157, in test_cargoflags
> > - args, _ = self.build_args(env={"CARGOFLAGS": "--timings"})
> > - File
> "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap_test.py",
> line 154, in build_args
> > - return build.build_bootstrap_cmd(env), env
> > - File
> "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap.py",
> line 960, in build_bootstrap_cmd
> > - raise Exception("no cargo executable found at `{}`".format(
> > -Exception: no cargo executable found at
> `/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo`
> > -
> > -Upstream-Status: Submitted [
> https://github.com/rust-lang/rust/pull/120125]
> > -
> > -Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
> > ----
> > -diff --git a/src/bootstrap/bootstrap.py b/src/bootstrap/bootstrap.py
> > ---- a/src/bootstrap/bootstrap.py
> > -+++ b/src/bootstrap/bootstrap.py
> > -@@ -954,9 +954,11 @@
> > - if "RUSTFLAGS_BOOTSTRAP" in env:
> > - env["RUSTFLAGS"] += " " + env["RUSTFLAGS_BOOTSTRAP"]
> > -
> > -- env["PATH"] = os.path.join(self.bin_root(), "bin") + \
> > -- os.pathsep + env["PATH"]
> > -- if not os.path.isfile(self.cargo()):
> > -+ cargo_bin_path = os.path.join(self.bin_root(), "bin", "cargo")
> > -+ if not os.path.isfile(cargo_bin_path):
> > -+ cargo_bin_path = os.getenv("RUST_TARGET_PATH") +
> "rust-snapshot/bin/cargo"
> > -+ env["PATH"] = os.path.dirname(cargo_bin_path) + os.pathsep
> + env["PATH"]
> > -+ else:
> > - raise Exception("no cargo executable found at `{}`".format(
> > - self.cargo()))
> > - args = [self.cargo(), "build", "--manifest-path",
> > diff --git a/meta/recipes-devtools/rust/rust-source.inc
> b/meta/recipes-devtools/rust/rust-source.inc
> > index b14221b6cb8..c83c8ec3a39 100644
> > --- a/meta/recipes-devtools/rust/rust-source.inc
> > +++ b/meta/recipes-devtools/rust/rust-source.inc
> > @@ -7,7 +7,6 @@ SRC_URI += "
> https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
> > file://rv32-missing-syscalls.patch;patchdir=${RUSTSRC} \
> > file://rv32-rustix-libc-backend.patch;patchdir=${RUSTSRC} \
> >
> file://rv32-cargo-rustix-0.38.19-fix.patch;patchdir=${RUSTSRC} \
> > - file://cargo-path.patch;patchdir=${RUSTSRC} \
> > file://custom-target-cfg.patch;patchdir=${RUSTSRC} \
> > file://rustc-bootstrap.patch;patchdir=${RUSTSRC} \
> > file://target-build-value.patch;patchdir=${RUSTSRC} \
> > diff --git a/meta/recipes-devtools/rust/rust_1.75.0.bb
> b/meta/recipes-devtools/rust/rust_1.75.0.bb
> > index 76e1fe2d84a..8ef838ee90c 100644
> > --- a/meta/recipes-devtools/rust/rust_1.75.0.bb
> > +++ b/meta/recipes-devtools/rust/rust_1.75.0.bb
> > @@ -35,8 +35,6 @@ RUST_ALTERNATE_EXE_PATH_NATIVE =
> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-co
> > # own vendoring.
> > CARGO_DISABLE_BITBAKE_VENDORING = "1"
> >
> > -# We can't use RUST_BUILD_SYS here because that may be "musl" if
> > -# TCLIBC="musl". Snapshots are always -unknown-linux-gnu
> > setup_cargo_environment () {
> > # The first step is to build bootstrap and some early stage tools,
> > # these are build for the same target as the snapshot, e.g.
> > @@ -54,8 +52,8 @@ do_rust_setup_snapshot () {
> >
> > # Some versions of rust (e.g. 1.18.0) tries to find cargo in
> stage0/bin/cargo
> > # and fail without it there.
> > - mkdir -p ${RUSTSRC}/build/${BUILD_SYS}
> > - ln -sf ${WORKDIR}/rust-snapshot/
> ${RUSTSRC}/build/${BUILD_SYS}/stage0
> > + mkdir -p ${RUSTSRC}/build/${RUST_BUILD_SYS}
> > + ln -sf ${WORKDIR}/rust-snapshot/
> ${RUSTSRC}/build/${RUST_BUILD_SYS}/stage0
> >
> > # Need to use uninative's loader if enabled/present since the
> library paths
> > # are used internally by rust and result in symbol mismatches if we
> don't
> > --
> > 2.39.2
> >
>
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#199464):
> https://lists.openembedded.org/g/openembedded-core/message/199464
> > Mute This Topic: https://lists.openembedded.org/mt/106132432/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
[-- Attachment #2: Type: text/html, Size: 10570 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [OE-core] [PATCH 1/4] rust: correctly link rust-snapshot into build/stage0
2024-05-19 0:27 ` Alexander Kanavin
@ 2024-05-19 0:28 ` Alexander Kanavin
0 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2024-05-19 0:28 UTC (permalink / raw)
To: Alexandre Belloni
Cc: Alexander Kanavin, Randy MacLeod, Richard Purdie,
openembedded-core
[-- Attachment #1: Type: text/plain, Size: 7499 bytes --]
I can however resend with the text disabled so that existing patches can
stay in master-next.
Alex
On Sun 19. May 2024 at 2.27, Alexander Kanavin <alex.kanavin@gmail.com>
wrote:
> This is intentional and known. The expectation is that fixes by
> maintainers of the test will come shortly.
>
> Alex
>
> On Sat 18. May 2024 at 23.55, Alexandre Belloni <
> alexandre.belloni@bootlin.com> wrote:
>
>> ello,
>>
>> The series broke the -tc builds as
>> rust.RustSelfTestSystemEmulated.test_rust is failing:
>>
>>
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/151/builds/1723/steps/12/logs/stdio
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/154/builds/1715/steps/12/logs/stdio
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/1711/steps/12/logs/stdio
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/145/builds/1700/steps/12/logs/stdio
>>
>> On 16/05/2024 13:21:08+0200, Alexander Kanavin wrote:
>> > From: Alexander Kanavin <alex@linutronix.de>
>> >
>> > This does not seem to be used in regular builds, but is beneficial
>> > in rust selftest, where it allows dropping a custom patch
>> > that is unsuitable for upstream (and was rejected by them).
>> >
>> > Also remove an obsolete comment that seems related to the code
>> > but describes something that was resolved long time ago.
>> >
>> > I have confirmed that the rust selftest continues to pass with just
>> > this one commit on top of master (as the following changes do break
>> > the selftest).
>> >
>> > Signed-off-by: Alexander Kanavin <alex@linutronix.de>
>> > ---
>> > .../rust/files/cargo-path.patch | 37 -------------------
>> > meta/recipes-devtools/rust/rust-source.inc | 1 -
>> > meta/recipes-devtools/rust/rust_1.75.0.bb | 6 +--
>> > 3 files changed, 2 insertions(+), 42 deletions(-)
>> > delete mode 100644 meta/recipes-devtools/rust/files/cargo-path.patch
>> >
>> > diff --git a/meta/recipes-devtools/rust/files/cargo-path.patch
>> b/meta/recipes-devtools/rust/files/cargo-path.patch
>> > deleted file mode 100644
>> > index 9a50c402201..00000000000
>> > --- a/meta/recipes-devtools/rust/files/cargo-path.patch
>> > +++ /dev/null
>> > @@ -1,37 +0,0 @@
>> > -Fix the cargo binary path error and ensure that it is fetched
>> > -during rustc bootstrap in rust oe-selftest.
>> > -
>> > -======================================================================
>> > -ERROR: test_cargoflags (bootstrap_test.BuildBootstrap)
>> > -----------------------------------------------------------------------
>> > -Traceback (most recent call last):
>> > - File
>> "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap_test.py",
>> line 157, in test_cargoflags
>> > - args, _ = self.build_args(env={"CARGOFLAGS": "--timings"})
>> > - File
>> "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap_test.py",
>> line 154, in build_args
>> > - return build.build_bootstrap_cmd(env), env
>> > - File
>> "/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/src/bootstrap/bootstrap.py",
>> line 960, in build_bootstrap_cmd
>> > - raise Exception("no cargo executable found at `{}`".format(
>> > -Exception: no cargo executable found at
>> `/home/build-st/tmp/work/cortexa57-poky-linux/rust/1.74.1/rustc-1.74.1-src/build/x86_64-unknown-linux-gnu/stage0/bin/cargo`
>> > -
>> > -Upstream-Status: Submitted [
>> https://github.com/rust-lang/rust/pull/120125]
>> > -
>> > -Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
>> > ----
>> > -diff --git a/src/bootstrap/bootstrap.py b/src/bootstrap/bootstrap.py
>> > ---- a/src/bootstrap/bootstrap.py
>> > -+++ b/src/bootstrap/bootstrap.py
>> > -@@ -954,9 +954,11 @@
>> > - if "RUSTFLAGS_BOOTSTRAP" in env:
>> > - env["RUSTFLAGS"] += " " + env["RUSTFLAGS_BOOTSTRAP"]
>> > -
>> > -- env["PATH"] = os.path.join(self.bin_root(), "bin") + \
>> > -- os.pathsep + env["PATH"]
>> > -- if not os.path.isfile(self.cargo()):
>> > -+ cargo_bin_path = os.path.join(self.bin_root(), "bin", "cargo")
>> > -+ if not os.path.isfile(cargo_bin_path):
>> > -+ cargo_bin_path = os.getenv("RUST_TARGET_PATH") +
>> "rust-snapshot/bin/cargo"
>> > -+ env["PATH"] = os.path.dirname(cargo_bin_path) +
>> os.pathsep + env["PATH"]
>> > -+ else:
>> > - raise Exception("no cargo executable found at
>> `{}`".format(
>> > - self.cargo()))
>> > - args = [self.cargo(), "build", "--manifest-path",
>> > diff --git a/meta/recipes-devtools/rust/rust-source.inc
>> b/meta/recipes-devtools/rust/rust-source.inc
>> > index b14221b6cb8..c83c8ec3a39 100644
>> > --- a/meta/recipes-devtools/rust/rust-source.inc
>> > +++ b/meta/recipes-devtools/rust/rust-source.inc
>> > @@ -7,7 +7,6 @@ SRC_URI += "
>> https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
>> > file://rv32-missing-syscalls.patch;patchdir=${RUSTSRC} \
>> > file://rv32-rustix-libc-backend.patch;patchdir=${RUSTSRC} \
>> >
>> file://rv32-cargo-rustix-0.38.19-fix.patch;patchdir=${RUSTSRC} \
>> > - file://cargo-path.patch;patchdir=${RUSTSRC} \
>> > file://custom-target-cfg.patch;patchdir=${RUSTSRC} \
>> > file://rustc-bootstrap.patch;patchdir=${RUSTSRC} \
>> > file://target-build-value.patch;patchdir=${RUSTSRC} \
>> > diff --git a/meta/recipes-devtools/rust/rust_1.75.0.bb
>> b/meta/recipes-devtools/rust/rust_1.75.0.bb
>> > index 76e1fe2d84a..8ef838ee90c 100644
>> > --- a/meta/recipes-devtools/rust/rust_1.75.0.bb
>> > +++ b/meta/recipes-devtools/rust/rust_1.75.0.bb
>> > @@ -35,8 +35,6 @@ RUST_ALTERNATE_EXE_PATH_NATIVE =
>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-co
>> > # own vendoring.
>> > CARGO_DISABLE_BITBAKE_VENDORING = "1"
>> >
>> > -# We can't use RUST_BUILD_SYS here because that may be "musl" if
>> > -# TCLIBC="musl". Snapshots are always -unknown-linux-gnu
>> > setup_cargo_environment () {
>> > # The first step is to build bootstrap and some early stage tools,
>> > # these are build for the same target as the snapshot, e.g.
>> > @@ -54,8 +52,8 @@ do_rust_setup_snapshot () {
>> >
>> > # Some versions of rust (e.g. 1.18.0) tries to find cargo in
>> stage0/bin/cargo
>> > # and fail without it there.
>> > - mkdir -p ${RUSTSRC}/build/${BUILD_SYS}
>> > - ln -sf ${WORKDIR}/rust-snapshot/
>> ${RUSTSRC}/build/${BUILD_SYS}/stage0
>> > + mkdir -p ${RUSTSRC}/build/${RUST_BUILD_SYS}
>> > + ln -sf ${WORKDIR}/rust-snapshot/
>> ${RUSTSRC}/build/${RUST_BUILD_SYS}/stage0
>> >
>> > # Need to use uninative's loader if enabled/present since the
>> library paths
>> > # are used internally by rust and result in symbol mismatches if
>> we don't
>> > --
>> > 2.39.2
>> >
>>
>> >
>> > -=-=-=-=-=-=-=-=-=-=-=-
>> > Links: You receive all messages sent to this group.
>> > View/Reply Online (#199464):
>> https://lists.openembedded.org/g/openembedded-core/message/199464
>> > Mute This Topic: https://lists.openembedded.org/mt/106132432/3617179
>> > Group Owner: openembedded-core+owner@lists.openembedded.org
>> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>> alexandre.belloni@bootlin.com]
>> > -=-=-=-=-=-=-=-=-=-=-=-
>> >
>>
>>
>> --
>> Alexandre Belloni, co-owner and COO, Bootlin
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
>>
>
[-- Attachment #2: Type: text/html, Size: 11133 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread