Re: [OE-core] [PATCH V4 1/2] rust: Use llvm instead of rust-llvm

["Mathieu Dubois-Briand" <mathieu.dubois-briand@bootlin.com>]

On Fri Oct 10, 2025 at 4:47 PM CEST, Deepesh via lists.openembedded.org Varatharajan wrote:
> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>
> Updated the Rust build to depend on llvm instead.
>
> *Summary of discussion with the rust upstream about using latest LLVM instead of Rust maintained LLVM fork.
> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>
> *Upstream LLVM is generally compatible:
> - Rust does support building with upstream (vanilla) LLVM, especially the latest
> major release and the one or two preceding ones.
> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>
> *Impact on Yocto Rust upgrades:
> - Rust upgrades shall always check for updates on rust forked llvm and backport
> the relevant patches to llvm.
>
> *Regarding the rust forked llvm local patches:
> - There are no local patches on rust forked llvm other than the backported fixes
> from llvm master.
>
> *We are copying the natively built `llvm-config` binary into the target sysroot and running
> it. However, this `llvm-config` has compile time dependencies on various other arch's LLVM
> libraries because native-llvm is built for all oe-core supported targets.
>
> Attempting to work around this by symlinking the missing libraries from the native sysroot
> into the target sysroot leads to mixed architectures in the final `.rlib`. Specifically,
> the object files extracted from those symlinked libraries within `librustc_llvm-<hash>.rlib`
> are built for the host, while others are correctly built for the target This results in linker
> failures due to file format not recognized.
>
> To resolve this, we now build llvm-target also for all oe-core supported architectures in
> addition to the native-llvm build. This ensures that `llvm-config` and all associated
> libraries are built for the correct target, eliminating cross-architecture contamination
> and linker issues.
>
> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of this following
> diff otherwise we will get errors during link time.
>
> Setup in rust-llvm
> -DLLVM_ENABLE_ZLIB=OFF \
> -DLLVM_ENABLE_ZSTD=OFF \
> -DLLVM_ENABLE_FFI=OFF \
>
> Setup in llvm
> -DLLVM_ENABLE_FFI=ON \
>
> *When multilibs enabled:
>
> llvm-config expects static libraries to be located in the lib directory rather than
> lib64. However, since we are copying the natively built llvm-config to target sysroot
> and running it and llvm-config doesn't know anything about lib64 existence. To accommodate
> this without breaking multilib behavior, we are creating a symlink from 'lib' to 'lib64'
> directory.
>
> Previously, when we depended on rust-llvm, this worked because we specified:
> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>
> With this setup, llvm-config was installed inside ${libdir}/llvm-rust, which included
> its own bin and lib directories. Thus, llvm-config located in bin would correctly find
> the libraries in the adjacent lib directory.
>
> Even when multilib was enabled or not, llvm-config would still look for libraries under
> lib in this structure, so everything functioned as expected.
>
> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> ---

Hi Deepesh,

Thanks for the new version.

I suspect this series is responsible of the following error on the
autobuilder:

AssertionError: Command 'export TARGET_VENDOR="-poky"; export PATH=/srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/work/cortexa57-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/python3-native:/srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/work/cortexa57-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin:/srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/work/cortexa57-poky-linux/rust/1.90.0/recipe-sysroot-native/usr/bin/aarch64-poky-linux:/srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/hosttools:$PATH; export RUST_TARGET_PATH=/srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/work/cortexa57-poky-linux/rust/1.90.0/rust-targets; export TEST_DEVICE_ADDR="192.168.7.2:12345"; cd /srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/work/cortexa57-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src; python3 src/bootstrap/bootstrap.py test  --exclude src/bootstrap  --exclude src/doc/rustc  --exclude src/doc/rustdoc  --exclude src/doc/unstable-book  --exclude src/etc/test-float-parse  --exclude src/librustdoc  --exclude src/rustdoc-json-types  --exclude src/tools/coverage-dump  --exclude src/tools/jsondoclint  --exclude src/tools/lint-docs  --exclude src/tools/replace-version-placeholder  --exclude src/tools/rust-analyzer  --exclude src/tools/rustdoc-themes  --exclude src/tools/rust-installer  --exclude src/tools/test-float-parse  --exclude src/tools/suggest-tests  --exclude src/tools/tidy  --exclude tests/assembly-llvm/asm/aarch64-outline-atomics.rs  --exclude tests/codegen-llvm/issues/issue-122805.rs  --exclude tests/codegen-llvm/thread-local.rs  --exclude tests/mir-opt/  --exclude tests/run-make  --exclude tests/run-make-fulldeps  --exclude tests/rustdoc  --exclude tests/rustdoc-json  --exclude tests/rustdoc-js-std  --exclude tests/ui/abi/stack-probes-lto.rs  --exclude tests/ui/abi/stack-probes.rs  --exclude tests/ui/codegen/mismatched-data-layouts.rs  --exclude tests/codegen-llvm/rust-abi-arch-specific-adjustment.rs  --exclude tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs  --exclude tests/ui/feature-gates/version_check.rs  --exclude tests/ui-fulldeps/  --exclude tests/ui/process/nofile-limit.rs  --exclude tidyselftest --no-doc --no-fail-fast --bless --target aarch64-poky-linux-gnu' returned non-zero exit status 1:
...
hread 'main' panicked at src/tools/remote-test-client/src/main.rs:351:5:
io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: test failed, to rerun pass `-p rustc_driver_impl --lib`
     Running unittests src/lib.rs (build/x86_64-unknown-linux-gnu/stage1-rustc/aarch64-poky-linux-gnu/release/deps/rustc_error_codes-e9a3d92a58a57e6f)
uploaded "/srv/pokybuild/yocto-worker/qemuarm64-tc/build/build-st-2164988/tmp/work/cortexa57-poky-linux/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/aarch64-poky-linux-gnu/release/deps/rustc_error_codes-e9a3d92a58a57e6f", waiting for result
...
2025-10-11 14:39:15,774 - oe-selftest - INFO - rust.RustSelfTestSystemEmulated.test_rust (subunit.RemotedTestCase)
2025-10-11 14:39:15,789 - oe-selftest - INFO -  ... FAIL

https://autobuilder.yoctoproject.org/valkyrie/#/builders/5/builds/2477
https://autobuilder.yoctoproject.org/valkyrie/#/builders/28/builds/2471
https://autobuilder.yoctoproject.org/valkyrie/#/builders/58/builds/571
https://autobuilder.yoctoproject.org/valkyrie/#/builders/66/builds/2490

Can you have a look at these errors, please?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com