Re: [OE-core] [PATCH V5 2/3] rust: Use llvm instead of rust-llvm

[Deepesh Varatharajan <deepesh.varatharajan@windriver.com>]

On 18-10-2025 22:41, Khem Raj wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Fri, Oct 17, 2025 at 11:42 PM Deepesh Varatharajan
> <deepesh.varatharajan@windriver.com> wrote:
>>
>> On 17-10-2025 11:31, Khem Raj wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> I am seeing a build regression when using clang as default compiler
>>>
>>> https://errors.yoctoproject.org/Errors/Details/885430/
>>>
>>> PREFERRED_TOOLCHAIN_TARGET ?= "clang"
>>> PREFERRED_TOOLCHAIN_SDK ?= "clang"
>>> TOOLCHAIN_NATIVE:pn-rust ?= "clang"
>>> TOOLCHAIN_NATIVE:pn-rust-native ?= "clang"
>>> TOOLCHAIN_NATIVE:pn-nativesdk-rust ?= "clang"
>> Hi Khem,
>>
>> WIth the above mentioned conf in local.conf, I tried building rust for
>> qemuarm and qemuarm64 quite
>> few times and the build completed successfully without any issues for
>> both native and target. Do I need
>> to add any other layers on top of poky ? Do you have any branch where I
>> can clone and reproduce the
>> issue ? Am I missing any step in reproducing the issue ?
>>
> Yeah, I missed a setting which I think could be the reason
>
> TC_CXX_RUNTIME = "llvm"
Hi Khem,

After adding the above in locl.conf, reproduced the issue. I've looked 
into the issue and
here’s what I found:

Currently, both rust-native and rust-target rely on the natively built 
llvm-config binary.
However, upon inspecting the runtime dependencies of llvm-config, I 
noticed the following:

$ ldd ./llvm-config
         linux-vdso.so.1 =>  (0x00007ffdec9eb000)
         libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
         libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
         libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
         libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6

As shown, llvm-config is linked against libstdc++. Meanwhile, we're 
enforcing Rust to be
built using libc++, and this mismatch is causing symbol resolution 
issues at runtime.

To address the ABI mismatch, I tried building llvm-native with libc++ by 
introducing a
dependency on libcxx-native. However, this leads to a circular dependency:
     >llvm-native depends on libcxx-native
     >libcxx-native depends on clang-native
     >clang-native depends back on llvm-native
This cycle blocks the build.

One possible solution could be to create a separate llvm-config recipe 
that builds with a
dependency on libcxx-native, and have Rust depend on that version 
specifically.

Do we have any alternative ways to break this circular dependency?

Regards,
Deepesh
>
>
>> Regards,
>> Deepesh
>>> On Wed, Oct 15, 2025 at 2:59 AM Varatharajan, Deepesh via
>>> lists.openembedded.org
>>> <deepesh.varatharajan=windriver.com@lists.openembedded.org> wrote:
>>>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>
>>>> Updated the Rust build to depend on llvm instead.
>>>>
>>>> *Summary of discussion with the rust upstream about using latest LLVM instead of Rust maintained LLVM fork.
>>>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>>>
>>>> *Upstream LLVM is generally compatible:
>>>> - Rust does support building with upstream (vanilla) LLVM, especially the latest
>>>> major release and the one or two preceding ones.
>>>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>>>
>>>> *Impact on Yocto Rust upgrades:
>>>> - Rust upgrades shall always check for updates on rust forked llvm and backport
>>>> the relevant patches to llvm.
>>>>
>>>> *Regarding the rust forked llvm local patches:
>>>> - There are no local patches on rust forked llvm other than the backported fixes
>>>> from llvm master.
>>>>
>>>> *We are copying the natively built `llvm-config` binary into the target sysroot and running
>>>> it. However, this `llvm-config` has compile time dependencies on various other arch's LLVM
>>>> libraries because native-llvm is built for all oe-core supported targets.
>>>>
>>>> Attempting to work around this by symlinking the missing libraries from the native sysroot
>>>> into the target sysroot leads to mixed architectures in the final `.rlib`. Specifically,
>>>> the object files extracted from those symlinked libraries within `librustc_llvm-<hash>.rlib`
>>>> are built for the host, while others are correctly built for the target This results in linker
>>>> failures due to file format not recognized.
>>>>
>>>> To resolve this, we now build llvm-target also for all oe-core supported architectures in
>>>> addition to the native-llvm build. This ensures that `llvm-config` and all associated
>>>> libraries are built for the correct target, eliminating cross-architecture contamination
>>>> and linker issues.
>>>>
>>>> *We are enabling -DLLVM_INSTALL_UTILS=ON to ensure essential LLVM utilities like FileCheck
>>>> are available, as they are required by the Rust build.
>>>>
>>>> Without this, the build fails with an error as below:
>>>> | thread 'main' panicked at src/bootstrap/src/core/sanity.rs:315:21:
>>>> | FileCheck executable "poky/build/tmp/work/x86_64-linux/rust-native/1.90.0/recipe-sysroot
>>>> -native/usr/bin/FileCheck" does not exist
>>>>
>>>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of this following
>>>> diff otherwise we will get errors during link time.
>>>>
>>>> Setup in rust-llvm
>>>> -DLLVM_ENABLE_ZLIB=OFF \
>>>> -DLLVM_ENABLE_ZSTD=OFF \
>>>> -DLLVM_ENABLE_FFI=OFF \
>>>>
>>>> Setup in llvm
>>>> -DLLVM_ENABLE_FFI=ON \
>>>>
>>>> *When multilibs enabled:
>>>>
>>>> llvm-config expects static libraries to be located in the lib directory rather than
>>>> lib64. However, since we are copying the natively built llvm-config to target sysroot
>>>> and running it and llvm-config doesn't know anything about lib64 existence. To accommodate
>>>> this without breaking multilib behavior, we are creating a symlink from 'lib' to 'lib64'
>>>> directory.
>>>>
>>>> Previously, when we depended on rust-llvm, this worked because we specified:
>>>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>>>
>>>> With this setup, llvm-config was installed inside ${libdir}/llvm-rust, which included
>>>> its own bin and lib directories. Thus, llvm-config located in bin would correctly find
>>>> the libraries in the adjacent lib directory.
>>>>
>>>> Even when multilib was enabled or not, llvm-config would still look for libraries under
>>>> lib in this structure, so everything functioned as expected.
>>>>
>>>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>> ---
>>>>    meta/recipes-devtools/clang/llvm_git.bb   |  2 +-
>>>>    meta/recipes-devtools/rust/rust_1.90.0.bb | 39 ++++++++++++++++++++---
>>>>    2 files changed, 36 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/meta/recipes-devtools/clang/llvm_git.bb b/meta/recipes-devtools/clang/llvm_git.bb
>>>> index d2b060ff88..2f47af8d7a 100644
>>>> --- a/meta/recipes-devtools/clang/llvm_git.bb
>>>> +++ b/meta/recipes-devtools/clang/llvm_git.bb
>>>> @@ -27,7 +27,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>>>>    # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>>>>    LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES', 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>>>>    LLVM_TARGETS_TO_BUILD ?= "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb, d)};BPF;${LLVM_TARGETS_GPU}"
>>>>
>>>>    LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>>>>
>>>> @@ -37,6 +36,7 @@ HF[vardepvalue] = "${HF}"
>>>>
>>>>    EXTRA_OECMAKE += "-DCMAKE_BUILD_TYPE=MinSizeRel \
>>>>                      -DLLVM_ENABLE_BINDINGS=OFF \
>>>> +                  -DLLVM_INSTALL_UTILS=ON \
>>>>                      -DLLVM_ENABLE_FFI=ON \
>>>>                      -DLLVM_ENABLE_RTTI=ON \
>>>>                      -DLLVM_TARGETS_TO_BUILD='${LLVM_TARGETS_TO_BUILD}' \
>>>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> index 5d804c7398..0319d73b93 100644
>>>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
>>>>    inherit rust
>>>>    inherit cargo_common
>>>>
>>>> -DEPENDS += "rust-llvm"
>>>> +DEPENDS += "llvm"
>>>>    # native rust uses cargo/rustc from binary snapshots to bootstrap
>>>>    # but everything else should use our native builds
>>>>    DEPENDS:append:class-target = " cargo-native rust-native"
>>>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL', 'stable', '', '-${RUST_CHANNEL}', d)
>>>>
>>>>    export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>>>>
>>>> -RUST_ALTERNATE_EXE_PATH ?= "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
>>>> -RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
>>>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
>>>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>>>>
>>>>    # We don't want to use bitbakes vendoring because the rust sources do their
>>>>    # own vendoring.
>>>> @@ -188,6 +188,37 @@ python do_configure() {
>>>>        bb.build.exec_func("setup_cargo_environment", d)
>>>>    }
>>>>
>>>> +# llvm-config expects static libraries to be in the 'lib' directory rather than 'lib64' when
>>>> +# multilibs enabled. Since we are copying the natively built llvm-config into the target sysroot
>>>> +# and executing it there, it will default to searching in 'lib', as it is unaware of the 'lib64'
>>>> +# directory. To ensure llvm-config can locate the necessary libraries, create a symlink from 'lib'
>>>> +do_compile:append:class-target() {
>>>> +    # Ensure llvm-config can find static libraries in multilib setup
>>>> +    lib64_dir="${STAGING_DIR_TARGET}/usr/lib64"
>>>> +    lib_dir="${STAGING_DIR_TARGET}/usr/lib"
>>>> +
>>>> +    if [ -d "$lib64_dir" ]; then
>>>> +        # If lib does not exist, symlink it to lib64
>>>> +        if [ ! -e "$lib_dir" ]; then
>>>> +            ln -s lib64 "$lib_dir"
>>>> +        fi
>>>> +
>>>> +        # Only do per-file symlinking if lib is a real directory (not symlink)
>>>> +        if [ -d "$lib_dir" ] && [ ! -L "$lib_dir" ]; then
>>>> +            for lib64_file in "${lib64_dir}"/libLLVM*.a; do
>>>> +                if [ -e "$lib64_file" ]; then
>>>> +                    lib_name=$(basename "${lib64_file}")
>>>> +                    target_link="${lib_dir}/${lib_name}"
>>>> +
>>>> +                    if [ ! -e "${target_link}" ]; then
>>>> +                        ln -s "../lib64/${lib_name}" "${target_link}"
>>>> +                    fi
>>>> +                fi
>>>> +            done
>>>> +        fi
>>>> +    fi
>>>> +}
>>>> +
>>>>    rust_runx () {
>>>>        echo "COMPILE ${PN}" "$@"
>>>>
>>>> @@ -199,7 +230,7 @@ rust_runx () {
>>>>        unset CXXFLAGS
>>>>        unset CPPFLAGS
>>>>
>>>> -    export RUSTFLAGS="${RUST_DEBUG_REMAP}"
>>>> +    export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz -Clink-arg=-lzstd"
>>>>
>>>>        # Copy the natively built llvm-config into the target so we can run it. Horrible,
>>>>        # but works!
>>>> --
>>>> 2.49.0
>>>>
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#224888): https://lists.openembedded.org/g/openembedded-core/message/224888
>>>> Mute This Topic: https://lists.openembedded.org/mt/115768136/1997914
>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>