Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm

[Deepesh Varatharajan <deepesh.varatharajan@windriver.com>]

On 08-10-2025 01:47, Khem Raj wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Tue, Oct 7, 2025 at 4:10 AM Deepesh Varatharajan
> <deepesh.varatharajan@windriver.com> wrote:
>>
>> On 02-10-2025 00:59, Khem Raj wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
>>> <deepesh.varatharajan@windriver.com> wrote:
>>>> On 26-09-2025 23:00, Khem Raj wrote:
>>>>> CAUTION: This email comes from a non Wind River email account!
>>>>> Do not click links or open attachments unless you recognize the sender
>>>>> and know the content is safe.
>>>>>
>>>>> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
>>>>> wrote:
>>>>>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>>>
>>>>>> Updated the Rust build to depend on Clang instead.
>>>>>>
>>>>>> *Summary of discussion with the rust upstream about using latest LLVM
>>>>>> instead of Rust maintained LLVM fork.
>>>>>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>>>>>
>>>>>>
>>>>>> *Upstream LLVM is generally compatible:
>>>>>> - Rust does support building with upstream (vanilla) LLVM, especially
>>>>>> the latest
>>>>>> major release and the one or two preceding ones.
>>>>>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>>>>>
>>>>>>
>>>>>> *Impact on Yocto Rust upgrades:
>>>>>> - Rust upgrades shall always check for updates on rust forked llvm
>>>>>> and backport
>>>>>> the relevant patches to clang's llvm.
>>>>>>
>>>>>> *Regarding the rust forked llvm local patches:
>>>>>> - There are no local patches on rust forked llvm other than the
>>>>>> backported fixes
>>>>>> from llvm master.
>>>>>>
>>>>>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
>>>>>> this following
>>>>>> diff otherwise we will get errors during link time.
>>>>>>
>>>>>> Setup in rust-llvm
>>>>>> -DLLVM_ENABLE_ZLIB=OFF \
>>>>>> -DLLVM_ENABLE_ZSTD=OFF \
>>>>>> -DLLVM_ENABLE_FFI=OFF \
>>>>>>
>>>>>> Setup in clang
>>>>>> -DLLVM_ENABLE_FFI=ON \
>>>>>> -DLLVM_ENABLE_ZSTD=ON \
>>>>>>
>>>>>> *When multilibs enabled:
>>>>>>
>>>>>> llvm-config expects static libraries to be located in the lib
>>>>>> directory rather than
>>>>>> lib64. However, since LLVM is built as a non-multilib component, the
>>>>>> lib directory
>>>>>> doesn't contain any library files. To accommodate this without
>>>>>> breaking multilib
>>>>>> behavior, we copy the required library files appropriately.
>>>>>>
>>>>>> Previously, when we depended on rust-llvm, this worked because we
>>>>>> specified:
>>>>>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>>>>>
>>>>>> With this setup, llvm-config was installed inside
>>>>>> ${libdir}/llvm-rust, which included
>>>>>> its own bin and lib directories. Thus, llvm-config located in bin
>>>>>> would correctly find
>>>>>> the libraries in the adjacent lib directory.
>>>>>>
>>>>>> Even when multilib was enabled or not, llvm-config would still look
>>>>>> for libraries under
>>>>>> lib in this structure, so everything functioned as expected.
>>>>>>
>>>>>> *Changes needs to be done when llvm splits from clang:
>>>>>> In rust recipe:
>>>>>> Update the dependency from:
>>>>>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>>>>>>
>>>>>> In llvm recipe:
>>>>>> Apply the same changes that were made in the Clang recipe, as those
>>>>>> configurations have now been moved to the LLVM recipe after the split.
>>>>>>
>>>>>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>>> ---
>>>>>>     meta/recipes-devtools/clang/clang_git.bb     |  4 ++--
>>>>>>     meta/recipes-devtools/clang/common-clang.inc |  6 +++---
>>>>>>     meta/recipes-devtools/rust/rust_1.90.0.bb    | 18 ++++++++++++++----
>>>>>>     3 files changed, 19 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
>>>>>> b/meta/recipes-devtools/clang/clang_git.bb
>>>>>> index 53bca1c24f..3e117b308b 100644
>>>>>> --- a/meta/recipes-devtools/clang/clang_git.bb
>>>>>> +++ b/meta/recipes-devtools/clang/clang_git.bb
>>>>>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>>>>>>     # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>>>>>>     LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
>>>>>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>>>>>>     LLVM_TARGETS_TO_BUILD ?=
>>>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>>>> compiler we build is building all architectures once because it then
>>>>> reuses same compiler for all cross compilers just by creating canonical
>>>>> symlinks, this change is not going to work.
>>>> We removed this line because, for the clang class-target build, LLVM was
>>>> only compiling
>>>> libraries for the target architecture and GPU targets and kept inside
>>>> the target
>>>> sysroot. Since we use the llvm-config built for the class-target during
>>>> the Rust target build,
>>>> it was failing due to the absence of static libraries for other
>>>> architectures in the target sysroot.
>>>> Here the target is x86 arch. So, it compiled libraries for x86 arch only
>>>> and we can see the error
>>>> for missing libraries for other archs as below:
>>>>
>>>> |   --- stderr
>>>> |   llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
>>>> |   llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
>>>> |   llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
>>>> |   llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
>>>> |   llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
>>>> |   llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
>>>> .
>>>> .
>>>> .
>>>> |   thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
>>>> |   command did not execute successfully:
>>>> "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
>>>>
>>>> "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
>>>> "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
>>>> "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
>>>> |   expected success, got: exit status: 1
>>>>
>>>> Maybe instead of removing. Can we change
>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>>> to
>>>> -LLVM_TARGETS_TO_BUILD:class-target ?=
>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
>>>> ?
>>> I think we don't want to build all the general arches for target, only
>>> the target arch and gpu arches
>>> is what is needed.  If we build all the backends, that will slow down
>>> things. But if target rust needs
>>> to support all the architectures to generate code for, then we don't
>>> have much choice. Can you see
>>> if we are ok to just target one architecture ?
>> The issue doesn't originate from Rust directly, it stems from the
>> following logic
>> in the Rust recipe, where the natively built llvm-config is copied into
>> the target sysroot:
>>
>>       # Copy the natively built llvm-config into the target so we can run
>> it. Horrible,
>>       # but works!
>>       if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
>> ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
>>           mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
>>           cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
>>           if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
>>               patchelf --set-rpath \$ORIGIN/../../../../../`basename
>> ${STAGING_DIR_NATIVE}`${libdir_native} ${RUST_ALTERNATE_EXE_PATH}
>>           else
>>               patchelf --remove-rpath ${RUST_ALTERNATE_EXE_PATH}
>>           fi
>>       fi
>>
>> Since llvm-config is being copied from ${STAGING_BINDIR_NATIVE} to
>> ${STAGING_BINDIR},
>> and the native Clang was built with the following configuration:
>> LLVM_TARGETS_TO_BUILD ?=
>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>
>> The copied llvm-config will expect the corresponding target libraries to
>> be present in the ${STAGING_LIBDIR}.
>>
>> To address this issue, as far as I can see we have three options.
>>
>> 1) Build clang-target also for all mentioned archs for clang-native.
>> 2) Copy all the required libraries from ${STAGING_LIBDIR_NATIVE} to
>> ${STAGING_LIBDIR}.
>> 3) Instead of copying llvm-config from ${STAGING_BINDIR_NATIVE} to
>> ${STAGING_BINDIR},
>>        directly make rust-target build dependent
>> on  ${STAGING_BINDIR_NATIVE}/llvm-config.
>>
>> Are there any better or cleaner alternatives to handle this situation?
>> Any suggestions would be greatly appreciated.
> I would prefer option 3 since these libs are inherently cross built,
> it perhaps does not matter if
> they come from native or cross built clang. But that would have been
> default before this copy dance
> was done. I might have forgotten now, if this was done specifically to address a
> case which we can testout now.
Making Rust depend directly on the native llvm-config is not feasible 
because librustc_llvm*.rlib
is built for the host architecture(x86-64) . While this works if the 
target is x86-64 or x86, for
other architectures we get the following error:
we encounter the following error:
ld: .../librustc_llvm-f50ec2a9934e24d2.rlib: error adding symbols: file 
format not recognized
collect2: error: ld returned 1 exit status

Therefore, copying the natively built llvm-config into the target 
sysroot and unsetting the RPATH
is currently the best approach, and that’s what we are doing. However, 
even after unsetting the
RPATH, the native llvm-config still references to static libraries for 
other architectures during
compile time. To fix this, those libraries must be present in the target 
sysroot.

 From what I see, the only feasible solution is to build LLVM target 
libraries for all supported
architectures. Copying these libraries from the native sysroot is not 
feasible because the native
sysroot contains libs for all OE-core targets, while the target sysroot 
contains libs only for its
specific target—copying would cause conflicts.

Below are the build timings and space consumption after splitting clang 
and llvm.

LLVM built for only target:
bitbake llvm
real    25m3.899s
user    0m6.904s
sys     0m0.673s

bitbake clang
real    28m28.827s
user    0m13.178s
sys     0m1.193s

../poky/build/tmp/work/x86-64-v3-poky-linux/clang/21.1.2 > du -sh .
5.4G
../poky/build/tmp/work/x86-64-v3-poky-linux/llvm/21.1.2 > du -sh .
5.6G

LLVM built for all Oe-core supported targets:
bitbake llvm
real    25m2.809s
user    0m7.017s
sys     0m0.679s

bitbake clang
real    31m7.457s
user    0m14.471s
sys     0m1.544s

../poky/build/tmp/work/x86-64-v3-poky-linux/clang/21.1.2 > du -sh .
5.6G
../poky/build/tmp/work/x86-64-v3-poky-linux/llvm/21.1.2 > du -sh .
6.0G
>> Regards,
>> Deepesh
>>
>>>>>>     LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>>>>>>
>>>>>> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
>>>>>>                       -DLLVM_ENABLE_PIC=ON \
>>>>>>                       -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
>>>>>>                       -DLLVM_BINDINGS_LIST='' \
>>>>>> +               -DLLVM_INSTALL_UTILS=ON \
>>>>> tabs vs spaces inconsistenty. Please fix it.
>>>> Sure.
>>>>>>                       -DLLVM_ENABLE_FFI=ON \
>>>>>>                       -DLLVM_ENABLE_ZSTD=ON \
>>>>>>                       -DFFI_INCLUDE_DIR=$(pkg-config
>>>>>> --variable=includedir libffi) \
>>>>>> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
>>>>>> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
>>>>>> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
>>>>>> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
>>>>>> -                  -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>>>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>>>> why is this change needed ?
>>>>>
>>>> I made this change to ensure that LLVM_TARGET_ARCH is set correctly to
>>>> the actual target architecture.
>>>> However, it appears that even without this change, the build completes
>>>> successfully for the target. I checked
>>>> and printed the value of HOST_ARCH and realized it was already set
>>>> correctly my mistake. This change is
>>>> not required.
>>> LLVM_TARGET_ARCH is different than the backend architecture which is
>>> denoted by LLVM_TARGETS_TO_BUILD
>>>
>>>>>> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
>>>>>>                       -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
>>>>>>                       -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
>>>>>> diff --git a/meta/recipes-devtools/clang/common-clang.inc
>>>>>> b/meta/recipes-devtools/clang/common-clang.inc
>>>>>> index bf3a63914a..c22e3c1b19 100644
>>>>>> --- a/meta/recipes-devtools/clang/common-clang.inc
>>>>>> +++ b/meta/recipes-devtools/clang/common-clang.inc
>>>>>> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
>>>>>>         elif re.match('aarch64$', a):                      return
>>>>>> 'AArch64'
>>>>>>         elif re.match('aarch64_be$', a):                   return
>>>>>> 'AArch64'
>>>>>>         elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
>>>>>> -    elif re.match('riscv32$', a):                      return 'RISCV'
>>>>>> -    elif re.match('riscv64$', a):                      return 'RISCV'
>>>>>> +    elif re.match('riscv32$', a):                      return 'riscv32'
>>>>>> +    elif re.match('riscv64$', a):                      return 'riscv64'
>>>>> This is representing LLVM backend name and not normal arch and clang
>>>>> uses RISCV for both rv64 and rv32
>>>>>
>>>> After this change were made,
>>>> -                  -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>> +                  -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>>> do_configure of clang failed
>>>> with the following error:
>>>>
>>>> | CMake Error at cmake/config-ix.cmake:585 (message):
>>>> |   Unknown architecture riscv
>>>>
>>>> Since this change is irrelevant
>>>> -                  -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>> +                  -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>>>
>>>> The below changes are also not needed.
>>>>     -    elif re.match('riscv32$', a):                      return 'RISCV'
>>>>     -    elif re.match('riscv64$', a):                      return 'RISCV'
>>>> +    elif re.match('riscv32$', a):                      return 'riscv32'
>>>> +    elif re.match('riscv64$', a):                      return 'riscv64'
>>>> -    elif re.match('loongarch64$', a):                  return 'LoongArch'
>>>> +    elif re.match('loongarch64$', a):                  return
>>>> 'loongarch64'
>>>>>>         elif re.match('p(pc|owerpc)(|64)', a):             return
>>>>>> 'PowerPC'
>>>>>> -    elif re.match('loongarch64$', a):                  return
>>>>>> 'LoongArch'
>>>>>> +    elif re.match('loongarch64$', a):                  return
>>>>>> 'loongarch64'
>>>>> same problem as above here
>>>>>
>>>>>>         else:
>>>>>>             bb.fatal("Unhandled architecture %s" % arch_val)
>>>>>>             return ""
>>>>>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> index 5d804c7398..c2cb8f8829 100644
>>>>>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
>>>>>> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
>>>>>>     inherit rust
>>>>>>     inherit cargo_common
>>>>>>
>>>>>> -DEPENDS += "rust-llvm"
>>>>>> +DEPENDS += "ninja-native clang"
>>>>> I think using 'llvm' instead of 'clang' here is more appropriate to
>>>>> represent the dependency, clang provides llvm as well, it also ensures
>>>>> that when we split llvm out of clang recipe then you do not need to
>>>>> change it.
>>>> Sure will do that.
>>>>>>     # native rust uses cargo/rustc from binary snapshots to bootstrap
>>>>>>     # but everything else should use our native builds
>>>>>>     DEPENDS:append:class-target = " cargo-native rust-native"
>>>>>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
>>>>>> 'stable', '', '-${RUST_CHANNEL}', d)
>>>>>>
>>>>>>     export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>>>>>>
>>>>>> -RUST_ALTERNATE_EXE_PATH ?=
>>>>>> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
>>>>>> -RUST_ALTERNATE_EXE_PATH_NATIVE =
>>>>>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
>>>>>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
>>>>>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>>>>>>
>>>>>>     # We don't want to use bitbakes vendoring because the rust sources
>>>>>> do their
>>>>>>     # own vendoring.
>>>>>> @@ -188,6 +188,16 @@ python do_configure() {
>>>>>>         bb.build.exec_func("setup_cargo_environment", d)
>>>>>>     }
>>>>>>
>>>>>> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
>>>>>> +#Since LLVM is built as a non-multilib component, the 'lib' directory
>>>>>> +#doesn't have any library files when multilibs enabled. So, copying
>>>>>> +#library files without impacting multilib behavior.
>>>>>> +do_compile:append:class-target() {
>>>>>> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
>>>>>> +    cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
>>>>>> ${STAGING_DIR_TARGET}/usr/lib/.
>>>>>> +fi
>>>>>> +}
>>>>>> +
>>>>>>     rust_runx () {
>>>>>>         echo "COMPILE ${PN}" "$@"
>>>>>>
>>>>>> @@ -199,7 +209,7 @@ rust_runx () {
>>>>>>         unset CXXFLAGS
>>>>>>         unset CPPFLAGS
>>>>>>
>>>>>> -    export RUSTFLAGS="${RUST_DEBUG_REMAP}"
>>>>>> +    export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
>>>>>> -Clink-arg=-lzstd"
>>>>>>
>>>>>>         # Copy the natively built llvm-config into the target so we can
>>>>>> run it. Horrible,
>>>>>>         # but works!
>>>>>>
>>>>>>
>>>>>>
>>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>> Links: You receive all messages sent to this group.
>>>>>> View/Reply Online (#224075):
>>>>>> https://lists.openembedded.org/g/openembedded-core/message/224075
>>>>>> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
>>>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>>>>>> [raj.khem@gmail.com]
>>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>>