* [PATCH V3 1/2] rust: Use clang instead of rust-llvm
@ 2025-09-26 10:24 Deepesh.Varatharajan
2025-09-26 10:24 ` [PATCH V3 2/2] rust: Drop rust-llvm Deepesh.Varatharajan
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Deepesh.Varatharajan @ 2025-09-26 10:24 UTC (permalink / raw)
To: openembedded-core; +Cc: Sundeep.Kokkonda, Deepesh.Varatharajan
From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Updated the Rust build to depend on Clang instead.
*Summary of discussion with the rust upstream about using latest LLVM instead of Rust maintained LLVM fork.
https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
*Upstream LLVM is generally compatible:
- Rust does support building with upstream (vanilla) LLVM, especially the latest
major release and the one or two preceding ones.
https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
*Impact on Yocto Rust upgrades:
- Rust upgrades shall always check for updates on rust forked llvm and backport
the relevant patches to clang's llvm.
*Regarding the rust forked llvm local patches:
- There are no local patches on rust forked llvm other than the backported fixes
from llvm master.
*We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of this following
diff otherwise we will get errors during link time.
Setup in rust-llvm
-DLLVM_ENABLE_ZLIB=OFF \
-DLLVM_ENABLE_ZSTD=OFF \
-DLLVM_ENABLE_FFI=OFF \
Setup in clang
-DLLVM_ENABLE_FFI=ON \
-DLLVM_ENABLE_ZSTD=ON \
*When multilibs enabled:
llvm-config expects static libraries to be located in the lib directory rather than
lib64. However, since LLVM is built as a non-multilib component, the lib directory
doesn't contain any library files. To accommodate this without breaking multilib
behavior, we copy the required library files appropriately.
Previously, when we depended on rust-llvm, this worked because we specified:
-DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
With this setup, llvm-config was installed inside ${libdir}/llvm-rust, which included
its own bin and lib directories. Thus, llvm-config located in bin would correctly find
the libraries in the adjacent lib directory.
Even when multilib was enabled or not, llvm-config would still look for libraries under
lib in this structure, so everything functioned as expected.
*Changes needs to be done when llvm splits from clang:
In rust recipe:
Update the dependency from:
DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
In llvm recipe:
Apply the same changes that were made in the Clang recipe, as those
configurations have now been moved to the LLVM recipe after the split.
Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
---
meta/recipes-devtools/clang/clang_git.bb | 4 ++--
meta/recipes-devtools/clang/common-clang.inc | 6 +++---
meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
3 files changed, 19 insertions(+), 9 deletions(-)
diff --git a/meta/recipes-devtools/clang/clang_git.bb b/meta/recipes-devtools/clang/clang_git.bb
index 53bca1c24f..3e117b308b 100644
--- a/meta/recipes-devtools/clang/clang_git.bb
+++ b/meta/recipes-devtools/clang/clang_git.bb
@@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
# https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES', 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
LLVM_TARGETS_TO_BUILD ?= "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
-LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb, d)};BPF;${LLVM_TARGETS_GPU}"
LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
@@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
-DLLVM_ENABLE_PIC=ON \
-DCLANG_DEFAULT_PIE_ON_LINUX=ON \
-DLLVM_BINDINGS_LIST='' \
+ -DLLVM_INSTALL_UTILS=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_ENABLE_ZSTD=ON \
-DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \
@@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
-DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
-DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
-DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
- -DLLVM_TARGET_ARCH=${HOST_ARCH} \
+ -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
-DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
-DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
-DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
diff --git a/meta/recipes-devtools/clang/common-clang.inc b/meta/recipes-devtools/clang/common-clang.inc
index bf3a63914a..c22e3c1b19 100644
--- a/meta/recipes-devtools/clang/common-clang.inc
+++ b/meta/recipes-devtools/clang/common-clang.inc
@@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
elif re.match('aarch64$', a): return 'AArch64'
elif re.match('aarch64_be$', a): return 'AArch64'
elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
- elif re.match('riscv32$', a): return 'RISCV'
- elif re.match('riscv64$', a): return 'RISCV'
+ elif re.match('riscv32$', a): return 'riscv32'
+ elif re.match('riscv64$', a): return 'riscv64'
elif re.match('p(pc|owerpc)(|64)', a): return 'PowerPC'
- elif re.match('loongarch64$', a): return 'LoongArch'
+ elif re.match('loongarch64$', a): return 'loongarch64'
else:
bb.fatal("Unhandled architecture %s" % arch_val)
return ""
diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb b/meta/recipes-devtools/rust/rust_1.90.0.bb
index 5d804c7398..c2cb8f8829 100644
--- a/meta/recipes-devtools/rust/rust_1.90.0.bb
+++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
inherit rust
inherit cargo_common
-DEPENDS += "rust-llvm"
+DEPENDS += "ninja-native clang"
# native rust uses cargo/rustc from binary snapshots to bootstrap
# but everything else should use our native builds
DEPENDS:append:class-target = " cargo-native rust-native"
@@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL', 'stable', '', '-${RUST_CHANNEL}', d)
export FORCE_CRATE_HASH = "${BB_TASKHASH}"
-RUST_ALTERNATE_EXE_PATH ?= "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
-RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
+RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
+RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
# We don't want to use bitbakes vendoring because the rust sources do their
# own vendoring.
@@ -188,6 +188,16 @@ python do_configure() {
bb.build.exec_func("setup_cargo_environment", d)
}
+#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
+#Since LLVM is built as a non-multilib component, the 'lib' directory
+#doesn't have any library files when multilibs enabled. So, copying
+#library files without impacting multilib behavior.
+do_compile:append:class-target() {
+if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
+ cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a ${STAGING_DIR_TARGET}/usr/lib/.
+fi
+}
+
rust_runx () {
echo "COMPILE ${PN}" "$@"
@@ -199,7 +209,7 @@ rust_runx () {
unset CXXFLAGS
unset CPPFLAGS
- export RUSTFLAGS="${RUST_DEBUG_REMAP}"
+ export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz -Clink-arg=-lzstd"
# Copy the natively built llvm-config into the target so we can run it. Horrible,
# but works!
--
2.49.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH V3 2/2] rust: Drop rust-llvm
2025-09-26 10:24 [PATCH V3 1/2] rust: Use clang instead of rust-llvm Deepesh.Varatharajan
@ 2025-09-26 10:24 ` Deepesh.Varatharajan
2025-09-26 16:50 ` [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm Khem Raj
` (2 subsequent siblings)
3 siblings, 0 replies; 14+ messages in thread
From: Deepesh.Varatharajan @ 2025-09-26 10:24 UTC (permalink / raw)
To: openembedded-core; +Cc: Sundeep.Kokkonda, Deepesh.Varatharajan
From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
Drop rust-llvm recipe because now rust depends on clang for building
*Dropped Patches:
0001-libunwind-Use-gcs-instead-of-gcs-target-attribute.patch(Merged)
https://github.com/llvm/llvm-project/commit/b46f34452e9dec50eee6ddbe07875f05e421a81c
0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch
0002-llvm-allow-env-override-of-exe-path.patch
(We have the above two patches in clang)
Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
---
meta/conf/distro/include/maintainers.inc | 1 -
...-gcs-instead-of-gcs-target-attribute.patch | 66 -------------
...r-sort-ClassInfo-lists-by-name-as-we.patch | 31 ------
...-llvm-allow-env-override-of-exe-path.patch | 33 -------
.../recipes-devtools/rust/rust-llvm_1.90.0.bb | 98 -------------------
meta/recipes-devtools/rust/rust-source.inc | 1 -
6 files changed, 230 deletions(-)
delete mode 100644 meta/recipes-devtools/rust/files/0001-libunwind-Use-gcs-instead-of-gcs-target-attribute.patch
delete mode 100644 meta/recipes-devtools/rust/rust-llvm/0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch
delete mode 100644 meta/recipes-devtools/rust/rust-llvm/0002-llvm-allow-env-override-of-exe-path.patch
delete mode 100644 meta/recipes-devtools/rust/rust-llvm_1.90.0.bb
diff --git a/meta/conf/distro/include/maintainers.inc b/meta/conf/distro/include/maintainers.inc
index 0988bf50a7..7eec664619 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -773,7 +773,6 @@ RECIPE_MAINTAINER:pn-ruby = "Ross Burton <ross.burton@arm.com>"
RECIPE_MAINTAINER:pn-run-postinsts = "Ross Burton <ross.burton@arm.com>"
RECIPE_MAINTAINER:pn-rust = "Randy MacLeod <Randy.MacLeod@windriver.com>"
RECIPE_MAINTAINER:pn-rust-cross-canadian-${TRANSLATED_TARGET_ARCH} = "Randy MacLeod <Randy.MacLeod@windriver.com>"
-RECIPE_MAINTAINER:pn-rust-llvm = "Randy MacLeod <Randy.MacLeod@windriver.com>"
RECIPE_MAINTAINER:pn-rxvt-unicode = "Unassigned <unassigned@yoctoproject.org>"
RECIPE_MAINTAINER:pn-sassc = "Simone Weiß <simone.p.weiss@posteo.com>"
RECIPE_MAINTAINER:pn-sato-screenshot = "Ross Burton <ross.burton@arm.com>"
diff --git a/meta/recipes-devtools/rust/files/0001-libunwind-Use-gcs-instead-of-gcs-target-attribute.patch b/meta/recipes-devtools/rust/files/0001-libunwind-Use-gcs-instead-of-gcs-target-attribute.patch
deleted file mode 100644
index a704671fb9..0000000000
--- a/meta/recipes-devtools/rust/files/0001-libunwind-Use-gcs-instead-of-gcs-target-attribute.patch
+++ /dev/null
@@ -1,66 +0,0 @@
-From e4f4094de8ddcbe6d5ff1cdf782d2b89e0563903 Mon Sep 17 00:00:00 2001
-From: Khem Raj <raj.khem@gmail.com>
-Date: Wed, 30 Apr 2025 19:51:19 -0700
-Subject: [PATCH] libunwind: Use +gcs instead of gcs target attribute
-
-__attribute__((target("gcs"))) does not work with gcc
-
-GCC-15 has added gcs intrinsics [1] but the syntax for enabling it is
-slightly different. This syntax works with clang too.
-
-With gcc15 compiler libunwind's check for this macros is succeeding and it
-ends up enabling 'gcs' by using function attribute, this works with clang
-but not with gcc but '+gcs' works with both
-
-We can see this in rust compiler bootstrap for aarch64/musl when system
-uses gcc15, it ends up with these errors
-
-Building libunwind.a for aarch64-poky-linux-musl
-cargo:warning=/mnt/b/yoe/master/sources/poky/build/tmp/work/cortexa57-poky-linux-musl/rust/1.85.1/rustc-1.85.1-src/src/llvm-project/libunwind/src/UnwindLevel1.c:191:1: error: arch extension 'gcs' should be prefixed by '+'
-cargo:warning= 191 | unwind_phase2(unw_context_t *uc, unw_cursor_t *cursor, _Unwind_Exception *exception_object) {
-cargo:warning= | ^~~~~~~~~~~~~
-cargo:warning=/mnt/b/yoe/master/sources/poky/build/tmp/work/cortexa57-poky-linux-musl/rust/1.85.1/rustc-1.85.1-src/src/llvm-project/libunwind/src/UnwindLevel1.c:337:22: error: arch extension 'gcs' should be prefixed by '+'
-cargo:warning= 337 | _Unwind_Stop_Fn stop, void *stop_parameter) {
-cargo:warning= | ^~~~~~~~~~~~~~~
-
-[1] https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=5a6af707f0af
-
-Upstream-Status: Submitted [https://github.com/llvm/llvm-project/pull/138077]
-
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
----
- src/llvm-project/libunwind/src/UnwindLevel1.c | 4 ++--
- 1 file changed, 2 insertions(+), 2 deletions(-)
-
---- a/src/llvm-project/libunwind/src/UnwindLevel1.c
-+++ b/src/llvm-project/libunwind/src/UnwindLevel1.c
-@@ -185,7 +185,7 @@ extern int __unw_step_stage2(unw_cursor_
-
- #if defined(_LIBUNWIND_USE_GCS)
- // Enable the GCS target feature to permit gcspop instructions to be used.
--__attribute__((target("gcs")))
-+__attribute__((target("+gcs")))
- #endif
- static _Unwind_Reason_Code
- unwind_phase2(unw_context_t *uc, unw_cursor_t *cursor, _Unwind_Exception *exception_object) {
-@@ -329,7 +329,7 @@ unwind_phase2(unw_context_t *uc, unw_cur
-
- #if defined(_LIBUNWIND_USE_GCS)
- // Enable the GCS target feature to permit gcspop instructions to be used.
--__attribute__((target("gcs")))
-+__attribute__((target("+gcs")))
- #endif
- static _Unwind_Reason_Code
- unwind_phase2_forced(unw_context_t *uc, unw_cursor_t *cursor,
---- a/src/llvm-project/libunwind/src/cet_unwind.h
-+++ b/src/llvm-project/libunwind/src/cet_unwind.h
-@@ -42,7 +42,8 @@
- #include <arm_acle.h>
-
- // We can only use GCS if arm_acle.h defines the GCS intrinsics.
--#ifdef _CHKFEAT_GCS
-+// Enable gcs with clang for now, gcc does not build unwindlevel1.c correctly
-+#if defined(_CHKFEAT_GCS) && defined(__clang__)
- #define _LIBUNWIND_USE_GCS 1
- #endif
-
diff --git a/meta/recipes-devtools/rust/rust-llvm/0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch b/meta/recipes-devtools/rust/rust-llvm/0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch
deleted file mode 100644
index 48af6fc283..0000000000
--- a/meta/recipes-devtools/rust/rust-llvm/0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch
+++ /dev/null
@@ -1,31 +0,0 @@
-From 86940d87026432683fb6741cd8a34d3b9b18e40d Mon Sep 17 00:00:00 2001
-From: Alexander Kanavin <alex.kanavin@gmail.com>
-Date: Fri, 27 Nov 2020 10:11:08 +0000
-Subject: [PATCH] AsmMatcherEmitter: sort ClassInfo lists by name as well
-
-Otherwise, there are instances which are identical in
-every other field and therefore sort non-reproducibly
-(which breaks binary and source reproducibiliy).
-
-Upstream-Status: Submitted [https://reviews.llvm.org/D97477]
-Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
----
- llvm/utils/TableGen/AsmMatcherEmitter.cpp | 5 ++++-
- 1 file changed, 4 insertions(+), 1 deletion(-)
-
-diff --git a/llvm/utils/TableGen/AsmMatcherEmitter.cpp b/llvm/utils/TableGen/AsmMatcherEmitter.cpp
-index ccf0959389b..1f801e83b7d 100644
---- a/llvm/utils/TableGen/AsmMatcherEmitter.cpp
-+++ b/llvm/utils/TableGen/AsmMatcherEmitter.cpp
-@@ -359,7 +359,10 @@ public:
- // name of a class shouldn't be significant. However, some of the backends
- // accidentally rely on this behaviour, so it will have to stay like this
- // until they are fixed.
-- return ValueName < RHS.ValueName;
-+ if (ValueName != RHS.ValueName)
-+ return ValueName < RHS.ValueName;
-+ // All else being equal, we should sort by name, for source and binary reproducibility
-+ return Name < RHS.Name;
- }
- };
-
diff --git a/meta/recipes-devtools/rust/rust-llvm/0002-llvm-allow-env-override-of-exe-path.patch b/meta/recipes-devtools/rust/rust-llvm/0002-llvm-allow-env-override-of-exe-path.patch
deleted file mode 100644
index 9be26677a9..0000000000
--- a/meta/recipes-devtools/rust/rust-llvm/0002-llvm-allow-env-override-of-exe-path.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 7111770e8290082530d920e120995bf81431b0aa Mon Sep 17 00:00:00 2001
-From: Martin Kelly <mkelly@xevo.com>
-Date: Fri, 19 May 2017 00:22:57 -0700
-Subject: [PATCH 12/18] llvm: allow env override of exe path
-
-When using a native llvm-config from inside a sysroot, we need llvm-config to
-return the libraries, include directories, etc. from inside the sysroot rather
-than from the native sysroot. Thus provide an env override for calling
-llvm-config from a target sysroot.
-
-Upstream-Status: Inappropriate [oe-core specific]
-Signed-off-by: Martin Kelly <mkelly@xevo.com>
-Signed-off-by: Khem Raj <raj.khem@gmail.com>
----
- llvm/tools/llvm-config/llvm-config.cpp | 7 +++++++
- 1 file changed, 7 insertions(+)
-
---- a/llvm/tools/llvm-config/llvm-config.cpp
-+++ b/llvm/tools/llvm-config/llvm-config.cpp
-@@ -226,6 +226,13 @@ Typical components:\n\
-
- /// Compute the path to the main executable.
- std::string GetExecutablePath(const char *Argv0) {
-+ // Hack for Yocto: we need to override the root path when we are using
-+ // llvm-config from within a target sysroot.
-+ const char *Sysroot = std::getenv("YOCTO_ALTERNATE_EXE_PATH");
-+ if (Sysroot != nullptr) {
-+ return Sysroot;
-+ }
-+
- // This just needs to be some symbol in the binary; C++ doesn't
- // allow taking the address of ::main however.
- void *P = (void *)(intptr_t)GetExecutablePath;
diff --git a/meta/recipes-devtools/rust/rust-llvm_1.90.0.bb b/meta/recipes-devtools/rust/rust-llvm_1.90.0.bb
deleted file mode 100644
index 42d4031da1..0000000000
--- a/meta/recipes-devtools/rust/rust-llvm_1.90.0.bb
+++ /dev/null
@@ -1,98 +0,0 @@
-SUMMARY = "LLVM compiler framework (packaged with rust)"
-LICENSE ?= "Apache-2.0-with-LLVM-exception"
-HOMEPAGE = "http://www.rust-lang.org"
-
-# check src/llvm-project/llvm/CMakeLists.txt for llvm version in use
-#
-LLVM_RELEASE = "20.1.8"
-
-require rust-source.inc
-
-SRC_URI += "file://0002-llvm-allow-env-override-of-exe-path.patch;striplevel=2 \
- file://0001-AsmMatcherEmitter-sort-ClassInfo-lists-by-name-as-we.patch;striplevel=2 \
- "
-
-S = "${RUSTSRC}/src/llvm-project/llvm"
-
-LIC_FILES_CHKSUM = "file://LICENSE.TXT;md5=8a15a0759ef07f2682d2ba4b893c9afe"
-
-inherit cmake
-
-DEPENDS += "ninja-native rust-llvm-native"
-
-ARM_INSTRUCTION_SET:armv5 = "arm"
-ARM_INSTRUCTION_SET:armv4t = "arm"
-
-# rustc_llvm with debug info is not recognized as a valid crate that's
-# generated by rust-llvm-native.
-CFLAGS:remove = "-g"
-CXXFLAGS:remove = "-g"
-
-LLVM_DIR = "llvm${LLVM_RELEASE}"
-
-RUST_LLVM_TARGETS ?= "ARM;AArch64;Mips;PowerPC;RISCV;X86"
-
-EXTRA_OECMAKE = " \
- -DCMAKE_BUILD_TYPE=Release \
- -DLLVM_TARGETS_TO_BUILD='${RUST_LLVM_TARGETS}' \
- -DLLVM_BUILD_DOCS=OFF \
- -DLLVM_ENABLE_TERMINFO=OFF \
- -DLLVM_ENABLE_ZLIB=OFF \
- -DLLVM_ENABLE_ZSTD=OFF \
- -DLLVM_ENABLE_LIBXML2=OFF \
- -DLLVM_ENABLE_FFI=OFF \
- -DLLVM_ENABLE_LIBEDIT=OFF \
- -DLLVM_INSTALL_UTILS=ON \
- -DLLVM_BUILD_EXAMPLES=OFF \
- -DLLVM_BUILD_LLVM_DYLIB=ON \
- -DLLVM_LINK_LLVM_DYLIB=ON \
- -DLLVM_INCLUDE_EXAMPLES=OFF \
- -DLLVM_BUILD_TESTS=OFF \
- -DLLVM_INCLUDE_TESTS=OFF \
- -DLLVM_INCLUDE_BENCHMARKS=OFF \
- -DLLVM_TARGET_ARCH=${TARGET_ARCH} \
- -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust \
-"
-
-# Forcibly disable the detection of these packages as otherwise
-# it will look at the host Python install
-EXTRA_OECMAKE += "\
- -DPY_PYGMENTS_FOUND=OFF \
- -DPY_PYGMENTS_LEXERS_C_CPP_FOUND=OFF \
- -DPY_YAML_FOUND=OFF \
-"
-
-EXTRA_OECMAKE:append:class-target = "\
- -DLLVM_BUILD_TOOLS=OFF \
- -DLLVM_TABLEGEN=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-tblgen \
- -DLLVM_CONFIG_PATH=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config \
-"
-
-EXTRA_OECMAKE:append:class-nativesdk = "\
- -DLLVM_BUILD_TOOLS=OFF \
- -DLLVM_TABLEGEN=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-tblgen \
- -DLLVM_CONFIG_PATH=${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config \
-"
-
-# The debug symbols are huge here (>2GB) so suppress them since they
-# provide almost no value. If you really need them then override this
-INHIBIT_PACKAGE_DEBUG_SPLIT = "1"
-
-export YOCTO_ALTERNATE_EXE_PATH = "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
-
-do_install:append () {
- # we don't need any of this stuff to build Rust
- rm -rf "${D}/usr/lib/cmake"
-}
-
-PACKAGES =+ "${PN}-bugpointpasses ${PN}-llvmhello ${PN}-liblto"
-
-# Add the extra locations to avoid the complaints about unpackaged files
-FILES:${PN}-bugpointpasses = "${libdir}/llvm-rust/lib/BugpointPasses.so"
-FILES:${PN}-llvmhello = "${libdir}/llvm-rust/lib/LLVMHello.so"
-FILES:${PN}-liblto = "${libdir}/llvm-rust/lib/libLTO.so.*"
-FILES:${PN}-staticdev =+ "${libdir}/llvm-rust/*/*.a"
-FILES:${PN} += "${libdir}/libLLVM*.so.* ${libdir}/llvm-rust/lib/*.so.* ${libdir}/llvm-rust/bin"
-FILES:${PN}-dev += "${datadir}/llvm ${libdir}/llvm-rust/lib/*.so ${libdir}/llvm-rust/include ${libdir}/llvm-rust/share ${libdir}/llvm-rust/lib/cmake"
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc
index cf8274b443..ac814c67b6 100644
--- a/meta/recipes-devtools/rust/rust-source.inc
+++ b/meta/recipes-devtools/rust/rust-source.inc
@@ -7,7 +7,6 @@ SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;n
file://oeqa-selftest-Increase-timeout-in-process-sigpipe-ru.patch;patchdir=${RUSTSRC} \
file://0001-src-core-build_steps-tool.rs-switch-off-lto-for-rust.patch;patchdir=${RUSTSRC} \
file://revert-link-std-statically-in-rustc_driver-feature.patch;patchdir=${RUSTSRC} \
- file://0001-libunwind-Use-gcs-instead-of-gcs-target-attribute.patch;patchdir=${RUSTSRC} \
file://0001-riscv32-Define-plain-syscalls-as-their-time64-varian.patch;patchdir=${RUSTSRC} \
"
SRC_URI[rust.sha256sum] = "6bfeaddd90ffda2f063492b092bfed925c4b8c701579baf4b1316e021470daac"
--
2.49.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-09-26 10:24 [PATCH V3 1/2] rust: Use clang instead of rust-llvm Deepesh.Varatharajan
2025-09-26 10:24 ` [PATCH V3 2/2] rust: Drop rust-llvm Deepesh.Varatharajan
@ 2025-09-26 16:50 ` Khem Raj
2025-09-29 10:17 ` Deepesh Varatharajan
2025-09-26 17:30 ` Khem Raj
2025-09-29 8:11 ` Mathieu Dubois-Briand
3 siblings, 1 reply; 14+ messages in thread
From: Khem Raj @ 2025-09-26 16:50 UTC (permalink / raw)
To: deepesh.varatharajan, openembedded-core; +Cc: Sundeep.Kokkonda
On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org wrote:
> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>
> Updated the Rust build to depend on Clang instead.
>
> *Summary of discussion with the rust upstream about using latest LLVM instead of Rust maintained LLVM fork.
> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>
> *Upstream LLVM is generally compatible:
> - Rust does support building with upstream (vanilla) LLVM, especially the latest
> major release and the one or two preceding ones.
> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>
> *Impact on Yocto Rust upgrades:
> - Rust upgrades shall always check for updates on rust forked llvm and backport
> the relevant patches to clang's llvm.
>
> *Regarding the rust forked llvm local patches:
> - There are no local patches on rust forked llvm other than the backported fixes
> from llvm master.
>
> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of this following
> diff otherwise we will get errors during link time.
>
> Setup in rust-llvm
> -DLLVM_ENABLE_ZLIB=OFF \
> -DLLVM_ENABLE_ZSTD=OFF \
> -DLLVM_ENABLE_FFI=OFF \
>
> Setup in clang
> -DLLVM_ENABLE_FFI=ON \
> -DLLVM_ENABLE_ZSTD=ON \
>
> *When multilibs enabled:
>
> llvm-config expects static libraries to be located in the lib directory rather than
> lib64. However, since LLVM is built as a non-multilib component, the lib directory
> doesn't contain any library files. To accommodate this without breaking multilib
> behavior, we copy the required library files appropriately.
>
> Previously, when we depended on rust-llvm, this worked because we specified:
> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>
> With this setup, llvm-config was installed inside ${libdir}/llvm-rust, which included
> its own bin and lib directories. Thus, llvm-config located in bin would correctly find
> the libraries in the adjacent lib directory.
>
> Even when multilib was enabled or not, llvm-config would still look for libraries under
> lib in this structure, so everything functioned as expected.
>
> *Changes needs to be done when llvm splits from clang:
> In rust recipe:
> Update the dependency from:
> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>
> In llvm recipe:
> Apply the same changes that were made in the Clang recipe, as those
> configurations have now been moved to the LLVM recipe after the split.
>
> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> ---
> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
> 3 files changed, 19 insertions(+), 9 deletions(-)
>
> diff --git a/meta/recipes-devtools/clang/clang_git.bb b/meta/recipes-devtools/clang/clang_git.bb
> index 53bca1c24f..3e117b308b 100644
> --- a/meta/recipes-devtools/clang/clang_git.bb
> +++ b/meta/recipes-devtools/clang/clang_git.bb
> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES', 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
> LLVM_TARGETS_TO_BUILD ?= "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb, d)};BPF;${LLVM_TARGETS_GPU}"
>
> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>
> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
> -DLLVM_ENABLE_PIC=ON \
> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
> -DLLVM_BINDINGS_LIST='' \
> + -DLLVM_INSTALL_UTILS=ON \
> -DLLVM_ENABLE_FFI=ON \
> -DLLVM_ENABLE_ZSTD=ON \
> -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \
> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
> diff --git a/meta/recipes-devtools/clang/common-clang.inc b/meta/recipes-devtools/clang/common-clang.inc
> index bf3a63914a..c22e3c1b19 100644
> --- a/meta/recipes-devtools/clang/common-clang.inc
> +++ b/meta/recipes-devtools/clang/common-clang.inc
> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
> elif re.match('aarch64$', a): return 'AArch64'
> elif re.match('aarch64_be$', a): return 'AArch64'
> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
> - elif re.match('riscv32$', a): return 'RISCV'
> - elif re.match('riscv64$', a): return 'RISCV'
> + elif re.match('riscv32$', a): return 'riscv32'
> + elif re.match('riscv64$', a): return 'riscv64'
> elif re.match('p(pc|owerpc)(|64)', a): return 'PowerPC'
> - elif re.match('loongarch64$', a): return 'LoongArch'
> + elif re.match('loongarch64$', a): return 'loongarch64'
> else:
> bb.fatal("Unhandled architecture %s" % arch_val)
> return ""
> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb b/meta/recipes-devtools/rust/rust_1.90.0.bb
> index 5d804c7398..c2cb8f8829 100644
> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
> inherit rust
> inherit cargo_common
>
> -DEPENDS += "rust-llvm"
> +DEPENDS += "ninja-native clang"
> # native rust uses cargo/rustc from binary snapshots to bootstrap
> # but everything else should use our native builds
> DEPENDS:append:class-target = " cargo-native rust-native"
> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL', 'stable', '', '-${RUST_CHANNEL}', d)
>
> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>
> -RUST_ALTERNATE_EXE_PATH ?= "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
> -RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>
> # We don't want to use bitbakes vendoring because the rust sources do their
> # own vendoring.
> @@ -188,6 +188,16 @@ python do_configure() {
> bb.build.exec_func("setup_cargo_environment", d)
> }
>
> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
> +#Since LLVM is built as a non-multilib component, the 'lib' directory
> +#doesn't have any library files when multilibs enabled. So, copying
> +#library files without impacting multilib behavior.
> +do_compile:append:class-target() {
> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a ${STAGING_DIR_TARGET}/usr/lib/.
> +fi
> +}
llvm-config can deduce paths to be self-relative, You might try to
create a copy of native llvm-config in STAGING_LIBDIR or STAGING_BINDIR
and that might help
> +
> rust_runx () {
> echo "COMPILE ${PN}" "$@"
>
> @@ -199,7 +209,7 @@ rust_runx () {
> unset CXXFLAGS
> unset CPPFLAGS
>
> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz -Clink-arg=-lzstd"
>
> # Copy the natively built llvm-config into the target so we can run it. Horrible,
> # but works!
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#224075): https://lists.openembedded.org/g/openembedded-core/message/224075
> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-09-26 10:24 [PATCH V3 1/2] rust: Use clang instead of rust-llvm Deepesh.Varatharajan
2025-09-26 10:24 ` [PATCH V3 2/2] rust: Drop rust-llvm Deepesh.Varatharajan
2025-09-26 16:50 ` [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm Khem Raj
@ 2025-09-26 17:30 ` Khem Raj
2025-09-29 8:29 ` Deepesh Varatharajan
2025-09-29 8:11 ` Mathieu Dubois-Briand
3 siblings, 1 reply; 14+ messages in thread
From: Khem Raj @ 2025-09-26 17:30 UTC (permalink / raw)
To: deepesh.varatharajan, openembedded-core; +Cc: Sundeep.Kokkonda
On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org wrote:
> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>
> Updated the Rust build to depend on Clang instead.
>
> *Summary of discussion with the rust upstream about using latest LLVM instead of Rust maintained LLVM fork.
> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>
> *Upstream LLVM is generally compatible:
> - Rust does support building with upstream (vanilla) LLVM, especially the latest
> major release and the one or two preceding ones.
> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>
> *Impact on Yocto Rust upgrades:
> - Rust upgrades shall always check for updates on rust forked llvm and backport
> the relevant patches to clang's llvm.
>
> *Regarding the rust forked llvm local patches:
> - There are no local patches on rust forked llvm other than the backported fixes
> from llvm master.
>
> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of this following
> diff otherwise we will get errors during link time.
>
> Setup in rust-llvm
> -DLLVM_ENABLE_ZLIB=OFF \
> -DLLVM_ENABLE_ZSTD=OFF \
> -DLLVM_ENABLE_FFI=OFF \
>
> Setup in clang
> -DLLVM_ENABLE_FFI=ON \
> -DLLVM_ENABLE_ZSTD=ON \
>
> *When multilibs enabled:
>
> llvm-config expects static libraries to be located in the lib directory rather than
> lib64. However, since LLVM is built as a non-multilib component, the lib directory
> doesn't contain any library files. To accommodate this without breaking multilib
> behavior, we copy the required library files appropriately.
>
> Previously, when we depended on rust-llvm, this worked because we specified:
> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>
> With this setup, llvm-config was installed inside ${libdir}/llvm-rust, which included
> its own bin and lib directories. Thus, llvm-config located in bin would correctly find
> the libraries in the adjacent lib directory.
>
> Even when multilib was enabled or not, llvm-config would still look for libraries under
> lib in this structure, so everything functioned as expected.
>
> *Changes needs to be done when llvm splits from clang:
> In rust recipe:
> Update the dependency from:
> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>
> In llvm recipe:
> Apply the same changes that were made in the Clang recipe, as those
> configurations have now been moved to the LLVM recipe after the split.
>
> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> ---
> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
> 3 files changed, 19 insertions(+), 9 deletions(-)
>
> diff --git a/meta/recipes-devtools/clang/clang_git.bb b/meta/recipes-devtools/clang/clang_git.bb
> index 53bca1c24f..3e117b308b 100644
> --- a/meta/recipes-devtools/clang/clang_git.bb
> +++ b/meta/recipes-devtools/clang/clang_git.bb
> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES', 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
> LLVM_TARGETS_TO_BUILD ?= "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb, d)};BPF;${LLVM_TARGETS_GPU}"
compiler we build is building all architectures once because it then
reuses same compiler for all cross compilers just by creating canonical
symlinks, this change is not going to work.
>
> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>
> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
> -DLLVM_ENABLE_PIC=ON \
> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
> -DLLVM_BINDINGS_LIST='' \
> + -DLLVM_INSTALL_UTILS=ON \
tabs vs spaces inconsistenty. Please fix it.
> -DLLVM_ENABLE_FFI=ON \
> -DLLVM_ENABLE_ZSTD=ON \
> -DFFI_INCLUDE_DIR=$(pkg-config --variable=includedir libffi) \
> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
why is this change needed ?
> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
> diff --git a/meta/recipes-devtools/clang/common-clang.inc b/meta/recipes-devtools/clang/common-clang.inc
> index bf3a63914a..c22e3c1b19 100644
> --- a/meta/recipes-devtools/clang/common-clang.inc
> +++ b/meta/recipes-devtools/clang/common-clang.inc
> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
> elif re.match('aarch64$', a): return 'AArch64'
> elif re.match('aarch64_be$', a): return 'AArch64'
> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
> - elif re.match('riscv32$', a): return 'RISCV'
> - elif re.match('riscv64$', a): return 'RISCV'
> + elif re.match('riscv32$', a): return 'riscv32'
> + elif re.match('riscv64$', a): return 'riscv64'
This is representing LLVM backend name and not normal arch and clang
uses RISCV for both rv64 and rv32
> elif re.match('p(pc|owerpc)(|64)', a): return 'PowerPC'
> - elif re.match('loongarch64$', a): return 'LoongArch'
> + elif re.match('loongarch64$', a): return 'loongarch64'
same problem as above here
> else:
> bb.fatal("Unhandled architecture %s" % arch_val)
> return ""
> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb b/meta/recipes-devtools/rust/rust_1.90.0.bb
> index 5d804c7398..c2cb8f8829 100644
> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
> inherit rust
> inherit cargo_common
>
> -DEPENDS += "rust-llvm"
> +DEPENDS += "ninja-native clang"
I think using 'llvm' instead of 'clang' here is more appropriate to
represent the dependency, clang provides llvm as well, it also ensures
that when we split llvm out of clang recipe then you do not need to
change it.
> # native rust uses cargo/rustc from binary snapshots to bootstrap
> # but everything else should use our native builds
> DEPENDS:append:class-target = " cargo-native rust-native"
> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL', 'stable', '', '-${RUST_CHANNEL}', d)
>
> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>
> -RUST_ALTERNATE_EXE_PATH ?= "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
> -RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>
> # We don't want to use bitbakes vendoring because the rust sources do their
> # own vendoring.
> @@ -188,6 +188,16 @@ python do_configure() {
> bb.build.exec_func("setup_cargo_environment", d)
> }
>
> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
> +#Since LLVM is built as a non-multilib component, the 'lib' directory
> +#doesn't have any library files when multilibs enabled. So, copying
> +#library files without impacting multilib behavior.
> +do_compile:append:class-target() {
> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a ${STAGING_DIR_TARGET}/usr/lib/.
> +fi
> +}
> +
> rust_runx () {
> echo "COMPILE ${PN}" "$@"
>
> @@ -199,7 +209,7 @@ rust_runx () {
> unset CXXFLAGS
> unset CPPFLAGS
>
> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz -Clink-arg=-lzstd"
>
> # Copy the natively built llvm-config into the target so we can run it. Horrible,
> # but works!
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#224075): https://lists.openembedded.org/g/openembedded-core/message/224075
> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-09-26 10:24 [PATCH V3 1/2] rust: Use clang instead of rust-llvm Deepesh.Varatharajan
` (2 preceding siblings ...)
2025-09-26 17:30 ` Khem Raj
@ 2025-09-29 8:11 ` Mathieu Dubois-Briand
3 siblings, 0 replies; 14+ messages in thread
From: Mathieu Dubois-Briand @ 2025-09-29 8:11 UTC (permalink / raw)
To: deepesh.varatharajan, openembedded-core
Cc: Sundeep.Kokkonda, Deepesh.Varatharajan
On Fri Sep 26, 2025 at 12:24 PM CEST, Deepesh via lists.openembedded.org Varatharajan wrote:
> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>
> Updated the Rust build to depend on Clang instead.
>
> *Summary of discussion with the rust upstream about using latest LLVM instead of Rust maintained LLVM fork.
> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>
> *Upstream LLVM is generally compatible:
> - Rust does support building with upstream (vanilla) LLVM, especially the latest
> major release and the one or two preceding ones.
> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>
> *Impact on Yocto Rust upgrades:
> - Rust upgrades shall always check for updates on rust forked llvm and backport
> the relevant patches to clang's llvm.
>
> *Regarding the rust forked llvm local patches:
> - There are no local patches on rust forked llvm other than the backported fixes
> from llvm master.
>
> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of this following
> diff otherwise we will get errors during link time.
>
> Setup in rust-llvm
> -DLLVM_ENABLE_ZLIB=OFF \
> -DLLVM_ENABLE_ZSTD=OFF \
> -DLLVM_ENABLE_FFI=OFF \
>
> Setup in clang
> -DLLVM_ENABLE_FFI=ON \
> -DLLVM_ENABLE_ZSTD=ON \
>
> *When multilibs enabled:
>
> llvm-config expects static libraries to be located in the lib directory rather than
> lib64. However, since LLVM is built as a non-multilib component, the lib directory
> doesn't contain any library files. To accommodate this without breaking multilib
> behavior, we copy the required library files appropriately.
>
> Previously, when we depended on rust-llvm, this worked because we specified:
> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>
> With this setup, llvm-config was installed inside ${libdir}/llvm-rust, which included
> its own bin and lib directories. Thus, llvm-config located in bin would correctly find
> the libraries in the adjacent lib directory.
>
> Even when multilib was enabled or not, llvm-config would still look for libraries under
> lib in this structure, so everything functioned as expected.
>
> *Changes needs to be done when llvm splits from clang:
> In rust recipe:
> Update the dependency from:
> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>
> In llvm recipe:
> Apply the same changes that were made in the Clang recipe, as those
> configurations have now been moved to the LLVM recipe after the split.
>
> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> ---
Hi Deepesh,
Thanks for your patch.
It looks like this is breaking some build, strangely only the arm64 one
with musl.
ERROR: rust-1.90.0-r0 do_install: Execution of '/srv/pokybuild/yocto-worker/musl-qemuarm64/build/build/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/temp/run.do_install.3194366' failed with exit code 1
...
| Building libunwind.a for aarch64-poky-linux-musl
| cargo:warning=/srv/pokybuild/yocto-worker/musl-qemuarm64/build/build/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/llvm-project/libunwind/src/UnwindLevel1.c:191:1: error: arch extension 'gcs' should be prefixed by '+'
| cargo:warning= 191 | unwind_phase2(unw_context_t *uc, unw_cursor_t *cursor, _Unwind_Exception *exception_object) {
| cargo:warning= | ^~~~~~~~~~~~~
| cargo:warning=/srv/pokybuild/yocto-worker/musl-qemuarm64/build/build/tmp/work/cortexa57-poky-linux-musl/rust/1.90.0/sources/rustc-1.90.0-src/src/llvm-project/libunwind/src/UnwindLevel1.c:337:22: error: arch extension 'gcs' should be prefixed by '+'
| cargo:warning= 337 | _Unwind_Stop_Fn stop, void *stop_parameter) {
| cargo:warning= | ^~~~~~~~~~~~~~~
|
Can you have a look at this failure please?
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-09-26 17:30 ` Khem Raj
@ 2025-09-29 8:29 ` Deepesh Varatharajan
2025-10-01 19:29 ` Khem Raj
0 siblings, 1 reply; 14+ messages in thread
From: Deepesh Varatharajan @ 2025-09-29 8:29 UTC (permalink / raw)
To: Khem Raj, openembedded-core; +Cc: Sundeep.Kokkonda
On 26-09-2025 23:00, Khem Raj wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender
> and know the content is safe.
>
> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
> wrote:
>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>
>> Updated the Rust build to depend on Clang instead.
>>
>> *Summary of discussion with the rust upstream about using latest LLVM
>> instead of Rust maintained LLVM fork.
>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>
>>
>> *Upstream LLVM is generally compatible:
>> - Rust does support building with upstream (vanilla) LLVM, especially
>> the latest
>> major release and the one or two preceding ones.
>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>
>>
>> *Impact on Yocto Rust upgrades:
>> - Rust upgrades shall always check for updates on rust forked llvm
>> and backport
>> the relevant patches to clang's llvm.
>>
>> *Regarding the rust forked llvm local patches:
>> - There are no local patches on rust forked llvm other than the
>> backported fixes
>> from llvm master.
>>
>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
>> this following
>> diff otherwise we will get errors during link time.
>>
>> Setup in rust-llvm
>> -DLLVM_ENABLE_ZLIB=OFF \
>> -DLLVM_ENABLE_ZSTD=OFF \
>> -DLLVM_ENABLE_FFI=OFF \
>>
>> Setup in clang
>> -DLLVM_ENABLE_FFI=ON \
>> -DLLVM_ENABLE_ZSTD=ON \
>>
>> *When multilibs enabled:
>>
>> llvm-config expects static libraries to be located in the lib
>> directory rather than
>> lib64. However, since LLVM is built as a non-multilib component, the
>> lib directory
>> doesn't contain any library files. To accommodate this without
>> breaking multilib
>> behavior, we copy the required library files appropriately.
>>
>> Previously, when we depended on rust-llvm, this worked because we
>> specified:
>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>
>> With this setup, llvm-config was installed inside
>> ${libdir}/llvm-rust, which included
>> its own bin and lib directories. Thus, llvm-config located in bin
>> would correctly find
>> the libraries in the adjacent lib directory.
>>
>> Even when multilib was enabled or not, llvm-config would still look
>> for libraries under
>> lib in this structure, so everything functioned as expected.
>>
>> *Changes needs to be done when llvm splits from clang:
>> In rust recipe:
>> Update the dependency from:
>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>>
>> In llvm recipe:
>> Apply the same changes that were made in the Clang recipe, as those
>> configurations have now been moved to the LLVM recipe after the split.
>>
>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>> ---
>> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
>> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
>> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
>> 3 files changed, 19 insertions(+), 9 deletions(-)
>>
>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
>> b/meta/recipes-devtools/clang/clang_git.bb
>> index 53bca1c24f..3e117b308b 100644
>> --- a/meta/recipes-devtools/clang/clang_git.bb
>> +++ b/meta/recipes-devtools/clang/clang_git.bb
>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>> LLVM_TARGETS_TO_BUILD ?=
>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>> d)};BPF;${LLVM_TARGETS_GPU}"
>
> compiler we build is building all architectures once because it then
> reuses same compiler for all cross compilers just by creating canonical
> symlinks, this change is not going to work.
We removed this line because, for the clang class-target build, LLVM was
only compiling
libraries for the target architecture and GPU targets and kept inside
the target
sysroot. Since we use the llvm-config built for the class-target during
the Rust target build,
it was failing due to the absence of static libraries for other
architectures in the target sysroot.
Here the target is x86 arch. So, it compiled libraries for x86 arch only
and we can see the error
for missing libraries for other archs as below:
| --- stderr
| llvm-config: error: missing:
poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
| llvm-config: error: missing:
poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
| llvm-config: error: missing:
poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
| llvm-config: error: missing:
poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
| llvm-config: error: missing:
poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
| llvm-config: error: missing:
poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
.
.
.
| thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
| command did not execute successfully:
"poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
"--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
"bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
"loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
| expected success, got: exit status: 1
Maybe instead of removing. Can we change
-LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
d)};BPF;${LLVM_TARGETS_GPU}"
to
-LLVM_TARGETS_TO_BUILD:class-target ?=
"AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
?
>
>>
>> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>>
>> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
>> -DLLVM_ENABLE_PIC=ON \
>> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
>> -DLLVM_BINDINGS_LIST='' \
>> + -DLLVM_INSTALL_UTILS=ON \
>
> tabs vs spaces inconsistenty. Please fix it.
Sure.
>
>> -DLLVM_ENABLE_FFI=ON \
>> -DLLVM_ENABLE_ZSTD=ON \
>> -DFFI_INCLUDE_DIR=$(pkg-config
>> --variable=includedir libffi) \
>> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
>> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
>> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
>> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>
> why is this change needed ?
>
I made this change to ensure that LLVM_TARGET_ARCH is set correctly to
the actual target architecture.
However, it appears that even without this change, the build completes
successfully for the target. I checked
and printed the value of HOST_ARCH and realized it was already set
correctly my mistake. This change is
not required.
>> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
>> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
>> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
>> diff --git a/meta/recipes-devtools/clang/common-clang.inc
>> b/meta/recipes-devtools/clang/common-clang.inc
>> index bf3a63914a..c22e3c1b19 100644
>> --- a/meta/recipes-devtools/clang/common-clang.inc
>> +++ b/meta/recipes-devtools/clang/common-clang.inc
>> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
>> elif re.match('aarch64$', a): return
>> 'AArch64'
>> elif re.match('aarch64_be$', a): return
>> 'AArch64'
>> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
>> - elif re.match('riscv32$', a): return 'RISCV'
>> - elif re.match('riscv64$', a): return 'RISCV'
>> + elif re.match('riscv32$', a): return 'riscv32'
>> + elif re.match('riscv64$', a): return 'riscv64'
>
> This is representing LLVM backend name and not normal arch and clang
> uses RISCV for both rv64 and rv32
>
After this change were made,
- -DLLVM_TARGET_ARCH=${HOST_ARCH} \
+ -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
do_configure of clang failed
with the following error:
| CMake Error at cmake/config-ix.cmake:585 (message):
| Unknown architecture riscv
Since this change is irrelevant
- -DLLVM_TARGET_ARCH=${HOST_ARCH} \
+ -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
The below changes are also not needed.
- elif re.match('riscv32$', a): return 'RISCV'
- elif re.match('riscv64$', a): return 'RISCV'
+ elif re.match('riscv32$', a): return 'riscv32'
+ elif re.match('riscv64$', a): return 'riscv64'
- elif re.match('loongarch64$', a): return 'LoongArch'
+ elif re.match('loongarch64$', a): return
'loongarch64'
>> elif re.match('p(pc|owerpc)(|64)', a): return
>> 'PowerPC'
>> - elif re.match('loongarch64$', a): return
>> 'LoongArch'
>> + elif re.match('loongarch64$', a): return
>> 'loongarch64'
>
> same problem as above here
>
>> else:
>> bb.fatal("Unhandled architecture %s" % arch_val)
>> return ""
>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
>> b/meta/recipes-devtools/rust/rust_1.90.0.bb
>> index 5d804c7398..c2cb8f8829 100644
>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
>> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
>> inherit rust
>> inherit cargo_common
>>
>> -DEPENDS += "rust-llvm"
>> +DEPENDS += "ninja-native clang"
>
> I think using 'llvm' instead of 'clang' here is more appropriate to
> represent the dependency, clang provides llvm as well, it also ensures
> that when we split llvm out of clang recipe then you do not need to
> change it.
Sure will do that.
>
>> # native rust uses cargo/rustc from binary snapshots to bootstrap
>> # but everything else should use our native builds
>> DEPENDS:append:class-target = " cargo-native rust-native"
>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
>> 'stable', '', '-${RUST_CHANNEL}', d)
>>
>> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>>
>> -RUST_ALTERNATE_EXE_PATH ?=
>> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
>> -RUST_ALTERNATE_EXE_PATH_NATIVE =
>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>>
>> # We don't want to use bitbakes vendoring because the rust sources
>> do their
>> # own vendoring.
>> @@ -188,6 +188,16 @@ python do_configure() {
>> bb.build.exec_func("setup_cargo_environment", d)
>> }
>>
>> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
>> +#Since LLVM is built as a non-multilib component, the 'lib' directory
>> +#doesn't have any library files when multilibs enabled. So, copying
>> +#library files without impacting multilib behavior.
>> +do_compile:append:class-target() {
>> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
>> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
>> ${STAGING_DIR_TARGET}/usr/lib/.
>> +fi
>> +}
>> +
>> rust_runx () {
>> echo "COMPILE ${PN}" "$@"
>>
>> @@ -199,7 +209,7 @@ rust_runx () {
>> unset CXXFLAGS
>> unset CPPFLAGS
>>
>> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
>> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
>> -Clink-arg=-lzstd"
>>
>> # Copy the natively built llvm-config into the target so we can
>> run it. Horrible,
>> # but works!
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#224075):
>> https://lists.openembedded.org/g/openembedded-core/message/224075
>> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>> [raj.khem@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-09-26 16:50 ` [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm Khem Raj
@ 2025-09-29 10:17 ` Deepesh Varatharajan
0 siblings, 0 replies; 14+ messages in thread
From: Deepesh Varatharajan @ 2025-09-29 10:17 UTC (permalink / raw)
To: Khem Raj, openembedded-core; +Cc: Sundeep.Kokkonda
On 26-09-2025 22:20, Khem Raj wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender
> and know the content is safe.
>
> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
> wrote:
>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>
>> Updated the Rust build to depend on Clang instead.
>>
>> *Summary of discussion with the rust upstream about using latest LLVM
>> instead of Rust maintained LLVM fork.
>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>
>>
>> *Upstream LLVM is generally compatible:
>> - Rust does support building with upstream (vanilla) LLVM, especially
>> the latest
>> major release and the one or two preceding ones.
>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>
>>
>> *Impact on Yocto Rust upgrades:
>> - Rust upgrades shall always check for updates on rust forked llvm
>> and backport
>> the relevant patches to clang's llvm.
>>
>> *Regarding the rust forked llvm local patches:
>> - There are no local patches on rust forked llvm other than the
>> backported fixes
>> from llvm master.
>>
>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
>> this following
>> diff otherwise we will get errors during link time.
>>
>> Setup in rust-llvm
>> -DLLVM_ENABLE_ZLIB=OFF \
>> -DLLVM_ENABLE_ZSTD=OFF \
>> -DLLVM_ENABLE_FFI=OFF \
>>
>> Setup in clang
>> -DLLVM_ENABLE_FFI=ON \
>> -DLLVM_ENABLE_ZSTD=ON \
>>
>> *When multilibs enabled:
>>
>> llvm-config expects static libraries to be located in the lib
>> directory rather than
>> lib64. However, since LLVM is built as a non-multilib component, the
>> lib directory
>> doesn't contain any library files. To accommodate this without
>> breaking multilib
>> behavior, we copy the required library files appropriately.
>>
>> Previously, when we depended on rust-llvm, this worked because we
>> specified:
>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>
>> With this setup, llvm-config was installed inside
>> ${libdir}/llvm-rust, which included
>> its own bin and lib directories. Thus, llvm-config located in bin
>> would correctly find
>> the libraries in the adjacent lib directory.
>>
>> Even when multilib was enabled or not, llvm-config would still look
>> for libraries under
>> lib in this structure, so everything functioned as expected.
>>
>> *Changes needs to be done when llvm splits from clang:
>> In rust recipe:
>> Update the dependency from:
>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>>
>> In llvm recipe:
>> Apply the same changes that were made in the Clang recipe, as those
>> configurations have now been moved to the LLVM recipe after the split.
>>
>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>> ---
>> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
>> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
>> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
>> 3 files changed, 19 insertions(+), 9 deletions(-)
>>
>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
>> b/meta/recipes-devtools/clang/clang_git.bb
>> index 53bca1c24f..3e117b308b 100644
>> --- a/meta/recipes-devtools/clang/clang_git.bb
>> +++ b/meta/recipes-devtools/clang/clang_git.bb
>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>> LLVM_TARGETS_TO_BUILD ?=
>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>> d)};BPF;${LLVM_TARGETS_GPU}"
>>
>> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>>
>> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
>> -DLLVM_ENABLE_PIC=ON \
>> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
>> -DLLVM_BINDINGS_LIST='' \
>> + -DLLVM_INSTALL_UTILS=ON \
>> -DLLVM_ENABLE_FFI=ON \
>> -DLLVM_ENABLE_ZSTD=ON \
>> -DFFI_INCLUDE_DIR=$(pkg-config
>> --variable=includedir libffi) \
>> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
>> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
>> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
>> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
>> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
>> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
>> diff --git a/meta/recipes-devtools/clang/common-clang.inc
>> b/meta/recipes-devtools/clang/common-clang.inc
>> index bf3a63914a..c22e3c1b19 100644
>> --- a/meta/recipes-devtools/clang/common-clang.inc
>> +++ b/meta/recipes-devtools/clang/common-clang.inc
>> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
>> elif re.match('aarch64$', a): return
>> 'AArch64'
>> elif re.match('aarch64_be$', a): return
>> 'AArch64'
>> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
>> - elif re.match('riscv32$', a): return 'RISCV'
>> - elif re.match('riscv64$', a): return 'RISCV'
>> + elif re.match('riscv32$', a): return 'riscv32'
>> + elif re.match('riscv64$', a): return 'riscv64'
>> elif re.match('p(pc|owerpc)(|64)', a): return
>> 'PowerPC'
>> - elif re.match('loongarch64$', a): return
>> 'LoongArch'
>> + elif re.match('loongarch64$', a): return
>> 'loongarch64'
>> else:
>> bb.fatal("Unhandled architecture %s" % arch_val)
>> return ""
>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
>> b/meta/recipes-devtools/rust/rust_1.90.0.bb
>> index 5d804c7398..c2cb8f8829 100644
>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
>> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
>> inherit rust
>> inherit cargo_common
>>
>> -DEPENDS += "rust-llvm"
>> +DEPENDS += "ninja-native clang"
>> # native rust uses cargo/rustc from binary snapshots to bootstrap
>> # but everything else should use our native builds
>> DEPENDS:append:class-target = " cargo-native rust-native"
>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
>> 'stable', '', '-${RUST_CHANNEL}', d)
>>
>> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>>
>> -RUST_ALTERNATE_EXE_PATH ?=
>> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
>> -RUST_ALTERNATE_EXE_PATH_NATIVE =
>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>>
>> # We don't want to use bitbakes vendoring because the rust sources
>> do their
>> # own vendoring.
>> @@ -188,6 +188,16 @@ python do_configure() {
>> bb.build.exec_func("setup_cargo_environment", d)
>> }
>>
>> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
>> +#Since LLVM is built as a non-multilib component, the 'lib' directory
>> +#doesn't have any library files when multilibs enabled. So, copying
>> +#library files without impacting multilib behavior.
>> +do_compile:append:class-target() {
>> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
>> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
>> ${STAGING_DIR_TARGET}/usr/lib/.
>> +fi
>> +}
>
> llvm-config can deduce paths to be self-relative, You might try to
> create a copy of native llvm-config in STAGING_LIBDIR or STAGING_BINDIR
> and that might help
The llvm-config binary built for the target is already available in
STAGING_BINDIR
and we're using that for the Rust target build. I also tried copying
the natively built
llvm-config into STAGING_LIBDIR and adjusted the Rust target build to
depend on it,
but the build still failed. The issue was that it continued to look in
the lib directory
instead of lib64 when multilibs are enabled. However, this can be
addressed in an
alternative way as below:
-RUST_ALTERNATE_EXE_PATH ?= "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
-RUST_ALTERNATE_EXE_PATH_NATIVE =
"${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
+RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
+RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
Currently I am making the above changes. Since as you suggested to copy
and use
natively built llvm-config, I'm wondering would it be better to simply
make the
Rust target build depend directly on the native llvm-config as below.
-RUST_ALTERNATE_EXE_PATH ?= "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
-RUST_ALTERNATE_EXE_PATH_NATIVE =
"${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
+RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR_NATIVE}/llvm-config"
+RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>
>> +
>> rust_runx () {
>> echo "COMPILE ${PN}" "$@"
>>
>> @@ -199,7 +209,7 @@ rust_runx () {
>> unset CXXFLAGS
>> unset CPPFLAGS
>>
>> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
>> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
>> -Clink-arg=-lzstd"
>>
>> # Copy the natively built llvm-config into the target so we can
>> run it. Horrible,
>> # but works!
>>
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#224075):
>> https://lists.openembedded.org/g/openembedded-core/message/224075
>> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>> [raj.khem@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-09-29 8:29 ` Deepesh Varatharajan
@ 2025-10-01 19:29 ` Khem Raj
2025-10-07 11:10 ` Deepesh Varatharajan
0 siblings, 1 reply; 14+ messages in thread
From: Khem Raj @ 2025-10-01 19:29 UTC (permalink / raw)
To: Deepesh Varatharajan; +Cc: openembedded-core, Sundeep.Kokkonda
On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
<deepesh.varatharajan@windriver.com> wrote:
>
>
> On 26-09-2025 23:00, Khem Raj wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender
> > and know the content is safe.
> >
> > On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
> > wrote:
> >> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> >>
> >> Updated the Rust build to depend on Clang instead.
> >>
> >> *Summary of discussion with the rust upstream about using latest LLVM
> >> instead of Rust maintained LLVM fork.
> >> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
> >>
> >>
> >> *Upstream LLVM is generally compatible:
> >> - Rust does support building with upstream (vanilla) LLVM, especially
> >> the latest
> >> major release and the one or two preceding ones.
> >> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
> >>
> >>
> >> *Impact on Yocto Rust upgrades:
> >> - Rust upgrades shall always check for updates on rust forked llvm
> >> and backport
> >> the relevant patches to clang's llvm.
> >>
> >> *Regarding the rust forked llvm local patches:
> >> - There are no local patches on rust forked llvm other than the
> >> backported fixes
> >> from llvm master.
> >>
> >> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
> >> this following
> >> diff otherwise we will get errors during link time.
> >>
> >> Setup in rust-llvm
> >> -DLLVM_ENABLE_ZLIB=OFF \
> >> -DLLVM_ENABLE_ZSTD=OFF \
> >> -DLLVM_ENABLE_FFI=OFF \
> >>
> >> Setup in clang
> >> -DLLVM_ENABLE_FFI=ON \
> >> -DLLVM_ENABLE_ZSTD=ON \
> >>
> >> *When multilibs enabled:
> >>
> >> llvm-config expects static libraries to be located in the lib
> >> directory rather than
> >> lib64. However, since LLVM is built as a non-multilib component, the
> >> lib directory
> >> doesn't contain any library files. To accommodate this without
> >> breaking multilib
> >> behavior, we copy the required library files appropriately.
> >>
> >> Previously, when we depended on rust-llvm, this worked because we
> >> specified:
> >> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
> >>
> >> With this setup, llvm-config was installed inside
> >> ${libdir}/llvm-rust, which included
> >> its own bin and lib directories. Thus, llvm-config located in bin
> >> would correctly find
> >> the libraries in the adjacent lib directory.
> >>
> >> Even when multilib was enabled or not, llvm-config would still look
> >> for libraries under
> >> lib in this structure, so everything functioned as expected.
> >>
> >> *Changes needs to be done when llvm splits from clang:
> >> In rust recipe:
> >> Update the dependency from:
> >> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
> >>
> >> In llvm recipe:
> >> Apply the same changes that were made in the Clang recipe, as those
> >> configurations have now been moved to the LLVM recipe after the split.
> >>
> >> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> >> ---
> >> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
> >> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
> >> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
> >> 3 files changed, 19 insertions(+), 9 deletions(-)
> >>
> >> diff --git a/meta/recipes-devtools/clang/clang_git.bb
> >> b/meta/recipes-devtools/clang/clang_git.bb
> >> index 53bca1c24f..3e117b308b 100644
> >> --- a/meta/recipes-devtools/clang/clang_git.bb
> >> +++ b/meta/recipes-devtools/clang/clang_git.bb
> >> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
> >> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
> >> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
> >> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
> >> LLVM_TARGETS_TO_BUILD ?=
> >> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
> >> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
> >> d)};BPF;${LLVM_TARGETS_GPU}"
> >
> > compiler we build is building all architectures once because it then
> > reuses same compiler for all cross compilers just by creating canonical
> > symlinks, this change is not going to work.
> We removed this line because, for the clang class-target build, LLVM was
> only compiling
> libraries for the target architecture and GPU targets and kept inside
> the target
> sysroot. Since we use the llvm-config built for the class-target during
> the Rust target build,
> it was failing due to the absence of static libraries for other
> architectures in the target sysroot.
> Here the target is x86 arch. So, it compiled libraries for x86 arch only
> and we can see the error
> for missing libraries for other archs as below:
>
> | --- stderr
> | llvm-config: error: missing:
> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
> | llvm-config: error: missing:
> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
> | llvm-config: error: missing:
> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
> | llvm-config: error: missing:
> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
> | llvm-config: error: missing:
> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
> | llvm-config: error: missing:
> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
> .
> .
> .
> | thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
> | command did not execute successfully:
> "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
>
> "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
> "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
> "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
> | expected success, got: exit status: 1
>
> Maybe instead of removing. Can we change
> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
> d)};BPF;${LLVM_TARGETS_GPU}"
> to
> -LLVM_TARGETS_TO_BUILD:class-target ?=
> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
> ?
I think we don't want to build all the general arches for target, only
the target arch and gpu arches
is what is needed. If we build all the backends, that will slow down
things. But if target rust needs
to support all the architectures to generate code for, then we don't
have much choice. Can you see
if we are ok to just target one architecture ?
> >
> >>
> >> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
> >>
> >> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
> >> -DLLVM_ENABLE_PIC=ON \
> >> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
> >> -DLLVM_BINDINGS_LIST='' \
> >> + -DLLVM_INSTALL_UTILS=ON \
> >
> > tabs vs spaces inconsistenty. Please fix it.
> Sure.
> >
> >> -DLLVM_ENABLE_FFI=ON \
> >> -DLLVM_ENABLE_ZSTD=ON \
> >> -DFFI_INCLUDE_DIR=$(pkg-config
> >> --variable=includedir libffi) \
> >> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
> >> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
> >> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
> >> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
> >> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> >> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
> >
> > why is this change needed ?
> >
> I made this change to ensure that LLVM_TARGET_ARCH is set correctly to
> the actual target architecture.
> However, it appears that even without this change, the build completes
> successfully for the target. I checked
> and printed the value of HOST_ARCH and realized it was already set
> correctly my mistake. This change is
> not required.
LLVM_TARGET_ARCH is different than the backend architecture which is
denoted by LLVM_TARGETS_TO_BUILD
> >> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
> >> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
> >> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
> >> diff --git a/meta/recipes-devtools/clang/common-clang.inc
> >> b/meta/recipes-devtools/clang/common-clang.inc
> >> index bf3a63914a..c22e3c1b19 100644
> >> --- a/meta/recipes-devtools/clang/common-clang.inc
> >> +++ b/meta/recipes-devtools/clang/common-clang.inc
> >> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
> >> elif re.match('aarch64$', a): return
> >> 'AArch64'
> >> elif re.match('aarch64_be$', a): return
> >> 'AArch64'
> >> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
> >> - elif re.match('riscv32$', a): return 'RISCV'
> >> - elif re.match('riscv64$', a): return 'RISCV'
> >> + elif re.match('riscv32$', a): return 'riscv32'
> >> + elif re.match('riscv64$', a): return 'riscv64'
> >
> > This is representing LLVM backend name and not normal arch and clang
> > uses RISCV for both rv64 and rv32
> >
> After this change were made,
> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
> do_configure of clang failed
> with the following error:
>
> | CMake Error at cmake/config-ix.cmake:585 (message):
> | Unknown architecture riscv
>
> Since this change is irrelevant
> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>
> The below changes are also not needed.
> - elif re.match('riscv32$', a): return 'RISCV'
> - elif re.match('riscv64$', a): return 'RISCV'
> + elif re.match('riscv32$', a): return 'riscv32'
> + elif re.match('riscv64$', a): return 'riscv64'
> - elif re.match('loongarch64$', a): return 'LoongArch'
> + elif re.match('loongarch64$', a): return
> 'loongarch64'
> >> elif re.match('p(pc|owerpc)(|64)', a): return
> >> 'PowerPC'
> >> - elif re.match('loongarch64$', a): return
> >> 'LoongArch'
> >> + elif re.match('loongarch64$', a): return
> >> 'loongarch64'
> >
> > same problem as above here
> >
> >> else:
> >> bb.fatal("Unhandled architecture %s" % arch_val)
> >> return ""
> >> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
> >> b/meta/recipes-devtools/rust/rust_1.90.0.bb
> >> index 5d804c7398..c2cb8f8829 100644
> >> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
> >> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
> >> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
> >> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
> >> inherit rust
> >> inherit cargo_common
> >>
> >> -DEPENDS += "rust-llvm"
> >> +DEPENDS += "ninja-native clang"
> >
> > I think using 'llvm' instead of 'clang' here is more appropriate to
> > represent the dependency, clang provides llvm as well, it also ensures
> > that when we split llvm out of clang recipe then you do not need to
> > change it.
> Sure will do that.
> >
> >> # native rust uses cargo/rustc from binary snapshots to bootstrap
> >> # but everything else should use our native builds
> >> DEPENDS:append:class-target = " cargo-native rust-native"
> >> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
> >> 'stable', '', '-${RUST_CHANNEL}', d)
> >>
> >> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
> >>
> >> -RUST_ALTERNATE_EXE_PATH ?=
> >> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
> >> -RUST_ALTERNATE_EXE_PATH_NATIVE =
> >> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
> >> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
> >> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
> >>
> >> # We don't want to use bitbakes vendoring because the rust sources
> >> do their
> >> # own vendoring.
> >> @@ -188,6 +188,16 @@ python do_configure() {
> >> bb.build.exec_func("setup_cargo_environment", d)
> >> }
> >>
> >> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
> >> +#Since LLVM is built as a non-multilib component, the 'lib' directory
> >> +#doesn't have any library files when multilibs enabled. So, copying
> >> +#library files without impacting multilib behavior.
> >> +do_compile:append:class-target() {
> >> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
> >> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
> >> ${STAGING_DIR_TARGET}/usr/lib/.
> >> +fi
> >> +}
> >> +
> >> rust_runx () {
> >> echo "COMPILE ${PN}" "$@"
> >>
> >> @@ -199,7 +209,7 @@ rust_runx () {
> >> unset CXXFLAGS
> >> unset CPPFLAGS
> >>
> >> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
> >> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
> >> -Clink-arg=-lzstd"
> >>
> >> # Copy the natively built llvm-config into the target so we can
> >> run it. Horrible,
> >> # but works!
> >>
> >>
> >>
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >> Links: You receive all messages sent to this group.
> >> View/Reply Online (#224075):
> >> https://lists.openembedded.org/g/openembedded-core/message/224075
> >> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
> >> Group Owner: openembedded-core+owner@lists.openembedded.org
> >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
> >> [raj.khem@gmail.com]
> >> -=-=-=-=-=-=-=-=-=-=-=-
> >>
> >
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-10-01 19:29 ` Khem Raj
@ 2025-10-07 11:10 ` Deepesh Varatharajan
2025-10-07 20:17 ` Khem Raj
0 siblings, 1 reply; 14+ messages in thread
From: Deepesh Varatharajan @ 2025-10-07 11:10 UTC (permalink / raw)
To: Khem Raj; +Cc: openembedded-core, Sundeep.Kokkonda
On 02-10-2025 00:59, Khem Raj wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
> <deepesh.varatharajan@windriver.com> wrote:
>>
>> On 26-09-2025 23:00, Khem Raj wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender
>>> and know the content is safe.
>>>
>>> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
>>> wrote:
>>>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>
>>>> Updated the Rust build to depend on Clang instead.
>>>>
>>>> *Summary of discussion with the rust upstream about using latest LLVM
>>>> instead of Rust maintained LLVM fork.
>>>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>>>
>>>>
>>>> *Upstream LLVM is generally compatible:
>>>> - Rust does support building with upstream (vanilla) LLVM, especially
>>>> the latest
>>>> major release and the one or two preceding ones.
>>>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>>>
>>>>
>>>> *Impact on Yocto Rust upgrades:
>>>> - Rust upgrades shall always check for updates on rust forked llvm
>>>> and backport
>>>> the relevant patches to clang's llvm.
>>>>
>>>> *Regarding the rust forked llvm local patches:
>>>> - There are no local patches on rust forked llvm other than the
>>>> backported fixes
>>>> from llvm master.
>>>>
>>>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
>>>> this following
>>>> diff otherwise we will get errors during link time.
>>>>
>>>> Setup in rust-llvm
>>>> -DLLVM_ENABLE_ZLIB=OFF \
>>>> -DLLVM_ENABLE_ZSTD=OFF \
>>>> -DLLVM_ENABLE_FFI=OFF \
>>>>
>>>> Setup in clang
>>>> -DLLVM_ENABLE_FFI=ON \
>>>> -DLLVM_ENABLE_ZSTD=ON \
>>>>
>>>> *When multilibs enabled:
>>>>
>>>> llvm-config expects static libraries to be located in the lib
>>>> directory rather than
>>>> lib64. However, since LLVM is built as a non-multilib component, the
>>>> lib directory
>>>> doesn't contain any library files. To accommodate this without
>>>> breaking multilib
>>>> behavior, we copy the required library files appropriately.
>>>>
>>>> Previously, when we depended on rust-llvm, this worked because we
>>>> specified:
>>>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>>>
>>>> With this setup, llvm-config was installed inside
>>>> ${libdir}/llvm-rust, which included
>>>> its own bin and lib directories. Thus, llvm-config located in bin
>>>> would correctly find
>>>> the libraries in the adjacent lib directory.
>>>>
>>>> Even when multilib was enabled or not, llvm-config would still look
>>>> for libraries under
>>>> lib in this structure, so everything functioned as expected.
>>>>
>>>> *Changes needs to be done when llvm splits from clang:
>>>> In rust recipe:
>>>> Update the dependency from:
>>>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>>>>
>>>> In llvm recipe:
>>>> Apply the same changes that were made in the Clang recipe, as those
>>>> configurations have now been moved to the LLVM recipe after the split.
>>>>
>>>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>> ---
>>>> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
>>>> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
>>>> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
>>>> 3 files changed, 19 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
>>>> b/meta/recipes-devtools/clang/clang_git.bb
>>>> index 53bca1c24f..3e117b308b 100644
>>>> --- a/meta/recipes-devtools/clang/clang_git.bb
>>>> +++ b/meta/recipes-devtools/clang/clang_git.bb
>>>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>>>> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>>>> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
>>>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>>>> LLVM_TARGETS_TO_BUILD ?=
>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>> compiler we build is building all architectures once because it then
>>> reuses same compiler for all cross compilers just by creating canonical
>>> symlinks, this change is not going to work.
>> We removed this line because, for the clang class-target build, LLVM was
>> only compiling
>> libraries for the target architecture and GPU targets and kept inside
>> the target
>> sysroot. Since we use the llvm-config built for the class-target during
>> the Rust target build,
>> it was failing due to the absence of static libraries for other
>> architectures in the target sysroot.
>> Here the target is x86 arch. So, it compiled libraries for x86 arch only
>> and we can see the error
>> for missing libraries for other archs as below:
>>
>> | --- stderr
>> | llvm-config: error: missing:
>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
>> | llvm-config: error: missing:
>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
>> | llvm-config: error: missing:
>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
>> | llvm-config: error: missing:
>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
>> | llvm-config: error: missing:
>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
>> | llvm-config: error: missing:
>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
>> .
>> .
>> .
>> | thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
>> | command did not execute successfully:
>> "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
>>
>> "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
>> "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
>> "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
>> | expected success, got: exit status: 1
>>
>> Maybe instead of removing. Can we change
>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>> d)};BPF;${LLVM_TARGETS_GPU}"
>> to
>> -LLVM_TARGETS_TO_BUILD:class-target ?=
>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
>> ?
> I think we don't want to build all the general arches for target, only
> the target arch and gpu arches
> is what is needed. If we build all the backends, that will slow down
> things. But if target rust needs
> to support all the architectures to generate code for, then we don't
> have much choice. Can you see
> if we are ok to just target one architecture ?
The issue doesn't originate from Rust directly, it stems from the
following logic
in the Rust recipe, where the natively built llvm-config is copied into
the target sysroot:
# Copy the natively built llvm-config into the target so we can run
it. Horrible,
# but works!
if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
patchelf --set-rpath \$ORIGIN/../../../../../`basename
${STAGING_DIR_NATIVE}`${libdir_native} ${RUST_ALTERNATE_EXE_PATH}
else
patchelf --remove-rpath ${RUST_ALTERNATE_EXE_PATH}
fi
fi
Since llvm-config is being copied from ${STAGING_BINDIR_NATIVE} to
${STAGING_BINDIR},
and the native Clang was built with the following configuration:
LLVM_TARGETS_TO_BUILD ?=
"AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
The copied llvm-config will expect the corresponding target libraries to
be present in the ${STAGING_LIBDIR}.
To address this issue, as far as I can see we have three options.
1) Build clang-target also for all mentioned archs for clang-native.
2) Copy all the required libraries from ${STAGING_LIBDIR_NATIVE} to
${STAGING_LIBDIR}.
3) Instead of copying llvm-config from ${STAGING_BINDIR_NATIVE} to
${STAGING_BINDIR},
directly make rust-target build dependent
on ${STAGING_BINDIR_NATIVE}/llvm-config.
Are there any better or cleaner alternatives to handle this situation?
Any suggestions would be greatly appreciated.
Regards,
Deepesh
>>>> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>>>>
>>>> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
>>>> -DLLVM_ENABLE_PIC=ON \
>>>> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
>>>> -DLLVM_BINDINGS_LIST='' \
>>>> + -DLLVM_INSTALL_UTILS=ON \
>>> tabs vs spaces inconsistenty. Please fix it.
>> Sure.
>>>> -DLLVM_ENABLE_FFI=ON \
>>>> -DLLVM_ENABLE_ZSTD=ON \
>>>> -DFFI_INCLUDE_DIR=$(pkg-config
>>>> --variable=includedir libffi) \
>>>> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
>>>> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
>>>> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
>>>> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
>>>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>> why is this change needed ?
>>>
>> I made this change to ensure that LLVM_TARGET_ARCH is set correctly to
>> the actual target architecture.
>> However, it appears that even without this change, the build completes
>> successfully for the target. I checked
>> and printed the value of HOST_ARCH and realized it was already set
>> correctly my mistake. This change is
>> not required.
> LLVM_TARGET_ARCH is different than the backend architecture which is
> denoted by LLVM_TARGETS_TO_BUILD
>
>>>> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
>>>> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
>>>> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
>>>> diff --git a/meta/recipes-devtools/clang/common-clang.inc
>>>> b/meta/recipes-devtools/clang/common-clang.inc
>>>> index bf3a63914a..c22e3c1b19 100644
>>>> --- a/meta/recipes-devtools/clang/common-clang.inc
>>>> +++ b/meta/recipes-devtools/clang/common-clang.inc
>>>> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
>>>> elif re.match('aarch64$', a): return
>>>> 'AArch64'
>>>> elif re.match('aarch64_be$', a): return
>>>> 'AArch64'
>>>> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
>>>> - elif re.match('riscv32$', a): return 'RISCV'
>>>> - elif re.match('riscv64$', a): return 'RISCV'
>>>> + elif re.match('riscv32$', a): return 'riscv32'
>>>> + elif re.match('riscv64$', a): return 'riscv64'
>>> This is representing LLVM backend name and not normal arch and clang
>>> uses RISCV for both rv64 and rv32
>>>
>> After this change were made,
>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>> do_configure of clang failed
>> with the following error:
>>
>> | CMake Error at cmake/config-ix.cmake:585 (message):
>> | Unknown architecture riscv
>>
>> Since this change is irrelevant
>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>
>> The below changes are also not needed.
>> - elif re.match('riscv32$', a): return 'RISCV'
>> - elif re.match('riscv64$', a): return 'RISCV'
>> + elif re.match('riscv32$', a): return 'riscv32'
>> + elif re.match('riscv64$', a): return 'riscv64'
>> - elif re.match('loongarch64$', a): return 'LoongArch'
>> + elif re.match('loongarch64$', a): return
>> 'loongarch64'
>>>> elif re.match('p(pc|owerpc)(|64)', a): return
>>>> 'PowerPC'
>>>> - elif re.match('loongarch64$', a): return
>>>> 'LoongArch'
>>>> + elif re.match('loongarch64$', a): return
>>>> 'loongarch64'
>>> same problem as above here
>>>
>>>> else:
>>>> bb.fatal("Unhandled architecture %s" % arch_val)
>>>> return ""
>>>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> index 5d804c7398..c2cb8f8829 100644
>>>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
>>>> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
>>>> inherit rust
>>>> inherit cargo_common
>>>>
>>>> -DEPENDS += "rust-llvm"
>>>> +DEPENDS += "ninja-native clang"
>>> I think using 'llvm' instead of 'clang' here is more appropriate to
>>> represent the dependency, clang provides llvm as well, it also ensures
>>> that when we split llvm out of clang recipe then you do not need to
>>> change it.
>> Sure will do that.
>>>> # native rust uses cargo/rustc from binary snapshots to bootstrap
>>>> # but everything else should use our native builds
>>>> DEPENDS:append:class-target = " cargo-native rust-native"
>>>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
>>>> 'stable', '', '-${RUST_CHANNEL}', d)
>>>>
>>>> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>>>>
>>>> -RUST_ALTERNATE_EXE_PATH ?=
>>>> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
>>>> -RUST_ALTERNATE_EXE_PATH_NATIVE =
>>>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
>>>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
>>>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>>>>
>>>> # We don't want to use bitbakes vendoring because the rust sources
>>>> do their
>>>> # own vendoring.
>>>> @@ -188,6 +188,16 @@ python do_configure() {
>>>> bb.build.exec_func("setup_cargo_environment", d)
>>>> }
>>>>
>>>> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
>>>> +#Since LLVM is built as a non-multilib component, the 'lib' directory
>>>> +#doesn't have any library files when multilibs enabled. So, copying
>>>> +#library files without impacting multilib behavior.
>>>> +do_compile:append:class-target() {
>>>> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
>>>> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
>>>> ${STAGING_DIR_TARGET}/usr/lib/.
>>>> +fi
>>>> +}
>>>> +
>>>> rust_runx () {
>>>> echo "COMPILE ${PN}" "$@"
>>>>
>>>> @@ -199,7 +209,7 @@ rust_runx () {
>>>> unset CXXFLAGS
>>>> unset CPPFLAGS
>>>>
>>>> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
>>>> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
>>>> -Clink-arg=-lzstd"
>>>>
>>>> # Copy the natively built llvm-config into the target so we can
>>>> run it. Horrible,
>>>> # but works!
>>>>
>>>>
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#224075):
>>>> https://lists.openembedded.org/g/openembedded-core/message/224075
>>>> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>>>> [raj.khem@gmail.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-10-07 11:10 ` Deepesh Varatharajan
@ 2025-10-07 20:17 ` Khem Raj
2025-10-08 12:49 ` Deepesh Varatharajan
0 siblings, 1 reply; 14+ messages in thread
From: Khem Raj @ 2025-10-07 20:17 UTC (permalink / raw)
To: Deepesh Varatharajan; +Cc: openembedded-core, Sundeep.Kokkonda
On Tue, Oct 7, 2025 at 4:10 AM Deepesh Varatharajan
<deepesh.varatharajan@windriver.com> wrote:
>
>
> On 02-10-2025 00:59, Khem Raj wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> > On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
> > <deepesh.varatharajan@windriver.com> wrote:
> >>
> >> On 26-09-2025 23:00, Khem Raj wrote:
> >>> CAUTION: This email comes from a non Wind River email account!
> >>> Do not click links or open attachments unless you recognize the sender
> >>> and know the content is safe.
> >>>
> >>> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
> >>> wrote:
> >>>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> >>>>
> >>>> Updated the Rust build to depend on Clang instead.
> >>>>
> >>>> *Summary of discussion with the rust upstream about using latest LLVM
> >>>> instead of Rust maintained LLVM fork.
> >>>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
> >>>>
> >>>>
> >>>> *Upstream LLVM is generally compatible:
> >>>> - Rust does support building with upstream (vanilla) LLVM, especially
> >>>> the latest
> >>>> major release and the one or two preceding ones.
> >>>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
> >>>>
> >>>>
> >>>> *Impact on Yocto Rust upgrades:
> >>>> - Rust upgrades shall always check for updates on rust forked llvm
> >>>> and backport
> >>>> the relevant patches to clang's llvm.
> >>>>
> >>>> *Regarding the rust forked llvm local patches:
> >>>> - There are no local patches on rust forked llvm other than the
> >>>> backported fixes
> >>>> from llvm master.
> >>>>
> >>>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
> >>>> this following
> >>>> diff otherwise we will get errors during link time.
> >>>>
> >>>> Setup in rust-llvm
> >>>> -DLLVM_ENABLE_ZLIB=OFF \
> >>>> -DLLVM_ENABLE_ZSTD=OFF \
> >>>> -DLLVM_ENABLE_FFI=OFF \
> >>>>
> >>>> Setup in clang
> >>>> -DLLVM_ENABLE_FFI=ON \
> >>>> -DLLVM_ENABLE_ZSTD=ON \
> >>>>
> >>>> *When multilibs enabled:
> >>>>
> >>>> llvm-config expects static libraries to be located in the lib
> >>>> directory rather than
> >>>> lib64. However, since LLVM is built as a non-multilib component, the
> >>>> lib directory
> >>>> doesn't contain any library files. To accommodate this without
> >>>> breaking multilib
> >>>> behavior, we copy the required library files appropriately.
> >>>>
> >>>> Previously, when we depended on rust-llvm, this worked because we
> >>>> specified:
> >>>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
> >>>>
> >>>> With this setup, llvm-config was installed inside
> >>>> ${libdir}/llvm-rust, which included
> >>>> its own bin and lib directories. Thus, llvm-config located in bin
> >>>> would correctly find
> >>>> the libraries in the adjacent lib directory.
> >>>>
> >>>> Even when multilib was enabled or not, llvm-config would still look
> >>>> for libraries under
> >>>> lib in this structure, so everything functioned as expected.
> >>>>
> >>>> *Changes needs to be done when llvm splits from clang:
> >>>> In rust recipe:
> >>>> Update the dependency from:
> >>>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
> >>>>
> >>>> In llvm recipe:
> >>>> Apply the same changes that were made in the Clang recipe, as those
> >>>> configurations have now been moved to the LLVM recipe after the split.
> >>>>
> >>>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> >>>> ---
> >>>> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
> >>>> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
> >>>> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
> >>>> 3 files changed, 19 insertions(+), 9 deletions(-)
> >>>>
> >>>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
> >>>> b/meta/recipes-devtools/clang/clang_git.bb
> >>>> index 53bca1c24f..3e117b308b 100644
> >>>> --- a/meta/recipes-devtools/clang/clang_git.bb
> >>>> +++ b/meta/recipes-devtools/clang/clang_git.bb
> >>>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
> >>>> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
> >>>> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
> >>>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
> >>>> LLVM_TARGETS_TO_BUILD ?=
> >>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
> >>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
> >>>> d)};BPF;${LLVM_TARGETS_GPU}"
> >>> compiler we build is building all architectures once because it then
> >>> reuses same compiler for all cross compilers just by creating canonical
> >>> symlinks, this change is not going to work.
> >> We removed this line because, for the clang class-target build, LLVM was
> >> only compiling
> >> libraries for the target architecture and GPU targets and kept inside
> >> the target
> >> sysroot. Since we use the llvm-config built for the class-target during
> >> the Rust target build,
> >> it was failing due to the absence of static libraries for other
> >> architectures in the target sysroot.
> >> Here the target is x86 arch. So, it compiled libraries for x86 arch only
> >> and we can see the error
> >> for missing libraries for other archs as below:
> >>
> >> | --- stderr
> >> | llvm-config: error: missing:
> >> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
> >> | llvm-config: error: missing:
> >> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
> >> | llvm-config: error: missing:
> >> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
> >> | llvm-config: error: missing:
> >> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
> >> | llvm-config: error: missing:
> >> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
> >> | llvm-config: error: missing:
> >> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
> >> .
> >> .
> >> .
> >> | thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
> >> | command did not execute successfully:
> >> "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
> >>
> >> "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
> >> "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
> >> "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
> >> | expected success, got: exit status: 1
> >>
> >> Maybe instead of removing. Can we change
> >> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
> >> d)};BPF;${LLVM_TARGETS_GPU}"
> >> to
> >> -LLVM_TARGETS_TO_BUILD:class-target ?=
> >> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
> >> ?
> > I think we don't want to build all the general arches for target, only
> > the target arch and gpu arches
> > is what is needed. If we build all the backends, that will slow down
> > things. But if target rust needs
> > to support all the architectures to generate code for, then we don't
> > have much choice. Can you see
> > if we are ok to just target one architecture ?
> The issue doesn't originate from Rust directly, it stems from the
> following logic
> in the Rust recipe, where the natively built llvm-config is copied into
> the target sysroot:
>
> # Copy the natively built llvm-config into the target so we can run
> it. Horrible,
> # but works!
> if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
> ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
> mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
> cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
> if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
> patchelf --set-rpath \$ORIGIN/../../../../../`basename
> ${STAGING_DIR_NATIVE}`${libdir_native} ${RUST_ALTERNATE_EXE_PATH}
> else
> patchelf --remove-rpath ${RUST_ALTERNATE_EXE_PATH}
> fi
> fi
>
> Since llvm-config is being copied from ${STAGING_BINDIR_NATIVE} to
> ${STAGING_BINDIR},
> and the native Clang was built with the following configuration:
> LLVM_TARGETS_TO_BUILD ?=
> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>
> The copied llvm-config will expect the corresponding target libraries to
> be present in the ${STAGING_LIBDIR}.
>
> To address this issue, as far as I can see we have three options.
>
> 1) Build clang-target also for all mentioned archs for clang-native.
> 2) Copy all the required libraries from ${STAGING_LIBDIR_NATIVE} to
> ${STAGING_LIBDIR}.
> 3) Instead of copying llvm-config from ${STAGING_BINDIR_NATIVE} to
> ${STAGING_BINDIR},
> directly make rust-target build dependent
> on ${STAGING_BINDIR_NATIVE}/llvm-config.
>
> Are there any better or cleaner alternatives to handle this situation?
> Any suggestions would be greatly appreciated.
I would prefer option 3 since these libs are inherently cross built,
it perhaps does not matter if
they come from native or cross built clang. But that would have been
default before this copy dance
was done. I might have forgotten now, if this was done specifically to address a
case which we can testout now.
>
> Regards,
> Deepesh
>
> >>>> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
> >>>>
> >>>> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
> >>>> -DLLVM_ENABLE_PIC=ON \
> >>>> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
> >>>> -DLLVM_BINDINGS_LIST='' \
> >>>> + -DLLVM_INSTALL_UTILS=ON \
> >>> tabs vs spaces inconsistenty. Please fix it.
> >> Sure.
> >>>> -DLLVM_ENABLE_FFI=ON \
> >>>> -DLLVM_ENABLE_ZSTD=ON \
> >>>> -DFFI_INCLUDE_DIR=$(pkg-config
> >>>> --variable=includedir libffi) \
> >>>> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
> >>>> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
> >>>> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
> >>>> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
> >>>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> >>>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
> >>> why is this change needed ?
> >>>
> >> I made this change to ensure that LLVM_TARGET_ARCH is set correctly to
> >> the actual target architecture.
> >> However, it appears that even without this change, the build completes
> >> successfully for the target. I checked
> >> and printed the value of HOST_ARCH and realized it was already set
> >> correctly my mistake. This change is
> >> not required.
> > LLVM_TARGET_ARCH is different than the backend architecture which is
> > denoted by LLVM_TARGETS_TO_BUILD
> >
> >>>> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
> >>>> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
> >>>> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
> >>>> diff --git a/meta/recipes-devtools/clang/common-clang.inc
> >>>> b/meta/recipes-devtools/clang/common-clang.inc
> >>>> index bf3a63914a..c22e3c1b19 100644
> >>>> --- a/meta/recipes-devtools/clang/common-clang.inc
> >>>> +++ b/meta/recipes-devtools/clang/common-clang.inc
> >>>> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
> >>>> elif re.match('aarch64$', a): return
> >>>> 'AArch64'
> >>>> elif re.match('aarch64_be$', a): return
> >>>> 'AArch64'
> >>>> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
> >>>> - elif re.match('riscv32$', a): return 'RISCV'
> >>>> - elif re.match('riscv64$', a): return 'RISCV'
> >>>> + elif re.match('riscv32$', a): return 'riscv32'
> >>>> + elif re.match('riscv64$', a): return 'riscv64'
> >>> This is representing LLVM backend name and not normal arch and clang
> >>> uses RISCV for both rv64 and rv32
> >>>
> >> After this change were made,
> >> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> >> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
> >> do_configure of clang failed
> >> with the following error:
> >>
> >> | CMake Error at cmake/config-ix.cmake:585 (message):
> >> | Unknown architecture riscv
> >>
> >> Since this change is irrelevant
> >> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
> >> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
> >>
> >> The below changes are also not needed.
> >> - elif re.match('riscv32$', a): return 'RISCV'
> >> - elif re.match('riscv64$', a): return 'RISCV'
> >> + elif re.match('riscv32$', a): return 'riscv32'
> >> + elif re.match('riscv64$', a): return 'riscv64'
> >> - elif re.match('loongarch64$', a): return 'LoongArch'
> >> + elif re.match('loongarch64$', a): return
> >> 'loongarch64'
> >>>> elif re.match('p(pc|owerpc)(|64)', a): return
> >>>> 'PowerPC'
> >>>> - elif re.match('loongarch64$', a): return
> >>>> 'LoongArch'
> >>>> + elif re.match('loongarch64$', a): return
> >>>> 'loongarch64'
> >>> same problem as above here
> >>>
> >>>> else:
> >>>> bb.fatal("Unhandled architecture %s" % arch_val)
> >>>> return ""
> >>>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
> >>>> b/meta/recipes-devtools/rust/rust_1.90.0.bb
> >>>> index 5d804c7398..c2cb8f8829 100644
> >>>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
> >>>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
> >>>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
> >>>> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
> >>>> inherit rust
> >>>> inherit cargo_common
> >>>>
> >>>> -DEPENDS += "rust-llvm"
> >>>> +DEPENDS += "ninja-native clang"
> >>> I think using 'llvm' instead of 'clang' here is more appropriate to
> >>> represent the dependency, clang provides llvm as well, it also ensures
> >>> that when we split llvm out of clang recipe then you do not need to
> >>> change it.
> >> Sure will do that.
> >>>> # native rust uses cargo/rustc from binary snapshots to bootstrap
> >>>> # but everything else should use our native builds
> >>>> DEPENDS:append:class-target = " cargo-native rust-native"
> >>>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
> >>>> 'stable', '', '-${RUST_CHANNEL}', d)
> >>>>
> >>>> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
> >>>>
> >>>> -RUST_ALTERNATE_EXE_PATH ?=
> >>>> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
> >>>> -RUST_ALTERNATE_EXE_PATH_NATIVE =
> >>>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
> >>>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
> >>>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
> >>>>
> >>>> # We don't want to use bitbakes vendoring because the rust sources
> >>>> do their
> >>>> # own vendoring.
> >>>> @@ -188,6 +188,16 @@ python do_configure() {
> >>>> bb.build.exec_func("setup_cargo_environment", d)
> >>>> }
> >>>>
> >>>> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
> >>>> +#Since LLVM is built as a non-multilib component, the 'lib' directory
> >>>> +#doesn't have any library files when multilibs enabled. So, copying
> >>>> +#library files without impacting multilib behavior.
> >>>> +do_compile:append:class-target() {
> >>>> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
> >>>> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
> >>>> ${STAGING_DIR_TARGET}/usr/lib/.
> >>>> +fi
> >>>> +}
> >>>> +
> >>>> rust_runx () {
> >>>> echo "COMPILE ${PN}" "$@"
> >>>>
> >>>> @@ -199,7 +209,7 @@ rust_runx () {
> >>>> unset CXXFLAGS
> >>>> unset CPPFLAGS
> >>>>
> >>>> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
> >>>> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
> >>>> -Clink-arg=-lzstd"
> >>>>
> >>>> # Copy the natively built llvm-config into the target so we can
> >>>> run it. Horrible,
> >>>> # but works!
> >>>>
> >>>>
> >>>>
> >>>> -=-=-=-=-=-=-=-=-=-=-=-
> >>>> Links: You receive all messages sent to this group.
> >>>> View/Reply Online (#224075):
> >>>> https://lists.openembedded.org/g/openembedded-core/message/224075
> >>>> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
> >>>> Group Owner: openembedded-core+owner@lists.openembedded.org
> >>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
> >>>> [raj.khem@gmail.com]
> >>>> -=-=-=-=-=-=-=-=-=-=-=-
> >>>>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-10-07 20:17 ` Khem Raj
@ 2025-10-08 12:49 ` Deepesh Varatharajan
2025-10-08 12:54 ` Richard Purdie
0 siblings, 1 reply; 14+ messages in thread
From: Deepesh Varatharajan @ 2025-10-08 12:49 UTC (permalink / raw)
To: Khem Raj; +Cc: openembedded-core, Sundeep.Kokkonda
On 08-10-2025 01:47, Khem Raj wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Tue, Oct 7, 2025 at 4:10 AM Deepesh Varatharajan
> <deepesh.varatharajan@windriver.com> wrote:
>>
>> On 02-10-2025 00:59, Khem Raj wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
>>> <deepesh.varatharajan@windriver.com> wrote:
>>>> On 26-09-2025 23:00, Khem Raj wrote:
>>>>> CAUTION: This email comes from a non Wind River email account!
>>>>> Do not click links or open attachments unless you recognize the sender
>>>>> and know the content is safe.
>>>>>
>>>>> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
>>>>> wrote:
>>>>>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>>>
>>>>>> Updated the Rust build to depend on Clang instead.
>>>>>>
>>>>>> *Summary of discussion with the rust upstream about using latest LLVM
>>>>>> instead of Rust maintained LLVM fork.
>>>>>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>>>>>
>>>>>>
>>>>>> *Upstream LLVM is generally compatible:
>>>>>> - Rust does support building with upstream (vanilla) LLVM, especially
>>>>>> the latest
>>>>>> major release and the one or two preceding ones.
>>>>>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>>>>>
>>>>>>
>>>>>> *Impact on Yocto Rust upgrades:
>>>>>> - Rust upgrades shall always check for updates on rust forked llvm
>>>>>> and backport
>>>>>> the relevant patches to clang's llvm.
>>>>>>
>>>>>> *Regarding the rust forked llvm local patches:
>>>>>> - There are no local patches on rust forked llvm other than the
>>>>>> backported fixes
>>>>>> from llvm master.
>>>>>>
>>>>>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
>>>>>> this following
>>>>>> diff otherwise we will get errors during link time.
>>>>>>
>>>>>> Setup in rust-llvm
>>>>>> -DLLVM_ENABLE_ZLIB=OFF \
>>>>>> -DLLVM_ENABLE_ZSTD=OFF \
>>>>>> -DLLVM_ENABLE_FFI=OFF \
>>>>>>
>>>>>> Setup in clang
>>>>>> -DLLVM_ENABLE_FFI=ON \
>>>>>> -DLLVM_ENABLE_ZSTD=ON \
>>>>>>
>>>>>> *When multilibs enabled:
>>>>>>
>>>>>> llvm-config expects static libraries to be located in the lib
>>>>>> directory rather than
>>>>>> lib64. However, since LLVM is built as a non-multilib component, the
>>>>>> lib directory
>>>>>> doesn't contain any library files. To accommodate this without
>>>>>> breaking multilib
>>>>>> behavior, we copy the required library files appropriately.
>>>>>>
>>>>>> Previously, when we depended on rust-llvm, this worked because we
>>>>>> specified:
>>>>>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>>>>>
>>>>>> With this setup, llvm-config was installed inside
>>>>>> ${libdir}/llvm-rust, which included
>>>>>> its own bin and lib directories. Thus, llvm-config located in bin
>>>>>> would correctly find
>>>>>> the libraries in the adjacent lib directory.
>>>>>>
>>>>>> Even when multilib was enabled or not, llvm-config would still look
>>>>>> for libraries under
>>>>>> lib in this structure, so everything functioned as expected.
>>>>>>
>>>>>> *Changes needs to be done when llvm splits from clang:
>>>>>> In rust recipe:
>>>>>> Update the dependency from:
>>>>>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>>>>>>
>>>>>> In llvm recipe:
>>>>>> Apply the same changes that were made in the Clang recipe, as those
>>>>>> configurations have now been moved to the LLVM recipe after the split.
>>>>>>
>>>>>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>>> ---
>>>>>> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
>>>>>> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
>>>>>> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
>>>>>> 3 files changed, 19 insertions(+), 9 deletions(-)
>>>>>>
>>>>>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
>>>>>> b/meta/recipes-devtools/clang/clang_git.bb
>>>>>> index 53bca1c24f..3e117b308b 100644
>>>>>> --- a/meta/recipes-devtools/clang/clang_git.bb
>>>>>> +++ b/meta/recipes-devtools/clang/clang_git.bb
>>>>>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>>>>>> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>>>>>> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
>>>>>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>>>>>> LLVM_TARGETS_TO_BUILD ?=
>>>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>>>> compiler we build is building all architectures once because it then
>>>>> reuses same compiler for all cross compilers just by creating canonical
>>>>> symlinks, this change is not going to work.
>>>> We removed this line because, for the clang class-target build, LLVM was
>>>> only compiling
>>>> libraries for the target architecture and GPU targets and kept inside
>>>> the target
>>>> sysroot. Since we use the llvm-config built for the class-target during
>>>> the Rust target build,
>>>> it was failing due to the absence of static libraries for other
>>>> architectures in the target sysroot.
>>>> Here the target is x86 arch. So, it compiled libraries for x86 arch only
>>>> and we can see the error
>>>> for missing libraries for other archs as below:
>>>>
>>>> | --- stderr
>>>> | llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
>>>> | llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
>>>> | llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
>>>> | llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
>>>> | llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
>>>> | llvm-config: error: missing:
>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
>>>> .
>>>> .
>>>> .
>>>> | thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
>>>> | command did not execute successfully:
>>>> "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
>>>>
>>>> "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
>>>> "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
>>>> "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
>>>> | expected success, got: exit status: 1
>>>>
>>>> Maybe instead of removing. Can we change
>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>>> to
>>>> -LLVM_TARGETS_TO_BUILD:class-target ?=
>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
>>>> ?
>>> I think we don't want to build all the general arches for target, only
>>> the target arch and gpu arches
>>> is what is needed. If we build all the backends, that will slow down
>>> things. But if target rust needs
>>> to support all the architectures to generate code for, then we don't
>>> have much choice. Can you see
>>> if we are ok to just target one architecture ?
>> The issue doesn't originate from Rust directly, it stems from the
>> following logic
>> in the Rust recipe, where the natively built llvm-config is copied into
>> the target sysroot:
>>
>> # Copy the natively built llvm-config into the target so we can run
>> it. Horrible,
>> # but works!
>> if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
>> ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
>> mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
>> cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
>> if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
>> patchelf --set-rpath \$ORIGIN/../../../../../`basename
>> ${STAGING_DIR_NATIVE}`${libdir_native} ${RUST_ALTERNATE_EXE_PATH}
>> else
>> patchelf --remove-rpath ${RUST_ALTERNATE_EXE_PATH}
>> fi
>> fi
>>
>> Since llvm-config is being copied from ${STAGING_BINDIR_NATIVE} to
>> ${STAGING_BINDIR},
>> and the native Clang was built with the following configuration:
>> LLVM_TARGETS_TO_BUILD ?=
>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>
>> The copied llvm-config will expect the corresponding target libraries to
>> be present in the ${STAGING_LIBDIR}.
>>
>> To address this issue, as far as I can see we have three options.
>>
>> 1) Build clang-target also for all mentioned archs for clang-native.
>> 2) Copy all the required libraries from ${STAGING_LIBDIR_NATIVE} to
>> ${STAGING_LIBDIR}.
>> 3) Instead of copying llvm-config from ${STAGING_BINDIR_NATIVE} to
>> ${STAGING_BINDIR},
>> directly make rust-target build dependent
>> on ${STAGING_BINDIR_NATIVE}/llvm-config.
>>
>> Are there any better or cleaner alternatives to handle this situation?
>> Any suggestions would be greatly appreciated.
> I would prefer option 3 since these libs are inherently cross built,
> it perhaps does not matter if
> they come from native or cross built clang. But that would have been
> default before this copy dance
> was done. I might have forgotten now, if this was done specifically to address a
> case which we can testout now.
Making Rust depend directly on the native llvm-config is not feasible
because librustc_llvm*.rlib
is built for the host architecture(x86-64) . While this works if the
target is x86-64 or x86, for
other architectures we get the following error:
we encounter the following error:
ld: .../librustc_llvm-f50ec2a9934e24d2.rlib: error adding symbols: file
format not recognized
collect2: error: ld returned 1 exit status
Therefore, copying the natively built llvm-config into the target
sysroot and unsetting the RPATH
is currently the best approach, and that’s what we are doing. However,
even after unsetting the
RPATH, the native llvm-config still references to static libraries for
other architectures during
compile time. To fix this, those libraries must be present in the target
sysroot.
From what I see, the only feasible solution is to build LLVM target
libraries for all supported
architectures. Copying these libraries from the native sysroot is not
feasible because the native
sysroot contains libs for all OE-core targets, while the target sysroot
contains libs only for its
specific target—copying would cause conflicts.
Below are the build timings and space consumption after splitting clang
and llvm.
LLVM built for only target:
bitbake llvm
real 25m3.899s
user 0m6.904s
sys 0m0.673s
bitbake clang
real 28m28.827s
user 0m13.178s
sys 0m1.193s
../poky/build/tmp/work/x86-64-v3-poky-linux/clang/21.1.2 > du -sh .
5.4G
../poky/build/tmp/work/x86-64-v3-poky-linux/llvm/21.1.2 > du -sh .
5.6G
LLVM built for all Oe-core supported targets:
bitbake llvm
real 25m2.809s
user 0m7.017s
sys 0m0.679s
bitbake clang
real 31m7.457s
user 0m14.471s
sys 0m1.544s
../poky/build/tmp/work/x86-64-v3-poky-linux/clang/21.1.2 > du -sh .
5.6G
../poky/build/tmp/work/x86-64-v3-poky-linux/llvm/21.1.2 > du -sh .
6.0G
>> Regards,
>> Deepesh
>>
>>>>>> LLVM_EXPERIMENTAL_TARGETS_TO_BUILD ?= ""
>>>>>>
>>>>>> @@ -107,6 +106,7 @@ EXTRA_OECMAKE += "-DLLVM_ENABLE_ASSERTIONS=OFF \
>>>>>> -DLLVM_ENABLE_PIC=ON \
>>>>>> -DCLANG_DEFAULT_PIE_ON_LINUX=ON \
>>>>>> -DLLVM_BINDINGS_LIST='' \
>>>>>> + -DLLVM_INSTALL_UTILS=ON \
>>>>> tabs vs spaces inconsistenty. Please fix it.
>>>> Sure.
>>>>>> -DLLVM_ENABLE_FFI=ON \
>>>>>> -DLLVM_ENABLE_ZSTD=ON \
>>>>>> -DFFI_INCLUDE_DIR=$(pkg-config
>>>>>> --variable=includedir libffi) \
>>>>>> @@ -137,7 +137,7 @@ EXTRA_OECMAKE:append:class-target = "\
>>>>>> -DCMAKE_AR=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-ar \
>>>>>> -DCMAKE_NM=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-nm \
>>>>>> -DCMAKE_STRIP=${STAGING_BINDIR_TOOLCHAIN}/${TARGET_PREFIX}llvm-strip \
>>>>>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>>>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>>>> why is this change needed ?
>>>>>
>>>> I made this change to ensure that LLVM_TARGET_ARCH is set correctly to
>>>> the actual target architecture.
>>>> However, it appears that even without this change, the build completes
>>>> successfully for the target. I checked
>>>> and printed the value of HOST_ARCH and realized it was already set
>>>> correctly my mistake. This change is
>>>> not required.
>>> LLVM_TARGET_ARCH is different than the backend architecture which is
>>> denoted by LLVM_TARGETS_TO_BUILD
>>>
>>>>>> -DLLVM_DEFAULT_TARGET_TRIPLE=${TARGET_SYS}${HF} \
>>>>>> -DLLVM_HOST_TRIPLE=${TARGET_SYS}${HF} \
>>>>>> -DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX} \
>>>>>> diff --git a/meta/recipes-devtools/clang/common-clang.inc
>>>>>> b/meta/recipes-devtools/clang/common-clang.inc
>>>>>> index bf3a63914a..c22e3c1b19 100644
>>>>>> --- a/meta/recipes-devtools/clang/common-clang.inc
>>>>>> +++ b/meta/recipes-devtools/clang/common-clang.inc
>>>>>> @@ -30,10 +30,10 @@ def get_clang_arch(bb, d, arch_var):
>>>>>> elif re.match('aarch64$', a): return
>>>>>> 'AArch64'
>>>>>> elif re.match('aarch64_be$', a): return
>>>>>> 'AArch64'
>>>>>> elif re.match('mips(isa|)(32|64|)(r6|)(el|)$', a): return 'Mips'
>>>>>> - elif re.match('riscv32$', a): return 'RISCV'
>>>>>> - elif re.match('riscv64$', a): return 'RISCV'
>>>>>> + elif re.match('riscv32$', a): return 'riscv32'
>>>>>> + elif re.match('riscv64$', a): return 'riscv64'
>>>>> This is representing LLVM backend name and not normal arch and clang
>>>>> uses RISCV for both rv64 and rv32
>>>>>
>>>> After this change were made,
>>>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>>> do_configure of clang failed
>>>> with the following error:
>>>>
>>>> | CMake Error at cmake/config-ix.cmake:585 (message):
>>>> | Unknown architecture riscv
>>>>
>>>> Since this change is irrelevant
>>>> - -DLLVM_TARGET_ARCH=${HOST_ARCH} \
>>>> + -DLLVM_TARGET_ARCH=${@get_clang_target_arch(bb, d)} \
>>>>
>>>> The below changes are also not needed.
>>>> - elif re.match('riscv32$', a): return 'RISCV'
>>>> - elif re.match('riscv64$', a): return 'RISCV'
>>>> + elif re.match('riscv32$', a): return 'riscv32'
>>>> + elif re.match('riscv64$', a): return 'riscv64'
>>>> - elif re.match('loongarch64$', a): return 'LoongArch'
>>>> + elif re.match('loongarch64$', a): return
>>>> 'loongarch64'
>>>>>> elif re.match('p(pc|owerpc)(|64)', a): return
>>>>>> 'PowerPC'
>>>>>> - elif re.match('loongarch64$', a): return
>>>>>> 'LoongArch'
>>>>>> + elif re.match('loongarch64$', a): return
>>>>>> 'loongarch64'
>>>>> same problem as above here
>>>>>
>>>>>> else:
>>>>>> bb.fatal("Unhandled architecture %s" % arch_val)
>>>>>> return ""
>>>>>> diff --git a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> index 5d804c7398..c2cb8f8829 100644
>>>>>> --- a/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> +++ b/meta/recipes-devtools/rust/rust_1.90.0.bb
>>>>>> @@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
>>>>>> "file://COPYRIGHT;md5=11a3899825f4376896e438c8c753f8dc"
>>>>>> inherit rust
>>>>>> inherit cargo_common
>>>>>>
>>>>>> -DEPENDS += "rust-llvm"
>>>>>> +DEPENDS += "ninja-native clang"
>>>>> I think using 'llvm' instead of 'clang' here is more appropriate to
>>>>> represent the dependency, clang provides llvm as well, it also ensures
>>>>> that when we split llvm out of clang recipe then you do not need to
>>>>> change it.
>>>> Sure will do that.
>>>>>> # native rust uses cargo/rustc from binary snapshots to bootstrap
>>>>>> # but everything else should use our native builds
>>>>>> DEPENDS:append:class-target = " cargo-native rust-native"
>>>>>> @@ -28,8 +28,8 @@ PV .= "${@bb.utils.contains('RUST_CHANNEL',
>>>>>> 'stable', '', '-${RUST_CHANNEL}', d)
>>>>>>
>>>>>> export FORCE_CRATE_HASH = "${BB_TASKHASH}"
>>>>>>
>>>>>> -RUST_ALTERNATE_EXE_PATH ?=
>>>>>> "${STAGING_LIBDIR}/llvm-rust/bin/llvm-config"
>>>>>> -RUST_ALTERNATE_EXE_PATH_NATIVE =
>>>>>> "${STAGING_LIBDIR_NATIVE}/llvm-rust/bin/llvm-config"
>>>>>> +RUST_ALTERNATE_EXE_PATH ?= "${STAGING_BINDIR}/llvm-config"
>>>>>> +RUST_ALTERNATE_EXE_PATH_NATIVE = "${STAGING_BINDIR_NATIVE}/llvm-config"
>>>>>>
>>>>>> # We don't want to use bitbakes vendoring because the rust sources
>>>>>> do their
>>>>>> # own vendoring.
>>>>>> @@ -188,6 +188,16 @@ python do_configure() {
>>>>>> bb.build.exec_func("setup_cargo_environment", d)
>>>>>> }
>>>>>>
>>>>>> +#llvm-config expecting static libraries in 'lib' instead of 'lib64'.
>>>>>> +#Since LLVM is built as a non-multilib component, the 'lib' directory
>>>>>> +#doesn't have any library files when multilibs enabled. So, copying
>>>>>> +#library files without impacting multilib behavior.
>>>>>> +do_compile:append:class-target() {
>>>>>> +if [ -d ${STAGING_DIR_TARGET}/usr/lib64 ]; then
>>>>>> + cp ${STAGING_DIR_TARGET}/usr/lib64/libLLVM*.a
>>>>>> ${STAGING_DIR_TARGET}/usr/lib/.
>>>>>> +fi
>>>>>> +}
>>>>>> +
>>>>>> rust_runx () {
>>>>>> echo "COMPILE ${PN}" "$@"
>>>>>>
>>>>>> @@ -199,7 +209,7 @@ rust_runx () {
>>>>>> unset CXXFLAGS
>>>>>> unset CPPFLAGS
>>>>>>
>>>>>> - export RUSTFLAGS="${RUST_DEBUG_REMAP}"
>>>>>> + export RUSTFLAGS="${RUST_DEBUG_REMAP} -Clink-arg=-lz
>>>>>> -Clink-arg=-lzstd"
>>>>>>
>>>>>> # Copy the natively built llvm-config into the target so we can
>>>>>> run it. Horrible,
>>>>>> # but works!
>>>>>>
>>>>>>
>>>>>>
>>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>> Links: You receive all messages sent to this group.
>>>>>> View/Reply Online (#224075):
>>>>>> https://lists.openembedded.org/g/openembedded-core/message/224075
>>>>>> Mute This Topic: https://lists.openembedded.org/mt/115446166/1997914
>>>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>>>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
>>>>>> [raj.khem@gmail.com]
>>>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>>>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-10-08 12:49 ` Deepesh Varatharajan
@ 2025-10-08 12:54 ` Richard Purdie
2025-10-10 9:27 ` Deepesh Varatharajan
0 siblings, 1 reply; 14+ messages in thread
From: Richard Purdie @ 2025-10-08 12:54 UTC (permalink / raw)
To: deepesh.varatharajan, Khem Raj, Ross Burton
Cc: openembedded-core, Sundeep.Kokkonda
On Wed, 2025-10-08 at 18:19 +0530, Varatharajan, Deepesh via lists.openembedded.org wrote:
>
> On 08-10-2025 01:47, Khem Raj wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> > On Tue, Oct 7, 2025 at 4:10 AM Deepesh Varatharajan
> > <deepesh.varatharajan@windriver.com> wrote:
> > >
> > > On 02-10-2025 00:59, Khem Raj wrote:
> > > > CAUTION: This email comes from a non Wind River email account!
> > > > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> > > >
> > > > On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
> > > > <deepesh.varatharajan@windriver.com> wrote:
> > > > > On 26-09-2025 23:00, Khem Raj wrote:
> > > > > > CAUTION: This email comes from a non Wind River email account!
> > > > > > Do not click links or open attachments unless you recognize the sender
> > > > > > and know the content is safe.
> > > > > >
> > > > > > On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
> > > > > > wrote:
> > > > > > > From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> > > > > > >
> > > > > > > Updated the Rust build to depend on Clang instead.
> > > > > > >
> > > > > > > *Summary of discussion with the rust upstream about using latest LLVM
> > > > > > > instead of Rust maintained LLVM fork.
> > > > > > > https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
> > > > > > >
> > > > > > >
> > > > > > > *Upstream LLVM is generally compatible:
> > > > > > > - Rust does support building with upstream (vanilla) LLVM, especially
> > > > > > > the latest
> > > > > > > major release and the one or two preceding ones.
> > > > > > > https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
> > > > > > >
> > > > > > >
> > > > > > > *Impact on Yocto Rust upgrades:
> > > > > > > - Rust upgrades shall always check for updates on rust forked llvm
> > > > > > > and backport
> > > > > > > the relevant patches to clang's llvm.
> > > > > > >
> > > > > > > *Regarding the rust forked llvm local patches:
> > > > > > > - There are no local patches on rust forked llvm other than the
> > > > > > > backported fixes
> > > > > > > from llvm master.
> > > > > > >
> > > > > > > *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
> > > > > > > this following
> > > > > > > diff otherwise we will get errors during link time.
> > > > > > >
> > > > > > > Setup in rust-llvm
> > > > > > > -DLLVM_ENABLE_ZLIB=OFF \
> > > > > > > -DLLVM_ENABLE_ZSTD=OFF \
> > > > > > > -DLLVM_ENABLE_FFI=OFF \
> > > > > > >
> > > > > > > Setup in clang
> > > > > > > -DLLVM_ENABLE_FFI=ON \
> > > > > > > -DLLVM_ENABLE_ZSTD=ON \
> > > > > > >
> > > > > > > *When multilibs enabled:
> > > > > > >
> > > > > > > llvm-config expects static libraries to be located in the lib
> > > > > > > directory rather than
> > > > > > > lib64. However, since LLVM is built as a non-multilib component, the
> > > > > > > lib directory
> > > > > > > doesn't contain any library files. To accommodate this without
> > > > > > > breaking multilib
> > > > > > > behavior, we copy the required library files appropriately.
> > > > > > >
> > > > > > > Previously, when we depended on rust-llvm, this worked because we
> > > > > > > specified:
> > > > > > > -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
> > > > > > >
> > > > > > > With this setup, llvm-config was installed inside
> > > > > > > ${libdir}/llvm-rust, which included
> > > > > > > its own bin and lib directories. Thus, llvm-config located in bin
> > > > > > > would correctly find
> > > > > > > the libraries in the adjacent lib directory.
> > > > > > >
> > > > > > > Even when multilib was enabled or not, llvm-config would still look
> > > > > > > for libraries under
> > > > > > > lib in this structure, so everything functioned as expected.
> > > > > > >
> > > > > > > *Changes needs to be done when llvm splits from clang:
> > > > > > > In rust recipe:
> > > > > > > Update the dependency from:
> > > > > > > DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
> > > > > > >
> > > > > > > In llvm recipe:
> > > > > > > Apply the same changes that were made in the Clang recipe, as those
> > > > > > > configurations have now been moved to the LLVM recipe after the split.
> > > > > > >
> > > > > > > Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
> > > > > > > ---
> > > > > > > meta/recipes-devtools/clang/clang_git.bb | 4 ++--
> > > > > > > meta/recipes-devtools/clang/common-clang.inc | 6 +++---
> > > > > > > meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
> > > > > > > 3 files changed, 19 insertions(+), 9 deletions(-)
> > > > > > >
> > > > > > > diff --git a/meta/recipes-devtools/clang/clang_git.bb
> > > > > > > b/meta/recipes-devtools/clang/clang_git.bb
> > > > > > > index 53bca1c24f..3e117b308b 100644
> > > > > > > --- a/meta/recipes-devtools/clang/clang_git.bb
> > > > > > > +++ b/meta/recipes-devtools/clang/clang_git.bb
> > > > > > > @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
> > > > > > > # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
> > > > > > > LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
> > > > > > > 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
> > > > > > > LLVM_TARGETS_TO_BUILD ?=
> > > > > > > "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
> > > > > > > -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
> > > > > > > d)};BPF;${LLVM_TARGETS_GPU}"
> > > > > > compiler we build is building all architectures once because it then
> > > > > > reuses same compiler for all cross compilers just by creating canonical
> > > > > > symlinks, this change is not going to work.
> > > > > We removed this line because, for the clang class-target build, LLVM was
> > > > > only compiling
> > > > > libraries for the target architecture and GPU targets and kept inside
> > > > > the target
> > > > > sysroot. Since we use the llvm-config built for the class-target during
> > > > > the Rust target build,
> > > > > it was failing due to the absence of static libraries for other
> > > > > architectures in the target sysroot.
> > > > > Here the target is x86 arch. So, it compiled libraries for x86 arch only
> > > > > and we can see the error
> > > > > for missing libraries for other archs as below:
> > > > >
> > > > > > --- stderr
> > > > > > llvm-config: error: missing:
> > > > > poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
> > > > > > llvm-config: error: missing:
> > > > > poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
> > > > > > llvm-config: error: missing:
> > > > > poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
> > > > > > llvm-config: error: missing:
> > > > > poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
> > > > > > llvm-config: error: missing:
> > > > > poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
> > > > > > llvm-config: error: missing:
> > > > > poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
> > > > > .
> > > > > .
> > > > > .
> > > > > > thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
> > > > > > command did not execute successfully:
> > > > > "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
> > > > >
> > > > > "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
> > > > > "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
> > > > > "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
> > > > > > expected success, got: exit status: 1
> > > > >
> > > > > Maybe instead of removing. Can we change
> > > > > -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
> > > > > d)};BPF;${LLVM_TARGETS_GPU}"
> > > > > to
> > > > > -LLVM_TARGETS_TO_BUILD:class-target ?=
> > > > > "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
> > > > > ?
> > > > I think we don't want to build all the general arches for target, only
> > > > the target arch and gpu arches
> > > > is what is needed. If we build all the backends, that will slow down
> > > > things. But if target rust needs
> > > > to support all the architectures to generate code for, then we don't
> > > > have much choice. Can you see
> > > > if we are ok to just target one architecture ?
> > > The issue doesn't originate from Rust directly, it stems from the
> > > following logic
> > > in the Rust recipe, where the natively built llvm-config is copied into
> > > the target sysroot:
> > >
> > > # Copy the natively built llvm-config into the target so we can run
> > > it. Horrible,
> > > # but works!
> > > if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
> > > ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
> > > mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
> > > cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
> > > if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
> > > patchelf --set-rpath \$ORIGIN/../../../../../`basename
> > > ${STAGING_DIR_NATIVE}`${libdir_native} ${RUST_ALTERNATE_EXE_PATH}
> > > else
> > > patchelf --remove-rpath ${RUST_ALTERNATE_EXE_PATH}
> > > fi
> > > fi
> > >
> > > Since llvm-config is being copied from ${STAGING_BINDIR_NATIVE} to
> > > ${STAGING_BINDIR},
> > > and the native Clang was built with the following configuration:
> > > LLVM_TARGETS_TO_BUILD ?=
> > > "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
> > >
> > > The copied llvm-config will expect the corresponding target libraries to
> > > be present in the ${STAGING_LIBDIR}.
> > >
> > > To address this issue, as far as I can see we have three options.
> > >
> > > 1) Build clang-target also for all mentioned archs for clang-native.
> > > 2) Copy all the required libraries from ${STAGING_LIBDIR_NATIVE} to
> > > ${STAGING_LIBDIR}.
> > > 3) Instead of copying llvm-config from ${STAGING_BINDIR_NATIVE} to
> > > ${STAGING_BINDIR},
> > > directly make rust-target build dependent
> > > on ${STAGING_BINDIR_NATIVE}/llvm-config.
> > >
> > > Are there any better or cleaner alternatives to handle this situation?
> > > Any suggestions would be greatly appreciated.
> > I would prefer option 3 since these libs are inherently cross built,
> > it perhaps does not matter if
> > they come from native or cross built clang. But that would have been
> > default before this copy dance
> > was done. I might have forgotten now, if this was done specifically to address a
> > case which we can testout now.
> Making Rust depend directly on the native llvm-config is not feasible
> because librustc_llvm*.rlib
> is built for the host architecture(x86-64) . While this works if the
> target is x86-64 or x86, for
> other architectures we get the following error:
> we encounter the following error:
> ld: .../librustc_llvm-f50ec2a9934e24d2.rlib: error adding symbols: file
> format not recognized
> collect2: error: ld returned 1 exit status
>
> Therefore, copying the natively built llvm-config into the target
> sysroot and unsetting the RPATH
> is currently the best approach, and that’s what we are doing. However,
> even after unsetting the
> RPATH, the native llvm-config still references to static libraries for
> other architectures during
> compile time. To fix this, those libraries must be present in the target
> sysroot.
>
> From what I see, the only feasible solution is to build LLVM target
> libraries for all supported
> architectures. Copying these libraries from the native sysroot is not
> feasible because the native
> sysroot contains libs for all OE-core targets, while the target sysroot
> contains libs only for its
> specific target—copying would cause conflicts.
Could it make sense to add in symlinks to the missing pieces between
the two sysroots? We could make that part sysroot specific, i.e. not
affect the target packages, just sysroot population?
Thanks for the timings btw. Given that, I could be tempted to merge
things, then look to improve things again in a follow up patch set.
Curious on others thoughts?
Cheers,
Richard
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-10-08 12:54 ` Richard Purdie
@ 2025-10-10 9:27 ` Deepesh Varatharajan
2025-10-10 9:36 ` Richard Purdie
0 siblings, 1 reply; 14+ messages in thread
From: Deepesh Varatharajan @ 2025-10-10 9:27 UTC (permalink / raw)
To: Richard Purdie, Khem Raj, Ross Burton; +Cc: openembedded-core, Sundeep.Kokkonda
On 08-10-2025 18:24, Richard Purdie wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Wed, 2025-10-08 at 18:19 +0530, Varatharajan, Deepesh via lists.openembedded.org wrote:
>> On 08-10-2025 01:47, Khem Raj wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> On Tue, Oct 7, 2025 at 4:10 AM Deepesh Varatharajan
>>> <deepesh.varatharajan@windriver.com> wrote:
>>>> On 02-10-2025 00:59, Khem Raj wrote:
>>>>> CAUTION: This email comes from a non Wind River email account!
>>>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>>>
>>>>> On Mon, Sep 29, 2025 at 1:29 AM Deepesh Varatharajan
>>>>> <deepesh.varatharajan@windriver.com> wrote:
>>>>>> On 26-09-2025 23:00, Khem Raj wrote:
>>>>>>> CAUTION: This email comes from a non Wind River email account!
>>>>>>> Do not click links or open attachments unless you recognize the sender
>>>>>>> and know the content is safe.
>>>>>>>
>>>>>>> On 9/26/25 3:24 AM, Varatharajan, Deepesh via lists.openembedded.org
>>>>>>> wrote:
>>>>>>>> From: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>>>>>
>>>>>>>> Updated the Rust build to depend on Clang instead.
>>>>>>>>
>>>>>>>> *Summary of discussion with the rust upstream about using latest LLVM
>>>>>>>> instead of Rust maintained LLVM fork.
>>>>>>>> https://internals.rust-lang.org/t/can-we-use-proper-clang-instead-of-llvm-fork-what-rust-uses/23489
>>>>>>>>
>>>>>>>>
>>>>>>>> *Upstream LLVM is generally compatible:
>>>>>>>> - Rust does support building with upstream (vanilla) LLVM, especially
>>>>>>>> the latest
>>>>>>>> major release and the one or two preceding ones.
>>>>>>>> https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html#updating-llvm
>>>>>>>>
>>>>>>>>
>>>>>>>> *Impact on Yocto Rust upgrades:
>>>>>>>> - Rust upgrades shall always check for updates on rust forked llvm
>>>>>>>> and backport
>>>>>>>> the relevant patches to clang's llvm.
>>>>>>>>
>>>>>>>> *Regarding the rust forked llvm local patches:
>>>>>>>> - There are no local patches on rust forked llvm other than the
>>>>>>>> backported fixes
>>>>>>>> from llvm master.
>>>>>>>>
>>>>>>>> *We now add these flags "-Clink-arg=-lz -Clink-arg=-lzstd" because of
>>>>>>>> this following
>>>>>>>> diff otherwise we will get errors during link time.
>>>>>>>>
>>>>>>>> Setup in rust-llvm
>>>>>>>> -DLLVM_ENABLE_ZLIB=OFF \
>>>>>>>> -DLLVM_ENABLE_ZSTD=OFF \
>>>>>>>> -DLLVM_ENABLE_FFI=OFF \
>>>>>>>>
>>>>>>>> Setup in clang
>>>>>>>> -DLLVM_ENABLE_FFI=ON \
>>>>>>>> -DLLVM_ENABLE_ZSTD=ON \
>>>>>>>>
>>>>>>>> *When multilibs enabled:
>>>>>>>>
>>>>>>>> llvm-config expects static libraries to be located in the lib
>>>>>>>> directory rather than
>>>>>>>> lib64. However, since LLVM is built as a non-multilib component, the
>>>>>>>> lib directory
>>>>>>>> doesn't contain any library files. To accommodate this without
>>>>>>>> breaking multilib
>>>>>>>> behavior, we copy the required library files appropriately.
>>>>>>>>
>>>>>>>> Previously, when we depended on rust-llvm, this worked because we
>>>>>>>> specified:
>>>>>>>> -DCMAKE_INSTALL_PREFIX:PATH=${libdir}/llvm-rust
>>>>>>>>
>>>>>>>> With this setup, llvm-config was installed inside
>>>>>>>> ${libdir}/llvm-rust, which included
>>>>>>>> its own bin and lib directories. Thus, llvm-config located in bin
>>>>>>>> would correctly find
>>>>>>>> the libraries in the adjacent lib directory.
>>>>>>>>
>>>>>>>> Even when multilib was enabled or not, llvm-config would still look
>>>>>>>> for libraries under
>>>>>>>> lib in this structure, so everything functioned as expected.
>>>>>>>>
>>>>>>>> *Changes needs to be done when llvm splits from clang:
>>>>>>>> In rust recipe:
>>>>>>>> Update the dependency from:
>>>>>>>> DEPENDS += "ninja-native clang" to DEPENDS += "ninja-native llvm"
>>>>>>>>
>>>>>>>> In llvm recipe:
>>>>>>>> Apply the same changes that were made in the Clang recipe, as those
>>>>>>>> configurations have now been moved to the LLVM recipe after the split.
>>>>>>>>
>>>>>>>> Signed-off-by: Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
>>>>>>>> ---
>>>>>>>> meta/recipes-devtools/clang/clang_git.bb | 4 ++--
>>>>>>>> meta/recipes-devtools/clang/common-clang.inc | 6 +++---
>>>>>>>> meta/recipes-devtools/rust/rust_1.90.0.bb | 18 ++++++++++++++----
>>>>>>>> 3 files changed, 19 insertions(+), 9 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/meta/recipes-devtools/clang/clang_git.bb
>>>>>>>> b/meta/recipes-devtools/clang/clang_git.bb
>>>>>>>> index 53bca1c24f..3e117b308b 100644
>>>>>>>> --- a/meta/recipes-devtools/clang/clang_git.bb
>>>>>>>> +++ b/meta/recipes-devtools/clang/clang_git.bb
>>>>>>>> @@ -83,7 +83,6 @@ OECMAKE_SOURCEPATH = "${S}/llvm"
>>>>>>>> # https://github.com/llvm/llvm-project/blob/main/llvm/CMakeLists.txt
>>>>>>>> LLVM_TARGETS_GPU ?= "${@bb.utils.contains_any('DISTRO_FEATURES',
>>>>>>>> 'opencl opengl vulkan', 'AMDGPU;NVPTX;SPIRV', '', d)}"
>>>>>>>> LLVM_TARGETS_TO_BUILD ?=
>>>>>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>>>>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>>>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>>>>>> compiler we build is building all architectures once because it then
>>>>>>> reuses same compiler for all cross compilers just by creating canonical
>>>>>>> symlinks, this change is not going to work.
>>>>>> We removed this line because, for the clang class-target build, LLVM was
>>>>>> only compiling
>>>>>> libraries for the target architecture and GPU targets and kept inside
>>>>>> the target
>>>>>> sysroot. Since we use the llvm-config built for the class-target during
>>>>>> the Rust target build,
>>>>>> it was failing due to the absence of static libraries for other
>>>>>> architectures in the target sysroot.
>>>>>> Here the target is x86 arch. So, it compiled libraries for x86 arch only
>>>>>> and we can see the error
>>>>>> for missing libraries for other archs as below:
>>>>>>
>>>>>>> --- stderr
>>>>>>> llvm-config: error: missing:
>>>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMAArch64Info.a
>>>>>>> llvm-config: error: missing:
>>>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchInfo.a
>>>>>>> llvm-config: error: missing:
>>>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMLoongArchDisassembler.a
>>>>>>> llvm-config: error: missing:
>>>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMMipsInfo.a
>>>>>>> llvm-config: error: missing:
>>>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMPowerPCInfo.a
>>>>>>> llvm-config: error: missing:
>>>>>> poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/libLLVMRISCVTargetMCA.a
>>>>>> .
>>>>>> .
>>>>>> .
>>>>>>> thread 'main' panicked at compiler/rustc_llvm/build.rs:277:16:
>>>>>>> command did not execute successfully:
>>>>>> "poky/build/tmp/work/x86-64-v3-poky-linux/rust/1.90.0/recipe-sysroot/usr/lib/llvm-config"
>>>>>>
>>>>>> "--link-static" "--libs" "aarch64" "amdgpu" "arm" "asmparser"
>>>>>> "bitreader" "bitwriter" "bpf" "coverage" "instrumentation" "ipo" "linker"
>>>>>> "loongarch" "lto" "mips" "nvptx" "powerpc" "riscv" "x86"
>>>>>>> expected success, got: exit status: 1
>>>>>> Maybe instead of removing. Can we change
>>>>>> -LLVM_TARGETS_TO_BUILD:class-target ?= "${@get_clang_host_arch(bb,
>>>>>> d)};BPF;${LLVM_TARGETS_GPU}"
>>>>>> to
>>>>>> -LLVM_TARGETS_TO_BUILD:class-target ?=
>>>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;;BPF;${LLVM_TARGETS_GPU}"
>>>>>> ?
>>>>> I think we don't want to build all the general arches for target, only
>>>>> the target arch and gpu arches
>>>>> is what is needed. If we build all the backends, that will slow down
>>>>> things. But if target rust needs
>>>>> to support all the architectures to generate code for, then we don't
>>>>> have much choice. Can you see
>>>>> if we are ok to just target one architecture ?
>>>> The issue doesn't originate from Rust directly, it stems from the
>>>> following logic
>>>> in the Rust recipe, where the natively built llvm-config is copied into
>>>> the target sysroot:
>>>>
>>>> # Copy the natively built llvm-config into the target so we can run
>>>> it. Horrible,
>>>> # but works!
>>>> if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
>>>> ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
>>>> mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
>>>> cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
>>>> if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
>>>> patchelf --set-rpath \$ORIGIN/../../../../../`basename
>>>> ${STAGING_DIR_NATIVE}`${libdir_native} ${RUST_ALTERNATE_EXE_PATH}
>>>> else
>>>> patchelf --remove-rpath ${RUST_ALTERNATE_EXE_PATH}
>>>> fi
>>>> fi
>>>>
>>>> Since llvm-config is being copied from ${STAGING_BINDIR_NATIVE} to
>>>> ${STAGING_BINDIR},
>>>> and the native Clang was built with the following configuration:
>>>> LLVM_TARGETS_TO_BUILD ?=
>>>> "AArch64;ARM;BPF;Mips;PowerPC;RISCV;X86;LoongArch;${LLVM_TARGETS_GPU}"
>>>>
>>>> The copied llvm-config will expect the corresponding target libraries to
>>>> be present in the ${STAGING_LIBDIR}.
>>>>
>>>> To address this issue, as far as I can see we have three options.
>>>>
>>>> 1) Build clang-target also for all mentioned archs for clang-native.
>>>> 2) Copy all the required libraries from ${STAGING_LIBDIR_NATIVE} to
>>>> ${STAGING_LIBDIR}.
>>>> 3) Instead of copying llvm-config from ${STAGING_BINDIR_NATIVE} to
>>>> ${STAGING_BINDIR},
>>>> directly make rust-target build dependent
>>>> on ${STAGING_BINDIR_NATIVE}/llvm-config.
>>>>
>>>> Are there any better or cleaner alternatives to handle this situation?
>>>> Any suggestions would be greatly appreciated.
>>> I would prefer option 3 since these libs are inherently cross built,
>>> it perhaps does not matter if
>>> they come from native or cross built clang. But that would have been
>>> default before this copy dance
>>> was done. I might have forgotten now, if this was done specifically to address a
>>> case which we can testout now.
>> Making Rust depend directly on the native llvm-config is not feasible
>> because librustc_llvm*.rlib
>> is built for the host architecture(x86-64) . While this works if the
>> target is x86-64 or x86, for
>> other architectures we get the following error:
>> we encounter the following error:
>> ld: .../librustc_llvm-f50ec2a9934e24d2.rlib: error adding symbols: file
>> format not recognized
>> collect2: error: ld returned 1 exit status
>>
>> Therefore, copying the natively built llvm-config into the target
>> sysroot and unsetting the RPATH
>> is currently the best approach, and that’s what we are doing. However,
>> even after unsetting the
>> RPATH, the native llvm-config still references to static libraries for
>> other architectures during
>> compile time. To fix this, those libraries must be present in the target
>> sysroot.
>>
>> From what I see, the only feasible solution is to build LLVM target
>> libraries for all supported
>> architectures. Copying these libraries from the native sysroot is not
>> feasible because the native
>> sysroot contains libs for all OE-core targets, while the target sysroot
>> contains libs only for its
>> specific target—copying would cause conflicts.
> Could it make sense to add in symlinks to the missing pieces between
> the two sysroots? We could make that part sysroot specific, i.e. not
> affect the target packages, just sysroot population?
>
> Thanks for the timings btw. Given that, I could be tempted to merge
> things, then look to improve things again in a follow up patch set.
>
> Curious on others thoughts?
>
> Cheers,
>
> Richard
>
As you suggested, I was able to resolve the missing library issue by
creating symlinks from the native sysroot to the target sysroot.
However, I'm now encountering a linker error during the build:
../poky/build/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi/../../libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/15.2.0/ld:
../poky/build/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/armv7-poky-linux-gnueabihf/release/deps/librustc_llvm-3e591ab97f1ae683.rlib:
error adding symbols: file format not recognized
| collect2: error: ld returned 1 exit status
Upon investigation, I found that the issue stems from architecture
mismatches within "librustc_llvm-3e591ab97f1ae683.rlib". I extracted
the object files from this archive and ran the file command to inspect
them. Most object files were correctly built for the ARM 32-bit (target):
ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), with
debug_info, not stripped
However, some of the object files (originating from the symlinked native
sysroot libraries) were built for the host architecture (x86_64):
ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
This mixture of architectures in a single .rlib is causing the linker to
fail, and confirms that symlinking missing libraries
from the native sysroot won't work. So to resolve the issue we've left
with only one option to build llvm target also for all
the architectures as we build for native llvm build.
Please let me know if you’re okay with this approach, and I can proceed
with sending the patch.
Regards,
Deepesh
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm
2025-10-10 9:27 ` Deepesh Varatharajan
@ 2025-10-10 9:36 ` Richard Purdie
0 siblings, 0 replies; 14+ messages in thread
From: Richard Purdie @ 2025-10-10 9:36 UTC (permalink / raw)
To: Deepesh Varatharajan, Khem Raj, Ross Burton
Cc: openembedded-core, Sundeep.Kokkonda
On Fri, 2025-10-10 at 14:57 +0530, Deepesh Varatharajan wrote:
>
> As you suggested, I was able to resolve the missing library issue by
> creating symlinks from the native sysroot to the target sysroot.
> However, I'm now encountering a linker error during the build:
>
> ../poky/build/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/recipe-sysroot-native/usr/bin/arm-poky-linux-gnueabi/../../libexec/arm-poky-linux-gnueabi/gcc/arm-poky-linux-gnueabi/15.2.0/ld:
> ../poky/build/tmp/work/cortexa15t2hf-neon-poky-linux-gnueabi/rust/1.90.0/sources/rustc-1.90.0-src/build/x86_64-unknown-linux-gnu/stage1-rustc/armv7-poky-linux-gnueabihf/release/deps/librustc_llvm-3e591ab97f1ae683.rlib:
>
> error adding symbols: file format not recognized
> > collect2: error: ld returned 1 exit status
>
> Upon investigation, I found that the issue stems from architecture
> mismatches within "librustc_llvm-3e591ab97f1ae683.rlib". I extracted
> the object files from this archive and ran the file command to inspect
> them. Most object files were correctly built for the ARM 32-bit (target):
> ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), with
> debug_info, not stripped
>
> However, some of the object files (originating from the symlinked native
> sysroot libraries) were built for the host architecture (x86_64):
> ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
>
> This mixture of architectures in a single .rlib is causing the linker to
> fail, and confirms that symlinking missing libraries
> from the native sysroot won't work. So to resolve the issue we've left
> with only one option to build llvm target also for all
> the architectures as we build for native llvm build.
>
> Please let me know if you’re okay with this approach, and I can proceed
> with sending the patch.
Thanks for trying that, I think it was worth trying and it is shame it
didn't quite work out. I agree with proceeding with the other
direction, thanks.
Cheers,
Richard
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2025-10-10 9:36 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-26 10:24 [PATCH V3 1/2] rust: Use clang instead of rust-llvm Deepesh.Varatharajan
2025-09-26 10:24 ` [PATCH V3 2/2] rust: Drop rust-llvm Deepesh.Varatharajan
2025-09-26 16:50 ` [OE-core] [PATCH V3 1/2] rust: Use clang instead of rust-llvm Khem Raj
2025-09-29 10:17 ` Deepesh Varatharajan
2025-09-26 17:30 ` Khem Raj
2025-09-29 8:29 ` Deepesh Varatharajan
2025-10-01 19:29 ` Khem Raj
2025-10-07 11:10 ` Deepesh Varatharajan
2025-10-07 20:17 ` Khem Raj
2025-10-08 12:49 ` Deepesh Varatharajan
2025-10-08 12:54 ` Richard Purdie
2025-10-10 9:27 ` Deepesh Varatharajan
2025-10-10 9:36 ` Richard Purdie
2025-09-29 8:11 ` Mathieu Dubois-Briand
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox