We're trying to keep the threshold for introducing buildtools high
(I'm aware I'm not always arguing for that, but in principle I agree
with RP :).
The bugzilla ticket mentions that the needed linker fix has been
backported to a lot of older binutils releases, so I'd rather ask them
to update their respective binutils packages. It would perhaps be good
to start a clear list of which distros at which versions need the fix,
which ones already have it, and which ones are covered by buildtools.
Can you look into that? Then we can consider further steps.
Alternatively, perhaps you Sunil could help Yoann to disable static
linking on older rust release which is in scarthgap, e.g. backport the
master fix to the scarthgap. I believe that might be the easiest
option. Yoann tried, but couldn't quite get it to work.
Hi Alex,
Thanks, that makes sense.
Just a quick recap of the master vs. scarthgap situation for anyone following this thread:
Master: Rust already links LLVM dynamically, so neither the worker nor the binutils version matters.
Scarthgap: Rust still links LLVM statically, so the host ld matters. Since the eligible scarthgap
workers use binutils older than 2.45, they do not normally produce the new relocation.
The issue only surfaced after Ubuntu 26.04 was added to the scarthgap pool (241a957),
and was later reverted (3d3e5ff).
Now for the worker check: I don't have access to the workers, so I verified the linker each one
uses in two ways: for host-toolchain workers, I ran that distro's binutils in a Docker image of
the same version; for buildtools workers, I downloaded the exact tarball pinned by the AB and tested
the ld inside it. In each case, I linked the small reproducer and checked whether the
push was converted into a call.
Host-toolchain workers (distro's own ld):
Ubuntu 22.04 - ld 2.38 - BROKEN
Ubuntu 24.04 - ld 2.42 - BROKEN
Ubuntu 25.04 - ld 2.44 - BROKEN
Ubuntu 25.10 - ld 2.45 - ok
Ubuntu 26.04 - ld 2.46 - ok
Debian 12 - ld 2.40 - BROKEN
Debian 13 - ld 2.44 - BROKEN
Fedora 42 - ld 2.44 - BROKEN
Fedora 43 - ld 2.45.1 - ok
Fedora 44 - ld 2.46 - ok
AlmaLinux 10 - ld 2.41 - BROKEN
openSUSE 16.0 - ld 2.45 - ok
Buildtools workers (alma8/9, rocky8/9, stream9, opensuse156,
perf-alma8, debian11) use the pinned tarball's ld:
master - buildtools 5.1 - ld 2.43.1.20241111 - BROKEN
scarthgap - buildtools 5.0.4 - ld 2.42.0.20240716 - BROKEN
- No distro has picked up the PR32991 backport yet. The "ok" ones simply ship binutils >= 2.45.
- Debian 13, Ubuntu 25.04, and Fedora 42 ship binutils 2.44. Their gas emits plain R_X86_64_GOTPCREL
relocations rather than the GOTPCRELX form introduced in 2.45, so they are fine as producers but
mis-link GOTPCRELX objects produced by >= 2.45 workers.
- Both pinned buildtools tarballs are broken.
So right now every x86 worker except Ubuntu 25.10/26.04, Fedora 43/44 and openSUSE 16.0 links rust
with a broken ld, and those >= 2.45 workers are the producers, so the rest are exposed through shared
sstate.
On next steps:
1. The scarthgap binutils recipe already carries the PR32991 fix (its SRCREV is the backport commit),
so a buildtools tarball rebuilt from current scarthgap would be fine - would refreshing the pinned
tarballs on the helper branches help here?
2. For the host-toolchain workers, if you're okay with it, I can start filing bugs with the relevant
distros (Debian, Ubuntu, and Fedora; the EL rebuilds follow RHEL). The reproducer makes each report
self-contained, but it will still take time for the fixes to land.
3. On the scarthgap backport, I got the dynamic LLVM linking change working with scarthgap's Rust.
Since rust-llvm on scarthgap never built a shared libLLVM, I first had to enable that,
along with some path and packaging adjustments because LLVM 17 installs the shared library
as libLLVM-17.so under the llvm-rust prefix.
The draft is here:
https://git.yoctoproject.org/poky-contrib/commit/?h=deepesh/rust-segfault-scarthgap&id=e8bdd23a09d58e4311988b158ca0b9a63820b723
I built core-image-minimal with Rust and Cargo, booted it, and verified that rustc and cargo work
on the target by compiling and running a test program. librustc_driver now carries a NEEDED entry
for libLLVM-17.so instead of linking LLVM statically. oe-selftest is still running on my side.
If you and Yoann are okay with this approach, I'll submit it to scarthgap once it completes.
~Sunil Dora