Re: [OE-core] [v4] [RFC] Merge meta-rust to oe-core - Aug 19 update

["Richard Purdie" <richard.purdie@linuxfoundation.org>]

On Wed, 2021-09-01 at 09:44 +0100, Richard Purdie via lists.openembedded.org
wrote:
> Hi Esteban,
> 
> On Fri, 2021-08-27 at 15:18 +0000, Kuber, Esteban wrote:
> > On 2021/8/27, 2:03 AM, "Richard Purdie" <richard.purdie@linuxfoundation.org> wrote:
> >     > and:
> >     >
> >     > 2. a reproducible build failure on CentOS-7:
> >     >
> >     >     https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/597
> >     >
> >     > where, we see:
> >     >     = note: /bin/sh: /lib64/libc.so.6: version `GLIBC_2.33' \
> >     >     not found (required by \
> >     >     /home/pokybuild/yocto-worker/reproducible-centos/build/\
> >     >     build-st/reproducibleB/tmp/work/x86_64-linux/cargo-native/\
> >     >     1.54.0-r0/recipe-sysroot-native/usr/lib/libtinfo.so.5)
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >     error: linking with `\
> >     >     /home/pokybuild/yocto-worker/reproducible-centos/build/\
> >     >     build-st/reproducibleB/tmp/work/x86_64-linux/cargo-native/\
> >     >     1.54.0-r0/wrapper/target-rust-ccld` failed: exit status: 1
> >    
> > 
> >     I had a quick look at this. It reproduces if you build cargo-native on a centos7
> >     machine with our M2 buildtools tarball in the environment of the build.
> > 
> >     Adding the uninative relocation hack to the cargo snapshot binary with:
> > 
> >      do_cargo_setup_snapshot () {
> >             ${WORKDIR}/rust-snapshot-components/${CARGO_SNAPSHOT}/install.sh --prefix="${WORKDIR}/${CARGO_SNAPSHOT}" --disable-ldconfig
> >     +       # Need to use uninative's loader if enabled/present since the library paths
> >     +       # are used internally by rust and result in symbol mismatches if we don't
> >     +       if [ ! -z "${UNINATIVE_LOADER}" -a -e "${UNINATIVE_LOADER}" ]; then
> >     +               patchelf-uninative ${WORKDIR}/${CARGO_SNAPSHOT}/bin/cargo --set-interpreter ${UNINATIVE_LOADER}
> >     +       fi
> >      }
> > 
> >     didn't help.
> > 
> >     Running the command it mentions failing by hand in the same toolchain enabled
> >     shell works. It therefore seems likely that something rust is putting into the
> >     environment is breaking things. What that is, I don't know, I'm out of time to
> >     debug further.
> > 
> > I'm looking at what that could be. If I can't give you the actual list (from the
> > code), I can give you a patch to print out to stderr *everything* that rustc is
> > setting during builds. We currently have a `-v` verbose  flag in cargo that
> > gives out the full command with wich rustc in invoked, but as you say, the
> > problem is likely the environment variables at play.
> 
> With some further debugging as I mentioned in my other reply, it is the
> LD_LIBRARY_PATH setting which is confusing things. Probably as a result of:
> 
> https://doc.rust-lang.org/cargo/reference/environment-variables.html#dynamic-library-paths
> 
> >     It looks to me like it is using the ld from the host instead of the buildtools
> >     tarball. I did change tmp/work/x86_64-linux/cargo-native/1.54.0-
> >     r0/wrapper/target-rust-ccld to a full path to gcc and messed with PATH to ensure
> >     it would find "our" ld first but that didn't help.
> > 
> > This is making me wonder, could it be that we are defaulting to a linker that is
> > not supplied in the buildtools tar.gz? That would explain why it ends up picking
> > up the system's even when changing the $PATH.
> 
> It is a bit more subtle than that. The ccld script has /bin/sh as the
> interpreter which uses the host libc and host libtinfo. The LD_LIBRARY_PATH
> injected by cargo causes it to find the libtinfo in the recipe-sysroot-native
> which is incompatible with it and things then fail :(.
> 
> > 
> >     In the error output is some:
> > 
> >     Usage: which [options] [--] COMMAND [...]
> >     Write the full path of COMMAND(s) to standard output.
> > 
> >     suggesting some which call might not be compatible with centos7?
> > 
> >     Cheers,
> > 
> >     Richard
> > 
> > One clarification I would like to have is, is this the *first* time you are 
> > trying to build rustc in this configuration, or is this a *recent* problem 
> > introduced in 1.54? Just want to make sure that my understanding that we are 
> > dealing with the former is correct.
> 
> This is the first time we've tried this as far as I know.
> 
> It is a horrible mix of different host tools and libcs from uninative,
> buildtools and the host all conflicting when LD_LIBRARY_PATH is set :(.
> 
> I'm not sure how to go about resolving this.

I noticed some code in cargo. I think the code is where the sysroot path is
injected. Since rust figures out its own paths to everything it is a little
surprising this is needed and the comment suggests the author had similar
thoughts. I therefore tried the following patch:

Index: cargo/src/cargo/core/compiler/compilation.rs
===================================================================
--- cargo.orig/src/cargo/core/compiler/compilation.rs
+++ cargo/src/cargo/core/compiler/compilation.rs
@@ -273,13 +273,6 @@ impl<'cfg> Compilation<'cfg> {
             ));
             search_path.push(self.deps_output[&kind].clone());
             search_path.push(self.root_output[&kind].clone());
-            // For build-std, we don't want to accidentally pull in any shared
-            // libs from the sysroot that ships with rustc. This may not be
-            // required (at least I cannot craft a situation where it
-            // matters), but is here to be safe.
-            if self.config.cli_unstable().build_std.is_none() {
-                search_path.push(self.sysroot_target_libdir[&kind].clone());
-            }
         }
 
         let dylib_path = paths::dylib_path();

however since I think we're running a prebuilt binary rather than the one we're
building, this does work. I'm therefore not sure how we can fix this. Is this
code something upstream might consider removing?

Cheers,

Richard