[PATCH] rust: Use llvm-config from native llvm-rust on rust target builds

[PATCH] rust: Use llvm-config from native llvm-rust on rust target builds

[Khem Raj]

target rust recipe builds ( cross compile ) calls llvm-config from
target sysroot which works ok as long as C++ runtime it needs is
available on build host e.g. libstdc++ etc. which is commonly the
case, however when using clang and llvm runtime this falters since
it should be using libc++ from native sysroot and if this does not
exist on build machine this fails to find libc++ shared object and
llvm-config fails to run. This ensures that llvm-config version in
use is correctly relocated and can use shared libraries from native
sysroot correctly. Adding ORIGIN to sysroot will look for the .so in
same dir as the binary and there is the libc++.so.1 copied in place

Fixes rust build with clang compiler.

| /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or director
y
| thread 'main' panicked at llvm.rs:551:19:
| command did not execute successfully: "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config" "--version"
| expected success, got: exit status: 127

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/rust/rust_1.74.1.bb b/meta/recipes-devtools/rust/rust_1.74.1.bb
index 30543ada7db..2dffe009827 100644
--- a/meta/recipes-devtools/rust/rust_1.74.1.bb
+++ b/meta/recipes-devtools/rust/rust_1.74.1.bb
@@ -198,9 +198,13 @@ rust_runx () {
     # Copy the natively built llvm-config into the target so we can run it. Horrible,
     # but works!
     if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} != ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
-        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
+        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
+        mkdir -p ${tgtdir}
         cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
-        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
+        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
+            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
+        fi
+        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
     fi
 
     oe_cargo_fix_env
-- 
2.43.0

Re: [OE-core] [PATCH] rust: Use llvm-config from native llvm-rust on rust target builds

[Richard Purdie]

On Sat, 2024-01-13 at 21:58 -0800, Khem Raj wrote:
> target rust recipe builds ( cross compile ) calls llvm-config from
> target sysroot which works ok as long as C++ runtime it needs is
> available on build host e.g. libstdc++ etc. which is commonly the
> case, however when using clang and llvm runtime this falters since
> it should be using libc++ from native sysroot and if this does not
> exist on build machine this fails to find libc++ shared object and
> llvm-config fails to run. This ensures that llvm-config version in
> use is correctly relocated and can use shared libraries from native
> sysroot correctly. Adding ORIGIN to sysroot will look for the .so in
> same dir as the binary and there is the libc++.so.1 copied in place
> 
> Fixes rust build with clang compiler.
> 
> > /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or director
> y
> > thread 'main' panicked at llvm.rs:551:19:
> > command did not execute successfully: "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config" "--version"
> > expected success, got: exit status: 127
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-devtools/rust/rust_1.74.1.bb b/meta/recipes-devtools/rust/rust_1.74.1.bb
> index 30543ada7db..2dffe009827 100644
> --- a/meta/recipes-devtools/rust/rust_1.74.1.bb
> +++ b/meta/recipes-devtools/rust/rust_1.74.1.bb
> @@ -198,9 +198,13 @@ rust_runx () {
>      # Copy the natively built llvm-config into the target so we can run it. Horrible,
>      # but works!
>      if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} != ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
> -        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
> +        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
> +        mkdir -p ${tgtdir}
>          cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
> -        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
> +        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
> +            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
> +        fi
> +        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
>      fi
>  
>      oe_cargo_fix_env

Copying a native library into the target sysroot goes beyond what I'm
comfortable with even for this horrible hack with llvm-config.

Since it seems to be finding it by RPATH, can you not just add an RPATH
to the native sysroot in the binary?

Cheers,

Richard

Re: [OE-core] [PATCH] rust: Use llvm-config from native llvm-rust on rust target builds

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 3247 bytes --]

Hi

On Mon, Jan 15, 2024 at 5:19 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Sat, 2024-01-13 at 21:58 -0800, Khem Raj wrote:
> > target rust recipe builds ( cross compile ) calls llvm-config from
> > target sysroot which works ok as long as C++ runtime it needs is
> > available on build host e.g. libstdc++ etc. which is commonly the
> > case, however when using clang and llvm runtime this falters since
> > it should be using libc++ from native sysroot and if this does not
> > exist on build machine this fails to find libc++ shared object and
> > llvm-config fails to run. This ensures that llvm-config version in
> > use is correctly relocated and can use shared libraries from native
> > sysroot correctly. Adding ORIGIN to sysroot will look for the .so in
> > same dir as the binary and there is the libc++.so.1 copied in place
> >
> > Fixes rust build with clang compiler.
> >
> > >
> /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config:
> error while loading shared libraries: libc++.so.1: cannot open shared
> object file: No such file or director
> > y
> > > thread 'main' panicked at llvm.rs:551:19:
> > > command did not execute successfully:
> "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config"
> "--version"
> > > expected success, got: exit status: 127
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
> >  1 file changed, 6 insertions(+), 2 deletions(-)
> >
> > diff --git a/meta/recipes-devtools/rust/rust_1.74.1.bb
> b/meta/recipes-devtools/rust/rust_1.74.1.bb
> > index 30543ada7db..2dffe009827 100644
> > --- a/meta/recipes-devtools/rust/rust_1.74.1.bb
> > +++ b/meta/recipes-devtools/rust/rust_1.74.1.bb
> > @@ -198,9 +198,13 @@ rust_runx () {
> >      # Copy the natively built llvm-config into the target so we can run
> it. Horrible,
> >      # but works!
> >      if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
> ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ]; then
> > -        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
> > +        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
> > +        mkdir -p ${tgtdir}
> >          cp ${RUST_ALTERNATE_EXE_PATH_NATIVE} ${RUST_ALTERNATE_EXE_PATH}
> > -        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
> > +        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
> > +            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
> > +        fi
> > +        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
> >      fi
> >
> >      oe_cargo_fix_env
>
> Copying a native library into the target sysroot goes beyond what I'm
> comfortable with even for this horrible hack with llvm-config.


It’s just supporting the original hack to work properly I don’t think it’s
any worse that the original hack

>
>
> Since it seems to be finding it by RPATH, can you not just add an RPATH
> to the native sysroot in the binary?


It does not work and I don’t know why but seems rpath in llvm-config
affects what paths it spits out

>


>
> Cheers,
>
> Richard
>

[-- Attachment #2: Type: text/html, Size: 4752 bytes --]

Re: [OE-core] [PATCH] rust: Use llvm-config from native llvm-rust on rust target builds

[Richard Purdie]

On Mon, 2024-01-15 at 08:56 -0800, Khem Raj wrote:
> Hi
> 
> On Mon, Jan 15, 2024 at 5:19 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Sat, 2024-01-13 at 21:58 -0800, Khem Raj wrote:
> > > target rust recipe builds ( cross compile ) calls llvm-config
> > > from
> > > target sysroot which works ok as long as C++ runtime it needs is
> > > available on build host e.g. libstdc++ etc. which is commonly the
> > > case, however when using clang and llvm runtime this falters
> > > since
> > > it should be using libc++ from native sysroot and if this does
> > > not
> > > exist on build machine this fails to find libc++ shared object
> > > and
> > > llvm-config fails to run. This ensures that llvm-config version
> > > in
> > > use is correctly relocated and can use shared libraries from
> > > native
> > > sysroot correctly. Adding ORIGIN to sysroot will look for the .so
> > > in
> > > same dir as the binary and there is the libc++.so.1 copied in
> > > place
> > > 
> > > Fixes rust build with clang compiler.
> > > 
> > > > /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-
> > > > linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-
> > > > config: error while loading shared libraries: libc++.so.1:
> > > > cannot open shared object file: No such file or director
> > > y
> > > > thread 'main' panicked at llvm.rs:551:19:
> > > > command did not execute successfully:
> > > > "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-
> > > > linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-
> > > > config" "--version"
> > > > expected success, got: exit status: 127
> > > 
> > > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > > ---
> > >   meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
> > >   1 file changed, 6 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > b/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > index 30543ada7db..2dffe009827 100644
> > > --- a/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > +++ b/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > @@ -198,9 +198,13 @@ rust_runx () {
> > >       # Copy the natively built llvm-config into the target so we
> > > can run it. Horrible,
> > >       # but works!
> > >       if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
> > > ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ];
> > > then
> > > -        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
> > > +        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
> > > +        mkdir -p ${tgtdir}
> > >           cp ${RUST_ALTERNATE_EXE_PATH_NATIVE}
> > > ${RUST_ALTERNATE_EXE_PATH}
> > > -        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
> > > +        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
> > > +            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
> > > +        fi
> > > +        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
> > >       fi
> > >   
> > >       oe_cargo_fix_env
> > 
> > Copying a native library into the target sysroot goes beyond what
> > I'm
> > comfortable with even for this horrible hack with llvm-config.
> > 
> 
> 
> It’s just supporting the original hack to work properly I don’t think
> it’s any worse that the original hack 

Putting a non-target library into the target sysroot could cause all
kinds of problems if the target and native are similar enough the
libraries can be seen and confused by the linker. I do see that as a
big and potentially very dangerous difference.

> > 
> > 
> > Since it seems to be finding it by RPATH, can you not just add an
> > RPATH
> > to the native sysroot in the binary?
> > 
> 
> 
> It does not work and I don’t know why but seems rpath in llvm-config
> affects what paths it spits out 

I am reluctant to take this without a better understanding of what is
going on here. We may need to fix the lvm-config issues properly.

Cheers,

Richard

Re: [OE-core] [PATCH] rust: Use llvm-config from native llvm-rust on rust target builds

[Khem Raj]

On Mon, Jan 15, 2024 at 9:49 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2024-01-15 at 08:56 -0800, Khem Raj wrote:
> > Hi
> >
> > On Mon, Jan 15, 2024 at 5:19 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > On Sat, 2024-01-13 at 21:58 -0800, Khem Raj wrote:
> > > > target rust recipe builds ( cross compile ) calls llvm-config
> > > > from
> > > > target sysroot which works ok as long as C++ runtime it needs is
> > > > available on build host e.g. libstdc++ etc. which is commonly the
> > > > case, however when using clang and llvm runtime this falters
> > > > since
> > > > it should be using libc++ from native sysroot and if this does
> > > > not
> > > > exist on build machine this fails to find libc++ shared object
> > > > and
> > > > llvm-config fails to run. This ensures that llvm-config version
> > > > in
> > > > use is correctly relocated and can use shared libraries from
> > > > native
> > > > sysroot correctly. Adding ORIGIN to sysroot will look for the .so
> > > > in
> > > > same dir as the binary and there is the libc++.so.1 copied in
> > > > place
> > > >
> > > > Fixes rust build with clang compiler.
> > > >
> > > > > /mnt/b/yoe/master/build/tmp/work/riscv64-yoe-
> > > > > linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-
> > > > > config: error while loading shared libraries: libc++.so.1:
> > > > > cannot open shared object file: No such file or director
> > > > y
> > > > > thread 'main' panicked at llvm.rs:551:19:
> > > > > command did not execute successfully:
> > > > > "/mnt/b/yoe/master/build/tmp/work/riscv64-yoe-
> > > > > linux/rust/1.74.1/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-
> > > > > config" "--version"
> > > > > expected success, got: exit status: 127
> > > >
> > > > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > > > ---
> > > >   meta/recipes-devtools/rust/rust_1.74.1.bb | 8 ++++++--
> > > >   1 file changed, 6 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > > b/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > > index 30543ada7db..2dffe009827 100644
> > > > --- a/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > > +++ b/meta/recipes-devtools/rust/rust_1.74.1.bb
> > > > @@ -198,9 +198,13 @@ rust_runx () {
> > > >       # Copy the natively built llvm-config into the target so we
> > > > can run it. Horrible,
> > > >       # but works!
> > > >       if [ ${RUST_ALTERNATE_EXE_PATH_NATIVE} !=
> > > > ${RUST_ALTERNATE_EXE_PATH} -a ! -f ${RUST_ALTERNATE_EXE_PATH} ];
> > > > then
> > > > -        mkdir -p `dirname ${RUST_ALTERNATE_EXE_PATH}`
> > > > +        tgtdir=`dirname ${RUST_ALTERNATE_EXE_PATH}`
> > > > +        mkdir -p ${tgtdir}
> > > >           cp ${RUST_ALTERNATE_EXE_PATH_NATIVE}
> > > > ${RUST_ALTERNATE_EXE_PATH}
> > > > -        chrpath -d ${RUST_ALTERNATE_EXE_PATH}
> > > > +        if [ -e ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ]; then
> > > > +            cp ${STAGING_LIBDIR_NATIVE}/libc++.so.1 ${tgtdir}/
> > > > +        fi
> > > > +        chrpath -r \$ORIGIN ${RUST_ALTERNATE_EXE_PATH}
> > > >       fi
> > > >
> > > >       oe_cargo_fix_env
> > >
> > > Copying a native library into the target sysroot goes beyond what
> > > I'm
> > > comfortable with even for this horrible hack with llvm-config.
> > >
> >
> >
> > It’s just supporting the original hack to work properly I don’t think
> > it’s any worse that the original hack
>
> Putting a non-target library into the target sysroot could cause all
> kinds of problems if the target and native are similar enough the
> libraries can be seen and confused by the linker. I do see that as a
> big and potentially very dangerous difference.

Agreed. although the conditions it must meet to fail make it quite unlikely
since it goes into specific directory llvm-rust/bin where llvm-rust
binaries are installed
under sysroot, and none of those target binaries use RPATH except
llvm-config but there is a chance.


>
> > >
> > >
> > > Since it seems to be finding it by RPATH, can you not just add an
> > > RPATH
> > > to the native sysroot in the binary?
> > >
> >
> >
> > It does not work and I don’t know why but seems rpath in llvm-config
> > affects what paths it spits out
>
> I am reluctant to take this without a better understanding of what is
> going on here. We may need to fix the lvm-config issues properly.
>

I think if we use ORIGIN relative paths into RPATH, it works ok, previously
I tried to hardcode native sysroot paths. I will send a revised
version to replace
it with RPATH

> Cheers,
>
> Richard
>