[PATCH V3 1/2] rust-cross-canadian: Fix environment setup script conflicts for multilibs

[PATCH V3 1/2] rust-cross-canadian: Fix environment setup script conflicts for multilibs

[Deepthi.Hemraj]

From: Deepthi Hemraj <Deepthi.Hemraj@windriver.com>

YOCTO [#15061]
The target independent variables like CARGO_HTTP_MULTIPLEXING and
CARGO_HTTP_CAINFO should be handled by the nativesdk-cargo recipe,
rather than being managed within rust-cross-canadian.

The build section of the Cargo configuration file is also
target-independent and hence moved to nativesdk-cargo.

The nativesdk-cargo recipe can then gather these entries from the individual
target-specific files to generate the final cargo_home/config file.
Target-specific configurations are specific to the rust-cross-canadian
variant and should remain there.

Signed-off-by: Deepthi Hemraj <Deepthi.Hemraj@windriver.com>
---
 meta/recipes-devtools/rust/cargo_1.80.1.bb    | 25 +++++++++++++++++++
 .../rust/rust-cross-canadian.inc              | 16 ++----------
 2 files changed, 27 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-devtools/rust/cargo_1.80.1.bb b/meta/recipes-devtools/rust/cargo_1.80.1.bb
index 123032cdf7..623cda0471 100644
--- a/meta/recipes-devtools/rust/cargo_1.80.1.bb
+++ b/meta/recipes-devtools/rust/cargo_1.80.1.bb
@@ -50,8 +50,33 @@ do_install:append:class-nativesdk() {
 	# sets to libdir but not base_libdir leading to symbol mismatches depending on the
 	# host OS. Fully set LD_LIBRARY_PATH to contain both to avoid this.
 	create_wrapper ${D}/${bindir}/cargo LD_LIBRARY_PATH=${libdir}:${base_libdir}
+
+    ENV_SETUP_DIR=${D}${base_prefix}/environment-setup.d
+    mkdir "${ENV_SETUP_DIR}"
+    CARGO_ENV_SETUP_SH="${ENV_SETUP_DIR}/cargo.sh"
+
+    RUST_HOST_TRIPLE=`echo ${RUST_HOST_SYS} | tr '[:lower:]' '[:upper:]' | sed 's/-/_/g'`
+
+    cat <<- EOF > "${CARGO_ENV_SETUP_SH}"
+	export CARGO_TARGET_${RUST_HOST_TRIPLE}_RUNNER="\$OECORE_NATIVE_SYSROOT/lib/${SDKLOADER}"
+	export CARGO_HOME="\$OECORE_TARGET_SYSROOT/home/cargo"
+	mkdir -p "\$CARGO_HOME"
+	if [ ! -f "\$CARGO_HOME/config" ]; then
+		touch "\$CARGO_HOME/config"
+		echo "[build]" >> "\$CARGO_HOME/config"
+		echo 'linker = "target-rust-ccld"' >> "\$CARGO_HOME/config"
+    fi
+
+	# Keep the below off as long as HTTP/2 is disabled.
+	export CARGO_HTTP_MULTIPLEXING=false
+
+	export CARGO_HTTP_CAINFO="\$OECORE_NATIVE_SYSROOT/etc/ssl/certs/ca-certificates.crt"
+	EOF
+
 }
 
+FILES:${PN} += "${base_prefix}/environment-setup.d"
+
 # Disabled due to incompatibility with libgit2 0.28.x (https://github.com/rust-lang/git2-rs/issues/458, https://bugs.gentoo.org/707746#c1)
 # as shipped by Yocto Dunfell.
 # According to https://github.com/rust-lang/git2-rs/issues/458#issuecomment-522567539, there are no compatibility guarantees between
diff --git a/meta/recipes-devtools/rust/rust-cross-canadian.inc b/meta/recipes-devtools/rust/rust-cross-canadian.inc
index c34b839d15..b2a212df40 100644
--- a/meta/recipes-devtools/rust/rust-cross-canadian.inc
+++ b/meta/recipes-devtools/rust/rust-cross-canadian.inc
@@ -57,36 +57,24 @@ do_install () {
     RUST_ENV_SETUP_SH="${ENV_SETUP_DIR}/rust.sh"
 
     RUST_TARGET_TRIPLE=`echo ${RUST_TARGET_SYS} | tr '[:lower:]' '[:upper:]' | sed 's/-/_/g'`
-    RUST_HOST_TRIPLE=`echo ${RUST_HOST_SYS} | tr '[:lower:]' '[:upper:]' | sed 's/-/_/g'`
     SDKLOADER=${@bb.utils.contains('SDK_ARCH', 'x86_64', 'ld-linux-x86-64.so.2', '', d)}${@bb.utils.contains('SDK_ARCH', 'i686', 'ld-linux.so.2', '', d)}${@bb.utils.contains('SDK_ARCH', 'aarch64', 'ld-linux-aarch64.so.1', '', d)}${@bb.utils.contains('SDK_ARCH', 'ppc64le', 'ld64.so.2', '', d)}${@bb.utils.contains('SDK_ARCH', 'riscv64', 'ld-linux-riscv64-lp64d.so.1', '', d)}
 
     cat <<- EOF > "${RUST_ENV_SETUP_SH}"
 	export CARGO_TARGET_${RUST_TARGET_TRIPLE}_RUSTFLAGS="--sysroot=\$OECORE_TARGET_SYSROOT/usr -C link-arg=--sysroot=\$OECORE_TARGET_SYSROOT"
-	export CARGO_TARGET_${RUST_HOST_TRIPLE}_RUNNER="\$OECORE_NATIVE_SYSROOT/lib/${SDKLOADER}"
 	export RUST_TARGET_PATH="\$OECORE_NATIVE_SYSROOT/usr/lib/${TARGET_SYS}/rustlib"
-	EOF
-
-    chown -R root.root ${D}
-
-    CARGO_ENV_SETUP_SH="${ENV_SETUP_DIR}/cargo.sh"
-    cat <<- EOF > "${CARGO_ENV_SETUP_SH}"
 	export CARGO_HOME="\$OECORE_TARGET_SYSROOT/home/cargo"
 	mkdir -p "\$CARGO_HOME"
         # Init the default target once, it might be otherwise user modified.
 	if [ ! -f "\$CARGO_HOME/config" ]; then
 		touch "\$CARGO_HOME/config"
-		echo "[build]" >> "\$CARGO_HOME/config"
 		echo 'target = "'${RUST_TARGET_SYS}'"' >> "\$CARGO_HOME/config"
 		echo '# TARGET_SYS' >> "\$CARGO_HOME/config"
 		echo '[target.'${RUST_TARGET_SYS}']' >> "\$CARGO_HOME/config"
-		echo 'linker = "target-rust-ccld"' >> "\$CARGO_HOME/config"
     fi
 
-	# Keep the below off as long as HTTP/2 is disabled.
-	export CARGO_HTTP_MULTIPLEXING=false
-
-	export CARGO_HTTP_CAINFO="\$OECORE_NATIVE_SYSROOT/etc/ssl/certs/ca-certificates.crt"
 	EOF
+
+    chown -R root.root ${D}
 }
 
 FILES:${PN} += "${base_prefix}/environment-setup.d"
-- 
2.43.0

[PATCH V3 2/2] rust-cross-canadian: Fix environment setup script conflicts for multilibs

[Deepthi.Hemraj]

From: Deepthi Hemraj <Deepthi.Hemraj@windriver.com>

YOCTO [#15061]
Rust multilib sdks broken because of the conflicts between attempted
installs of rust-cross-canadian for multilibs

Multilibs trying to install cargo.sh and rust.sh in the same path
which resulted in the issue

The target dependent variables are retained on the rust-cross-canadian and
others are moved to the nativesdk-cargo recipe.
By appending ${RUST_TARGET_SYS} to environment script filenames,
conflicts between architectures are resolved.

Signed-off-by: Deepthi Hemraj <Deepthi.Hemraj@windriver.com>
---
 meta/recipes-devtools/rust/cargo_1.80.1.bb         |  2 +-
 meta/recipes-devtools/rust/rust-cross-canadian.inc |  6 ++----
 meta/recipes-devtools/rust/rust_1.80.1.bb          | 13 +++++++++++++
 3 files changed, 16 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/rust/cargo_1.80.1.bb b/meta/recipes-devtools/rust/cargo_1.80.1.bb
index 623cda0471..e67e7e9254 100644
--- a/meta/recipes-devtools/rust/cargo_1.80.1.bb
+++ b/meta/recipes-devtools/rust/cargo_1.80.1.bb
@@ -61,7 +61,7 @@ do_install:append:class-nativesdk() {
 	export CARGO_TARGET_${RUST_HOST_TRIPLE}_RUNNER="\$OECORE_NATIVE_SYSROOT/lib/${SDKLOADER}"
 	export CARGO_HOME="\$OECORE_TARGET_SYSROOT/home/cargo"
 	mkdir -p "\$CARGO_HOME"
-	if [ ! -f "\$CARGO_HOME/config" ]; then
+	if ! grep -q '^\[build\]' "\$CARGO_HOME/config"; then
 		touch "\$CARGO_HOME/config"
 		echo "[build]" >> "\$CARGO_HOME/config"
 		echo 'linker = "target-rust-ccld"' >> "\$CARGO_HOME/config"
diff --git a/meta/recipes-devtools/rust/rust-cross-canadian.inc b/meta/recipes-devtools/rust/rust-cross-canadian.inc
index b2a212df40..4941914b74 100644
--- a/meta/recipes-devtools/rust/rust-cross-canadian.inc
+++ b/meta/recipes-devtools/rust/rust-cross-canadian.inc
@@ -54,18 +54,16 @@ do_install () {
 
     ENV_SETUP_DIR=${D}${base_prefix}/environment-setup.d
     mkdir "${ENV_SETUP_DIR}"
-    RUST_ENV_SETUP_SH="${ENV_SETUP_DIR}/rust.sh"
+    RUST_ENV_SETUP_SH="${ENV_SETUP_DIR}/${RUST_TARGET_SYS}_rust.sh"
 
     RUST_TARGET_TRIPLE=`echo ${RUST_TARGET_SYS} | tr '[:lower:]' '[:upper:]' | sed 's/-/_/g'`
     SDKLOADER=${@bb.utils.contains('SDK_ARCH', 'x86_64', 'ld-linux-x86-64.so.2', '', d)}${@bb.utils.contains('SDK_ARCH', 'i686', 'ld-linux.so.2', '', d)}${@bb.utils.contains('SDK_ARCH', 'aarch64', 'ld-linux-aarch64.so.1', '', d)}${@bb.utils.contains('SDK_ARCH', 'ppc64le', 'ld64.so.2', '', d)}${@bb.utils.contains('SDK_ARCH', 'riscv64', 'ld-linux-riscv64-lp64d.so.1', '', d)}
 
     cat <<- EOF > "${RUST_ENV_SETUP_SH}"
 	export CARGO_TARGET_${RUST_TARGET_TRIPLE}_RUSTFLAGS="--sysroot=\$OECORE_TARGET_SYSROOT/usr -C link-arg=--sysroot=\$OECORE_TARGET_SYSROOT"
-	export RUST_TARGET_PATH="\$OECORE_NATIVE_SYSROOT/usr/lib/${TARGET_SYS}/rustlib"
 	export CARGO_HOME="\$OECORE_TARGET_SYSROOT/home/cargo"
 	mkdir -p "\$CARGO_HOME"
-        # Init the default target once, it might be otherwise user modified.
-	if [ ! -f "\$CARGO_HOME/config" ]; then
+	if ! grep -q '^\[target.${RUST_TARGET_SYS}]' "\$CARGO_HOME/config"; then
 		touch "\$CARGO_HOME/config"
 		echo 'target = "'${RUST_TARGET_SYS}'"' >> "\$CARGO_HOME/config"
 		echo '# TARGET_SYS' >> "\$CARGO_HOME/config"
diff --git a/meta/recipes-devtools/rust/rust_1.80.1.bb b/meta/recipes-devtools/rust/rust_1.80.1.bb
index eae1f28bb8..544f2ac8a0 100644
--- a/meta/recipes-devtools/rust/rust_1.80.1.bb
+++ b/meta/recipes-devtools/rust/rust_1.80.1.bb
@@ -271,8 +271,21 @@ rust_do_install:class-nativesdk() {
     rm ${D}${libdir}/rustlib/uninstall.sh
     rm ${D}${libdir}/rustlib/install.log
     rm ${D}${libdir}/rustlib/manifest*
+
+    ENV_SETUP_DIR=${D}${base_prefix}/environment-setup.d
+    mkdir "${ENV_SETUP_DIR}"
+    RUST_ENV_SETUP_SH="${ENV_SETUP_DIR}/rust.sh"
+
+    cat <<- EOF > "${RUST_ENV_SETUP_SH}"
+	export RUST_TARGET_PATH="\$RUST_TARGET_PATH:\$OECORE_NATIVE_SYSROOT/usr/lib/${TARGET_SYS}/rustlib"
+	EOF
+
+    chown -R root.root ${D}
+
 }
 
+FILES:${PN} += "${base_prefix}/environment-setup.d"
+
 EXTRA_TOOLS ?= "cargo-clippy clippy-driver rustfmt"
 rust_do_install:class-target() {
     export PSEUDO_UNLOAD=1
-- 
2.43.0

Re: [OE-core] [PATCH V3 1/2] rust-cross-canadian: Fix environment setup script conflicts for multilibs

[Mathieu Dubois-Briand]

On Fri, Nov 29, 2024 at 12:08:19AM -0800, Hemraj, Deepthi via lists.openembedded.org wrote:
> From: Deepthi Hemraj <Deepthi.Hemraj@windriver.com>
> 
> YOCTO [#15061]
> The target independent variables like CARGO_HTTP_MULTIPLEXING and
> CARGO_HTTP_CAINFO should be handled by the nativesdk-cargo recipe,
> rather than being managed within rust-cross-canadian.
> 
> The build section of the Cargo configuration file is also
> target-independent and hence moved to nativesdk-cargo.
> 
> The nativesdk-cargo recipe can then gather these entries from the individual
> target-specific files to generate the final cargo_home/config file.
> Target-specific configurations are specific to the rust-cross-canadian
> variant and should remain there.
> 
> Signed-off-by: Deepthi Hemraj <Deepthi.Hemraj@windriver.com>
> ---

Hi,

I believe these patches are breaking some builds on the autobuilder:

...
Caused by:
  could not parse TOML configuration in `/srv/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0/testimage-sdk/sysroots/cortexa8hf-neon-poky-linux-gnueabi/home/cargo/config`
Caused by:
  TOML parse error at line 3, column 1
    |
  3 | [target.armv7-poky-linux-gnueabihf]
    | ^
  invalid table header
  dotted key `target` attempted to extend non-table type (string)

https://valkyrie.yoctoproject.org/#/builders/2/builds/558/steps/12/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/60/builds/558/steps/12/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/80/builds/488/steps/13/logs/stdio

oeqa.utils.subprocesstweak.OETestCalledProcessError: Command '. /srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0/testimage-sdk/environment-setup-core2-64-poky-linux > /dev/null; cd /srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0/testimage-sdk//hello; cargo build;' returned non-zero exit status 101
Standard Output: warning: `/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0/testimage-sdk/sysroots/core2-64-poky-linux/home/cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
warning: unused config key `build.linker` in `/srv/pokybuild/yocto-worker/genericx86-64/build/build/tmp/work/genericx86_64-poky-linux/core-image-sato/1.0/testimage-sdk/sysroots/core2-64-poky-linux/home/cargo/config`
error: process didn't exit successfully: `rustc -vV` (exit status: 1)
--- stdout
rustc 1.80.1 (3f5fd8dd4 2024-08-06) (built from a source tarball)
binary: rustc
commit-hash: 3f5fd8dd41153bc5fdca9427e9e05be2c767ba23
commit-date: 2024-08-06
host: x86_64-pokysdk-linux-gnu
release: 1.80.1
--- stderr
error: Error loading target specification: Could not find specification for target "x86_64-pokysdk-linux-gnu". Run `rustc --print target-list` for a list of built-in targets

https://valkyrie.yoctoproject.org/#/builders/4/builds/563/steps/12/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/19/builds/559/steps/12/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/31/builds/549/steps/13/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/32/builds/541/steps/13/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/39/builds/546/steps/13/logs/stdio
https://valkyrie.yoctoproject.org/#/builders/41/builds/494/steps/13/logs/stdio

Can you have a look at these issues, please ?

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com