Re: [PATCH 2/3] rust: Upgrade 1.84.1->1.85.0

[Randy MacLeod <randy.macleod@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 9181 bytes --]

On 2025-03-28 10:30 a.m., Yash.Shinde@windriver.com wrote:
> From: Yash Shinde<Yash.Shinde@windriver.com>
>
> Rust stable version updated to 1.85.0
> https://blog.rust-lang.org/2025/02/20/Rust-1.85.0.html
>
> Some of the major updates:
>
> - Update LIC_FILES_CHKSUM in libstd-rs and rust recipes.
>
> - Pass '-Zforce-unstable-if-unmarked' to RUSTFLAGS in libstd-rs.bb
> Fix:https://github.com/rust-lang/rust/issues/133857#issuecomment-2526341227
>
> - Downgrade bootstrap cc version causing bootstrap to fail on custom targets. (Backported from v1.85.1)
> Fix:https://github.com/rust-lang/rust/pull/137460/commits/e4ca11f87ffca8c63aa56d45b46e62b6acc58bd7
>
> - Explicitly set float ABI for all ARM 32 bits targets.
> Fix:https://github.com/rust-lang/rust/commit/a51fefcaab835b310e2e26005b50982d0049d905
>
> - Rust v1.85.0 tarball doesn't ship gcc tree. Drop the unpack:append() which removed it and prevented the bloat.
> Fix:https://github.com/rust-lang/rust/commit/13c3f9b9498013837782b46120085ea19ca75518
>
> Adapted the patch changes with v1.85.0:
>    repro-issue-fix-with-cc-crate-hashmap.patch
>    revert-link-std-statically-in-rustc_driver-feature.patch
>    rust-oe-selftest.patch
>    rv32-cargo-rustix-0.38.40-fix.patch
>
> Dropped patches:
>    fix-tidy-check-failure.patch since it's merged with v1.85.0.
>
> Signed-off-by: Yash Shinde<Yash.Shinde@windriver.com>
> ---
>   .../classes-recipe/rust-target-config.bbclass |   4 +
>   meta/conf/distro/include/tcmode-default.inc   |   2 +-
>   .../rust/{cargo_1.84.1.bb => cargo_1.85.0.bb} |   0
>   .../rust/files/downgrade-bootstrap-cc.patch   |  41 +++
>   .../rust/files/fix-tidy-check-failure.patch   |  19 --
>   ...epro-issue-fix-with-cc-crate-hashmap.patch |  60 ++++-
>   ...d-statically-in-rustc_driver-feature.patch |   4 +-
>   .../rust/files/rust-oe-selftest.patch         |  46 ++--
>   .../files/rv32-cargo-rustix-0.38.38-fix.patch | 236 ------------------
>   .../files/rv32-cargo-rustix-0.38.40-fix.patch | 236 ++++++++++++++++++
>   ...ibstd-rs_1.84.1.bb => libstd-rs_1.85.0.bb} |   4 +-
>   ....84.1.bb => rust-cross-canadian_1.85.0.bb} |   0
>   ...ust-llvm_1.84.1.bb => rust-llvm_1.85.0.bb} |   0
>   meta/recipes-devtools/rust/rust-snapshot.inc  |  81 +++---
>   meta/recipes-devtools/rust/rust-source.inc    |  23 +-
>   .../rust/{rust_1.84.1.bb => rust_1.85.0.bb}   |   0
>   16 files changed, 415 insertions(+), 341 deletions(-)
>   rename meta/recipes-devtools/rust/{cargo_1.84.1.bb => cargo_1.85.0.bb} (100%)
>   create mode 100644 meta/recipes-devtools/rust/files/downgrade-bootstrap-cc.patch
>   delete mode 100644 meta/recipes-devtools/rust/files/fix-tidy-check-failure.patch
>   delete mode 100644 meta/recipes-devtools/rust/files/rv32-cargo-rustix-0.38.38-fix.patch
>   create mode 100644 meta/recipes-devtools/rust/files/rv32-cargo-rustix-0.38.40-fix.patch
>   rename meta/recipes-devtools/rust/{libstd-rs_1.84.1.bb => libstd-rs_1.85.0.bb} (88%)
>   rename meta/recipes-devtools/rust/{rust-cross-canadian_1.84.1.bb => rust-cross-canadian_1.85.0.bb} (100%)
>   rename meta/recipes-devtools/rust/{rust-llvm_1.84.1.bb => rust-llvm_1.85.0.bb} (100%)
>   rename meta/recipes-devtools/rust/{rust_1.84.1.bb => rust_1.85.0.bb} (100%)
>
Others have pointed out a few problems, including a build failure.

...

>   RUST_DIST_SERVER ="https://static.rust-lang.org"
> +
>   RUST_STD_SNAPSHOT = "rust-std-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
>   RUSTC_SNAPSHOT = "rustc-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
>   CARGO_SNAPSHOT = "cargo-${SNAPSHOT_VERSION}-${RUST_BUILD_ARCH}-unknown-linux-gnu"
> diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc
> index 92bb5e4779..e07055882c 100644
> --- a/meta/recipes-devtools/rust/rust-source.inc
> +++ b/meta/recipes-devtools/rust/rust-source.inc
> @@ -1,30 +1,19 @@
>   RUST_VERSION ?="${@d.getVar('PV').split('-')[0]}"
>   
>   SRC_URI +="https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;name=rust 
> \ - file://rv32-cargo-rustix-0.38.38-fix.patch;patchdir=${RUSTSRC} \ + 
> file://rv32-cargo-rustix-0.38.40-fix.patch;patchdir=${RUSTSRC} \ 
> file://rust-oe-selftest.patch;patchdir=${RUSTSRC} \ - 
> file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} 
> \ + 
> file://repro-issue-fix-with-cc-crate-hashmap.patch;patchdir=${RUSTSRC} 
> \ 
> file://oeqa-selftest-Increase-timeout-in-process-sigpipe-ru.patch;patchdir=${RUSTSRC} 
> \ 
> file://0001-src-core-build_steps-tool.rs-switch-off-lto-for-rust.patch;patchdir=${RUSTSRC} 
> \ - 
> file://revert-link-std-statically-in-rustc_driver-feature.patch;patchdir=${RUSTSRC} 
> \ - 
> file://Zdual-proc-macros-additional-check.patch;patchdir=${RUSTSRC} \ 
> - file://fix-tidy-check-failure.patch;patchdir=${RUSTSRC} \ + 
> file://revert-link-std-statically-in-rustc_driver-feature.patch;patchdir=${RUSTSRC} 
> \ + 
> file://Zdual-proc-macros-additional-check.patch;patchdir=${RUSTSRC} \ 
> + file://downgrade-bootstrap-cc.patch;patchdir=${RUSTSRC} \ "
> -SRC_URI[rust.sha256sum] = "e23ec747a06ffd3e94155046f40b6664ac152c9ee3c2adfd90353a7ccff24226"
> +SRC_URI[rust.sha256sum] = "d542c397217b5ba5bac7eb274f5ca62d031f61842c3ba4cc5328c709c38ea1e7"
>   
>   RUSTSRC = "${WORKDIR}/rustc-${RUST_VERSION}-src"
>   
> -#After updating to Rust 1.85 we can revert this removal of "src/gcc"
> -remove_gcc_directory() {
> -    # Ensure the gcc directory exists
> -    gcc_dir="${S}/src/gcc"
> -    if [ -d "$gcc_dir" ]; then
> -        rm -rf "$gcc_dir"
> -    fi
> -}
> -# Add postfunc to the do_unpack task
> -do_unpack[postfuncs] += "remove_gcc_directory"
> -


On IRC, Richard said:
     I'm puzzled as the rust space fix didn't have the impact on the 
metrics I'd hoped for :/
     I think there is something else going on. ...

Let's review.

Rust-1.83, is where the src/gcc bloat was added.
That was merged here:

    commit 72e21bef1a6bcc8f05ad4e24cfef4228384f1c3a
    Author:     Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
    AuthorDate: Wed Mar 5 01:01:56 2025
    Commit:     Richard Purdie <richard.purdie@linuxfoundation.org>
    CommitDate: Wed Mar 5 16:26:54 2025

        rust: Upgrade 1.82.0->1.83.0

The postfunc removal commit was here:

    commit 2cb5d1accca23df67dd55d01416ea404ab92683b
    Author:     Deepesh Varatharajan <Deepesh.Varatharajan@windriver.com>
    AuthorDate: Thu Mar 27 05:06:03 2025
    Commit:     Richard Purdie <richard.purdie@linuxfoundation.org>
    CommitDate: Thu Mar 27 09:41:58 2025

        rust: Fix bloat issue in recent rust updates

so we should see a large tmpfs usage increase on March 5th and a 
significant reduction on March 27th.

Look at the buildperf-alma8  (or buildperf-debian11) on March 27th:

https://valkyrie.yocto.io/pub/non-release/?type=buildperf-alma8&date=2025-03-27

There are 4 perf builds that day and build number: 20250327-93 includes 
the bloat removal commit

which I found by opening each run, starting with the lowest run numbers 
and clicking on the
"commit" in the "Current commit" column at the top of each page.

You can see that the FS usage did decrease but only by 785 MB. Maybe 
that's the 1 GB for src/gcc
and some other packages are adding 215 MB.



If you look at the FS usage for
    test1: Build core-image-sato

You can see the significant increase on March 5th of 103504 - 97857 = 
5647 MB
that Richard pointed out week(s) ago:

There's another significant increase on March 7, and a decrease on March 
11/12 which would be good to understand.


So, it's Friday afternoon and I'm not going to dig any more today.


Yash, Deepesh,

You did mentioned that you dropped  the rm -rf src/gcc in the commit log but
in addition please do some work to check what the FS usage is for a 
core-image-minimal build
at each stage probably initially bug checking out 1.82, 1.83, 1.84, 
local commit 1.85. in poky's git repo.

Are you able to reproduce the initial ~5647 MB increase in 1.83 ?
What tmpdir size do you see once the:
    rust: Fix bloat issue in recent rust updates
commit is added?


If things are confusing using the public poky repo and master branch 
commits you could
start with poky with rust-1.82 and create a branch where you just add the
rust updates (and dependent commits as needed) to eliminate changes from
other parts of the system.


Please send an summary of findings email to this list each day as you 
are testing and reviewing the data.

Thanks,

../Randy



>   UPSTREAM_CHECK_URI ="https://forge.rust-lang.org/infra/other-installation-methods.html"
>   UPSTREAM_CHECK_REGEX = "rustc-(?P<pver>\d+(\.\d+)+)-src"
>   
> diff --git a/meta/recipes-devtools/rust/rust_1.84.1.bb b/meta/recipes-devtools/rust/rust_1.85.0.bb
> similarity index 100%
> rename from meta/recipes-devtools/rust/rust_1.84.1.bb
> rename to meta/recipes-devtools/rust/rust_1.85.0.bb


-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2.1: Type: text/html, Size: 13944 bytes --]

[-- Attachment #2.2: Nua1q4ux5Rq8uEFS.png --]
[-- Type: image/png, Size: 34144 bytes --]

[-- Attachment #2.3: wIto4RDoMX0WSvkF.png --]
[-- Type: image/png, Size: 28667 bytes --]