Re: [OE-core] [RFC] rust: Upgrade 1.85.1->1.86.0

[Randy MacLeod <randy.macleod@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 14397 bytes --]

On 2025-06-02 1:09 p.m., Khem Raj wrote:
>
>
> On Mon, Jun 2, 2025 at 6:03 AM Randy MacLeod <rwmacleod@gmail.com> wrote:
>
>     Hi Yash,
>
>
>     On Mon, Jun 2, 2025, 07:15 Yash Shinde via lists.openembedded.org
>     <https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!eFbErouxWe_8sEEVI1krYJ8mhuv-rIrrvLEk7rYp2ZeCBYDpMmsVsNya8myxiMywaOFMMtB__0YvY7rtMhcpAg$>
>     <Yash.Shinde=windriver.com@lists.openembedded.org> wrote:
>
>
>         On 30-05-2025 18:45, Alexander Kanavin wrote:
>>         On Fri, 30 May 2025 at 14:42, Yash Shinde<Yash.Shinde@windriver.com> wrote:
>>
>>>         As Khem mentioned, setting lld = false in config.toml doesn't build LLD.
>>>         Furthermore, using "use-lld = false" ensures that LLD is not used as the
>>>         default linker,
>>>         causing the build to fall back to the system's default linker. This
>>>         restores the earlier behavior.
>>>         Also, the package QA error for rust-lld is not seen with this.
>>         What about build times though? Are they back to what they were?
>>
>         The build time differences on my machine are as follows:
>
>         rust version
>         	rust upstream builds
>         	rust-native (yocto build)
>         	rust target (yocto build)
>         v1.85.1
>         	10 mins
>         	20 mins
>         	33 mins
>         v1.86.0
>         	22 mins (with "lld" and "use-lld" set to "false")
>         30 mins (with default settings i.e lld = true)
>
>         	31 mins (with "lld" and
>         "use-lld" set to "false") 	43 mins (with "lld" and
>         "use-lld" set to "false")
>
>         Note: The machine being used is a shared resource and fairly
>         busy. Build times may differ due to other user's builds.
>
>
>         The FS usage is as follows:
>
>         Rust version
>         	rust upstream builds
>         	Yocto rust builds
>         (build/tmp/work/core2-64-poky-linux/rust/1.86.0/rustc-1.86.0-src/build)
>         v1.85.1
>         	4.6 G
>         	6.3 G
>         v1.86.0
>         	15 G
>         	8.8 G
>
>         The latest rust v1.86.0 upgrade changes are available at:
>         https://git.yoctoproject.org/poky-contrib/log/?h=harish/rust_1.86.0_v2
>         <https://urldefense.com/v3/__https://git.yoctoproject.org/poky-contrib/log/?h=harish*rust_1.86.0_v2__;Lw!!AjveYdw8EvQ!eFbErouxWe_8sEEVI1krYJ8mhuv-rIrrvLEk7rYp2ZeCBYDpMmsVsNya8myxiMywaOFMMtB__0YvY7qkai0dkw$>
>

FS bloat is partly due to:
    1. we forgot to remove cargo from rust pkg,
    2. rust-analyzer added in 1.86, - do we need it?
    3. wasm-component-ld  built - do we need it?
    4. llvm-bitcode-linker      built - do we need it?

For now, drop the things we don't need by changing config.toml.
We can consider adding them as disabled by default 
PACKAGECONFIG[feature] options later.

This is from looking at the packages-split contents and a filtered 
log.do_install.
Details and more info below.

../Randy



I expect that rust-analyzer is also the primary reason for the build 
time increase but hopefully Yash can confirm.

I dug through the log files a bit which I don't do often these days so 
maybe there's a better way to get
this info.

❯ rg  "^COMPILE|^Building" 1.85.1/log.do_install | sed 's/ (.*)$//'  > 
1.85.1/COMPILE-Building
❯ rg  "^COMPILE|^Building" 1.86/log.do_install | sed 's/ (.*)$//' > 
1.86/COMPILE-Building


Diffs below (1).


Initially I was looking at log.do_install and filtering out noise using 
a combination of grep, sed, and
emacs macros of all things!

I ended up reducing the log.do_install file length by an order of magnitude:
❯ wc -l */log.do_install
    13351 1.85.1/log.do_install
    22144 1.86/log.do_install

❯ wc -l */log.do_install.Compiling.fix-with-emacs.Compiling.sed-path
   1211 1.85.1/log.do_install.Compiling.fix-with-emacs.Compiling.sed-path
   2103 1.86/log.do_install.Compiling.fix-with-emacs.Compiling.sed-path

Diff here:
https://www.diffchecker.com/hduQYfmq/

in case anyone is interested in the lower level build view.



1) diff -u and sdiff of high level COMPILE|Building

✦ ❯ diff -u 1.8*/COMPILE-Building
--- 1.85.1/COMPILE-Building    2025-06-03 14:11:34.838212146 -0400
+++ 1.86/COMPILE-Building    2025-06-03 14:11:44.997206613 -0400
@@ -2,18 +2,36 @@
  Building bootstrap
  Building stage0 library artifacts
  Building compiler artifacts
+Building stage0 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building stage1 library artifacts
  Building compiler artifacts
+Building stage1 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building stage1 library artifacts
  Building stage0 tool rust-installer
  Building compiler artifacts
+Building stage1 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building tool rustdoc
+Building tool rust-analyzer-proc-macro-srv
+Building stage0 tool generate-copyright
+Building tool cargo
+Building tool rust-analyzer
+Building tool rustfmt
+Building tool cargo-fmt
+Building tool clippy-driver
+Building tool cargo-clippy
  COMPILE rust install clippy
  Building bootstrap
  Building stage0 library artifacts
  Building compiler artifacts
+Building stage0 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building stage1 library artifacts
  Building compiler artifacts
+Building stage1 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building stage1 library artifacts
  Building compiler artifacts
  Building tool clippy-driver
@@ -23,8 +41,12 @@
  Building bootstrap
  Building stage0 library artifacts
  Building compiler artifacts
+Building stage0 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building stage1 library artifacts
  Building compiler artifacts
+Building stage1 tool wasm-component-ld
+Building tool llvm-bitcode-linker
  Building stage1 library artifacts
  Building compiler artifacts
  Building tool rustfmt

✦ ❯ sdiff 1.8*/COMPILE-Building
COMPILE rust install                        COMPILE rust install
Building bootstrap                        Building bootstrap
Building stage0 library artifacts                Building stage0 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage0 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building stage1 library artifacts                Building stage1 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage1 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building stage1 library artifacts                Building stage1 library 
artifacts
Building stage0 tool rust-installer                Building stage0 tool 
rust-installer
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage1 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building tool rustdoc                        Building tool rustdoc
                                   >    Building tool 
rust-analyzer-proc-macro-srv
                                   >    Building stage0 tool 
generate-copyright
                                   >    Building tool cargo
                                   >    Building tool rust-analyzer
                                   >    Building tool rustfmt
                                   >    Building tool cargo-fmt
                                   >    Building tool clippy-driver
                                   >    Building tool cargo-clippy
COMPILE rust install clippy                    COMPILE rust install clippy
Building bootstrap                        Building bootstrap
Building stage0 library artifacts                Building stage0 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage0 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building stage1 library artifacts                Building stage1 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage1 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building stage1 library artifacts                Building stage1 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
Building tool clippy-driver                    Building tool clippy-driver
Building tool cargo-clippy                    Building tool cargo-clippy
Building stage0 tool rust-installer                Building stage0 tool 
rust-installer
COMPILE rust install rustfmt                    COMPILE rust install rustfmt
Building bootstrap                        Building bootstrap
Building stage0 library artifacts                Building stage0 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage0 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building stage1 library artifacts                Building stage1 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
                                   >    Building stage1 tool 
wasm-component-ld
                                   >    Building tool llvm-bitcode-linker
Building stage1 library artifacts                Building stage1 library 
artifacts
Building compiler artifacts                    Building compiler artifacts
Building tool rustfmt                        Building tool rustfmt
Building tool cargo-fmt                        Building tool cargo-fmt
Building stage0 tool rust-installer                Building stage0 tool 
rust-installer

>
>         Regards,
>         Yash
>
>
>     Hi Yash
>
>
>     That's good progress. We'll need to think about what, it anything,
>     we're missing by not building rust-lld. We should run some tests
>     to compare the link times but that can likely wait until after we
>     get the update merged.
>
>
> Though the internet has some information on lld improvements having 
> this in OE context will be valuable.
> I also think disabling lld is going to give us some reprieve. As more 
> targets in rust seem to use it by default we need to consider that we 
> do not start disabling defaults and wear off from upstream.
>
>     Remaining concerns:
>
>     1. Significant increase in FS usage, compile time. Have you looked
>     into why there's still such a significant increase?
>
>     2.  * Enable INHIBIT_PACKAGE_DEBUG_SPLIT/STRIP to fix QA stripping
>     errors. Fixes: ERROR: rust-1.86.0-r0 do_package: QA Issue: File
>     '/usr/lib/rustlib/x86_64-poky-linux-gnu/bin/wasm-component-ld'
>     from rust was already stripped, this will prevent future
>     debugging! [already-stripped]
>
>
>     Do we use this component? Is it new? Why is it produced without
>     debug symbols?
>
>
>     I'll take a more careful look at the commit code changes once we
>     have answers to these questions.
>
>
>     Thanks for your hard work.
>     Almost done this upgrade hopefully.
>
>     Randy
>
>>         Alex
>
>         -=-=-=-=-=-=-=-=-=-=-=-
>         Links: You receive all messages sent to this group.
>         View/Reply Online (#217714):
>         https://lists.openembedded.org/g/openembedded-core/message/217714
>         <https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/message/217714__;!!AjveYdw8EvQ!eFbErouxWe_8sEEVI1krYJ8mhuv-rIrrvLEk7rYp2ZeCBYDpMmsVsNya8myxiMywaOFMMtB__0YvY7oWKRQz_A$>
>         Mute This Topic:
>         https://lists.openembedded.org/mt/113268009/953399
>         <https://urldefense.com/v3/__https://lists.openembedded.org/mt/113268009/953399__;!!AjveYdw8EvQ!eFbErouxWe_8sEEVI1krYJ8mhuv-rIrrvLEk7rYp2ZeCBYDpMmsVsNya8myxiMywaOFMMtB__0YvY7pWRz_sBA$>
>         Group Owner: openembedded-core+owner@lists.openembedded.org
>         <mailto:openembedded-core%2Bowner@lists.openembedded.org>
>         Unsubscribe:
>         https://lists.openembedded.org/g/openembedded-core/unsub
>         <https://urldefense.com/v3/__https://lists.openembedded.org/g/openembedded-core/unsub__;!!AjveYdw8EvQ!eFbErouxWe_8sEEVI1krYJ8mhuv-rIrrvLEk7rYp2ZeCBYDpMmsVsNya8myxiMywaOFMMtB__0YvY7ryeheIkg$>
>         [rwmacleod@gmail.com]
>         -=-=-=-=-=-=-=-=-=-=-=-
>

-- 
# Randy MacLeod
# Wind River Linux

[-- Attachment #2: Type: text/html, Size: 30288 bytes --]