Re: [OE-core] [PATCH V4 2/3] rust: Upgrade 1.82.0->1.83.0 --> bloat alert !!

[Yash Shinde <Yash.Shinde@windriver.com>]

[-- Attachment #1: Type: text/plain, Size: 4744 bytes --]


On 14-03-2025 19:18, Randy MacLeod wrote:
> On 2025-03-14 8:52 a.m., Yash Shinde wrote:
>>
>>
>> On 14-03-2025 00:18, Randy MacLeod wrote:
>>> On 2025-03-07 6:37 p.m., Richard Purdie wrote:
>>>> On Fri, 2025-03-07 at 16:02 -0500, Randy MacLeod wrote:
>>>>> ❯ for i in `ls -d *src`; do du -sh $i; done
>>>>>
>>>>> 3.5G    rustc-1.82.0-src
>>>>> 4.8G    rustc-1.83.0-src
>>>>> 4.9G    rustc-1.84.0-src
>>>>> 3.9G    rustc-1.85.0-src
>>>>>
>>>>> I didn't check for the other 4 GB yet...
>>>>>
>>>>>
>>>> rust, rust-native, nativesdk-rust, cargo and so on.
>>>>
>>>>> Should we do that an host the tarball  on yoctoproject.org? 
>>>>> <https://urldefense.com/v3/__http://yoctoproject.org?__;!!AjveYdw8EvQ!fJXJLq5zUX_ORAdT0imsy5PfIAHVBjw3rwJRpJH0HgmwRJiTyMFIQebHf7jG9o_l7GKu06t9zK2RNaNX3CujsTYP40pyXJxYOXO78vw$>
>>>>>
>>>>> Should we write a custom unpack rule to 
>>>>> --exclude=rustc-1.83.0-src/src/gcc ?
>>>>>
>>>> Or just delete the directory in a do_unpack postfunc?
>>>
>>> Yash has 1.84 mostly working and
>>>
>>> will add a post unpack change to rm -rf rustc-1.83.0-src/src/gcc
>>>
>>>>
>>>> Or wait for 1.85?
>>>
>>> This likely won't get merged until YP-5.3. We'll see how long it 
>>> takes to be ready for review.
>>>
>>>
>>>>
>>>> This is annoying "bloat" which we at least need to identify and 
>>>> work out a plan for.
>>>>>
>>>>>
>>>>>
>>>>> Next week, I'll also dig into the 'tmp' usage increase to 
>>>>> understand where the 5 GB that Richard reported comes from.
>>>>>
>>>> See above, I'm guessing so it needs to be confirmed but...
>>>
>>>
>>> Yashe,
>>>
>>> Can you review the reason for the FSspace usage difference between 
>>> 1.82, 1.83 when you have time?
>>>
>>
>> The following differences were seen when compared the FS usage 
>> between v1.82 and v1.83:
>>
>> I did a basic "bitbake rust" build (in separate dir paths) to check 
>> the effect on Yocto build FS:
>>
>> The TMPDIR FS usage:
>>
>> TMDIR FS usage                               v1.82.0                 
>>                    v1.83.0
>> du -sh poky/build/tmp/                               65 G             
>>                         73 G
>> du -sh 
>> poky/build/tmp/work/core2-64-poky-linux/rust/1.8x.0/rustc-1.8x.0-src/ 
>>                           9.8 G                                12 G
>>
>>
>> It's more 8G in v1.83.
>>
>
> Thanks Yash!
>
> It's a bit odd that you see an 8G diff and YP AB saw 5 GB but that's 
> close enough for now.
>
> Please run the same analysis for 1.84 update both before and after 
> removing the src/gcc .
>
> Please summarize the data for 1.82, 1.83 and the two sets of data for 
> 1.84.
>
> I'd expect 1.84 + "rm -rf src/gcc" to be somewhat larger than 1.82 but 
> much smaller than 1.83.
>
TMDIR FS usage            v1.82.0                          v1.83.0     
        v1.84.0                          v1.84 + rm -rf src/gcc
du -sh poky/build/tmp/           65 G                               73 G 
                 73 G                                            72 G

du -sh  poky/build/../1.8x.0/rustc-1.8x.0-src/               9.8 
G                             12 G 12 
G                                           11 G

The size of TMPDIR increased from 65 G in 1.82 to 73 G in 1.83 and 
remained same in 1.84.
After removing "src/gcc"(1.1 G) the size is reduced to 72G.

Thanks,
Yash.

>
> ../Randy
>
>
>>
>> The main reason for FS usuage differences in v1.83 are:
>> 1. Addition of gcc  ~1.1 G.2. The addition of enzyme AD sub module ~392 Mb.
>>
>> References:
>> https://github.com/rust-lang/rust/commit/9028177115718b38c0a189c4fb9eda03a14841d7
>> https://github.com/rust-lang/rust/issues/124509
>> https://github.com/rust-lang/rust/pull/129175 
>> https://github.com/rust-lang/rust/commit/59d4114b2d1aaac9a6dfe770997f2e79ccfd28ab
>>
>>
>> Dir
>> 	1.82.0
>> 	1.83.0
>> 	Diff
>> rustc-1.8x.0-src.tar.xz
>> 	210Mb
>> 	338Mb
>> 	128Mb
>> rustc-1.8x.0-src
>> 	3.5G
>> 	4.8G
>> 	1.3G
>>
>> (PS: Please ignore the previous mail.)
>>
>> Regards,
>> Yash
>>>
>>>
>>> ../Randy
>>>
>>>
>>>
>>>
>>>>
>>>> Cheers,
>>>>
>>>> Richard
>>>>
>>>>
>>>
>>> -- 
>>> # Randy MacLeod
>>> # Wind River Linux
>
>
> -- 
> # Randy MacLeod
> # Wind River Linux

-- 
Regards,
Yash Shinde

[-- Attachment #2: Type: text/html, Size: 14285 bytes --]