Re: [Buildroot] [PATCH v2] package/pkg-utils: Prevent per-package syncing stale deps

[John Ernberg via buildroot <buildroot@buildroot.org>]

Hi Arnout,

On 2/4/26 5:34 PM, Arnout Vandecappelle wrote:
>   Hi John, Jonas,
> 
> On 27/12/2024 17:06, John Ernberg wrote:
>> From: Jonas Blixt <jonas.blixt@actia.se>
>>
>> From: Jonas Blixt <jonas.blixt@actia.se>
>>
>> This filters out the dependency inputs when syncing the per-package
>> target directory to only include 'skeleton' and its dependencies, if
>> they exist. Since other dependencies are not needed in the target dir
>> for successful building they can be ignored and depend on the ppd-merge
>> to populate the files from the other packages, unlike the staging dir
>> which is untouched here.
> 
>   I may be missing something, but doesn't this break the combination of 
> busybox
> with e.g. attr? The busybox install script checks if the file already 
> exists and
> does not create the symlink when it does. busybox depends on attr (and 
> dozens of
> other packages) for exactly that reason.
> 
>   There are other packages as well that rely on the presence of 
> something in the
> target directory.

We didn't notice any breakages of busybox, but we also only use about
10 utils out of busybox (none of them are part of the conflict
packages on our config).

It did not even cross my mind packages might do that. Thank you for
bringing it up!

(I haven't inventoried which other packages yet, will do if it becomes
necessary after below points considered)

I currently see 2 ways to approach this problem, please let me know
what you think about them.

1) We move such steps (such as generating the busybox applet links)
    to the TARGET_FINALIZE_HOOKS stage. This will need us to install
    more files into target/ that will need to be ignored later like
    the TARGET_DIR_WARNING_FILE is ignored.

    Packages may need to be able to define additional files themselves
    that need to be skipped in rootfs generation.

    Looking at busybox it might not be exactly trivial but it should
    be doable.

2) Introduce <pkg>-rebuild-rdepends and <pkg>-reconfigure-rdepends,
    which trigger the appropriate rebuild of all the rdepends.

    This would be the same thing we do today to work around the
    problem (make <pkg>-rebuild && make <rdep1>-reconfigure ...) but
    automatically from inside buildroot.

> 
>> This solves the problem where stale files from dependencies will
>> linger in the per-package target directories and may up in the final
>> target directory after the ppd-merge, overwriting the fresh ones,
>> causing unecessarly many rebuild steps to ensure that the latest
>> version of a dependency is included in the final target directory.
> 
>   Yep, we're aware of this problem, and haven't found a solution :-)

It is indeed a tricky problem, it is bothering me enough though that
I'm willing to work on it as much as I can.

Best regards // John Ernberg

> 
>   Regards,
>   Arnout
> 
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot