[Buildroot] [RFC v3 00/30] Add per-package staging feature

[Arnout Vandecappelle <arnout@mind.be>]

On 03/14/15 17:15, Fabio Porcedda wrote:
> On Thu, Mar 5, 2015 at 11:48 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
[snip]
>>  However, I don't like at all the approach of passing the custom -I and -L
>> flags. I think it's much better to start with something that was suggested
>> before in the BR developer meeting: use the toolchain wrapper also for the
>> internal toolchain. Then we can pass the per-package staging dir through the
>> environment, and use it in the --sysroot argument of the compiler.
>>
>>  We'll also have to do something about ccache then - that could be a problem.
>> Perhaps ccache should be called from within the wrapper as well, that may solve
>> a bunch of other ccache problems.
> 
> Using an internal wrapper we can also pass pass the -I and -L
> automatically like the --sysroot option?

 Yes we can. Of course, the path to the package sysroot has to be passed through
the environment.

> 
> If we use a per-package sysroot we need to copy the sysroot directory
> too, because the size of a uclibc sysroot is at least 11MB (if i
> remember correctly the size of a glibc sysroot is something like
> 40MB), so if we copy without using hard links, it will take at least
> 10MB up to 40MB for every package.
> 
> The size of the output directory of a defconfig + allyespackageconfig is 11GB.
> IMHO we have the following choices:
> a) common sysroot & per-package staging & without hard links
>   - the -I -L options are passed trough the toolchain wrapper
>   - output size is 16GB (11GB+5GB)

 Where does the 5GB come from? Experimental evidence from running this series?

>   - no side effects to fear, because hard links are not used
> b) per-package sysroot & without hard links
>   - sysroot are passed trough the toolchain wrapper
>   - output size will be 16GB plus 11MB * 694 = 16GB + 7GB = 23 GB
>   - no side effects to fear, because hard links are not used
> c) per-package sysroot & with hard links for toolchain
>  - sysroot are passed trough the toolchain wrapper
>   - output size will be 16GB
>   - side effects to fear for the toolchain files because hard links
> are used for them
> d) per-package sysroot & with hard links for all files
>  - sysroot are passed trough the toolchain wrapper
>   - output size will be 11GB, so no increased size
>   - side effects to fear because hard links are used

 Actually I'm not really concerned about the size, even if it's more than
doubled by this feature. We use factors less space than yocto so we can afford
to double it :-)

 I'm more concerned about the time penalty of copying things around all the
time. So in that respect, option a) or d) are more attractive. But even creating
all the hardlinks takes a lot of time (probably not that much faster than copying).

 A completely wild alternative is to build and use unionfs-fuse, but that
requires fusermount to be properly installed on the host. Can we assume that?

> 
> The cheaper one with almost no size and speed penalization is d) but
> if some file is overwritten other package sysroot could be affected.
> Maybe we can prevent the overwriting by removing the write permission
> on staging files?

 Yep, that's an option. It can actually be done while rsyncing.


 Regards,
 Arnout

> 
> Which solution is most suited for  buildroot?
> 
> Thanks and my BR
> 


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F