[Buildroot] build speedup possible?

[Buildroot] build speedup possible?

[Andreas Naumann]

Hi guys,

First of all kudos again for this very good tool! I'm using it almsot 
daily and am stoked about the functionality but simple usage at the same 
time.
Now one issue with build systems is always build time. It seems the 
speed is not impacted much by just spending more horsepower (cpu cores). 
When looking at the load I see very frequently that just one core is 
busy with one package, especially during the configure step. Some 
packages themselves are build with just one core, I guess we cant do 
very much about that.
However, why are independent packages not build in parallel? Are the 
dependencies too complex to make this work? Thanks for any insight!


cheers,
Andreas

[Buildroot] build speedup possible?

[Maxime Hadjinlian]

Hi Andreas, all
On Fri, Feb 7, 2014 at 9:51 AM, Andreas Naumann <dev@andin.de> wrote:
> Hi guys,
>
> First of all kudos again for this very good tool! I'm using it almsot daily
> and am stoked about the functionality but simple usage at the same time.
> Now one issue with build systems is always build time. It seems the speed is
> not impacted much by just spending more horsepower (cpu cores). When looking
> at the load I see very frequently that just one core is busy with one
> package, especially during the configure step. Some packages themselves are
> build with just one core, I guess we cant do very much about that.
> However, why are independent packages not build in parallel? Are the
> dependencies too complex to make this work? Thanks for any insight!
>
You should look at the following thread:
http://permalink.gmane.org/gmane.comp.lib.uclibc.buildroot/74047
Cheers
>
> cheers,
> Andreas
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] Antw: build speedup possible?

[Frank Ihle]

Hi Andreas,

This is not a solution more my experience:

I'm using a 4 Core 3.2 GHz on a Fedora 20 system and for a at91sam9x5ek_defconfig it takes about 7 min from 0 (without an downloaded DLs) to a bootable Image.

During build I'm using GKrellM System Monitor and and shows me that all cores are in use. ATM I'm ok with these results, since i started with a compile time about 45 min.

Regards,

Frank

>>> Andreas Naumann <dev@andin.de> 07.02.14 9.52 Uhr >>>
Hi guys,

First of all kudos again for this very good tool! I'm using it almsot 
daily and am stoked about the functionality but simple usage at the same 
time.
Now one issue with build systems is always build time. It seems the 
speed is not impacted much by just spending more horsepower (cpu cores). 
When looking at the load I see very frequently that just one core is 
busy with one package, especially during the configure step. Some 
packages themselves are build with just one core, I guess we cant do 
very much about that.
However, why are independent packages not build in parallel? Are the 
dependencies too complex to make this work? Thanks for any insight!


cheers,
Andreas

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20140207/fb0e750a/attachment.html>

[Buildroot] Antw: build speedup possible?

[Samuel Martin]

Andreas, Maxime, Frank, all,


On Fri, Feb 7, 2014 at 10:00 AM, Frank Ihle <frank.ihle@hs-offenburg.de>
wrote:
>
> Hi Andreas,
>
> This is not a solution more my experience:
>
> I'm using a 4 Core 3.2 GHz on a Fedora 20 system and for a
at91sam9x5ek_defconfig it takes about 7 min from 0 (without an downloaded
DLs) to a bootable Image.
>
> During build I'm using GKrellM System Monitor and and shows me that all
cores are in use. ATM I'm ok with these results, since i started with a
compile time about 45 min.
>
> Regards,
>
> Frank
>
> >>> Andreas Naumann <dev@andin.de> 07.02.14 9.52 Uhr >>>
>
> Hi guys,
>
> First of all kudos again for this very good tool! I'm using it almsot
> daily and am stoked about the functionality but simple usage at the same
> time.
> Now one issue with build systems is always build time. It seems the
> speed is not impacted much by just spending more horsepower (cpu cores).
> When looking at the load I see very frequently that just one core is
> busy with one package, especially during the configure step. Some
> packages themselves are build with just one core, I guess we cant do
> very much about that.
> However, why are independent packages not build in parallel? Are the
> dependencies too complex to make this work? Thanks for any insight!

Well, this is a tricky thing. I see 3 points on this topic:
- building a package in parallel:
  We already do this, unless the package itself (more precisely its
build-system) does not support it.
  In such cases, it is explicitly stated in the *.mk file by setting
<PKG>_MAKE = $(MAKE1).
- the package configuration step:
  Unfortunately the package configuration is highly serialized and depends
on the build-system used
  by the package.
- building packages in parallel:
  This point tends to hit some Buildroot's principles:
  - simplicity;
  - and build reproducibility (don't know if this word really exists in
English...).
  Being able to reproduce build (mainly build failures) implies that the
sysroot should not change during
  the configuration/build steps of a package, so the sysroot should be
copied for each package built in
  parallel (iow, if 4 packages are built in parallel, there will be 4
sysroots at the same time, each of them
  with a content that may be slightly different one from the others). This
may introduce some complexity
  in Buildroot and it may also make build time worse due to the copies of
the sysroot for each package.


The patch series pointed by Maxime introduces support for building packages
in parallel, but does not
implement the sysroot copy thing.
During the Buildroot Developer Days in last November, it has been decided
that we won't enable building
packages in parallel (see:
http://elinux.org/Buildroot:DeveloperDaysELCE2013#Parallel_top-level_make ).


@others developers: feel free to correct me, if I speak with my ass ;)

BTW, this could certainly be added in the FAQ.


Regards,


--
Samuel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20140207/43563654/attachment.html>

[Buildroot] Antw: build speedup possible?

[Andreas Naumann]

Thanks for all your input!

I understand the issues with the implicit/optional dependencies which 
may vary because of unpredictable execution order during the config 
step. Since determinism is more important to me than speed I wont remove 
the .NOTPARALLEL directive for now.
However...

Am 07.02.2014 11:31, schrieb Samuel Martin:
> Andreas, Maxime, Frank, all,
>
>
> On Fri, Feb 7, 2014 at 10:00 AM, Frank Ihle <frank.ihle@hs-offenburg.de
> <mailto:frank.ihle@hs-offenburg.de>> wrote:
>  >
>  > Hi Andreas,
>  >
>  > This is not a solution more my experience:
>  >
>  > I'm using a 4 Core 3.2 GHz on a Fedora 20 system and for a
> at91sam9x5ek_defconfig it takes about 7 min from 0 (without an
> downloaded DLs) to a bootable Image.
>  >
>  > During build I'm using GKrellM System Monitor and and shows me that
> all cores are in use. ATM I'm ok with these results, since i started
> with a compile time about 45 min.
>  >
>  > Regards,
>  >
>  > Frank
>  >
>  > >>> Andreas Naumann <dev at andin.de <mailto:dev@andin.de>> 07.02.14
> 9.52 Uhr >>>
>  >
>  > Hi guys,
>  >
>  > First of all kudos again for this very good tool! I'm using it almsot
>  > daily and am stoked about the functionality but simple usage at the same
>  > time.
>  > Now one issue with build systems is always build time. It seems the
>  > speed is not impacted much by just spending more horsepower (cpu cores).
>  > When looking at the load I see very frequently that just one core is
>  > busy with one package, especially during the configure step. Some
>  > packages themselves are build with just one core, I guess we cant do
>  > very much about that.
>  > However, why are independent packages not build in parallel? Are the
>  > dependencies too complex to make this work? Thanks for any insight!
>
> Well, this is a tricky thing. I see 3 points on this topic:
> - building a package in parallel:
>    We already do this, unless the package itself (more precisely its
> build-system) does not support it.
>    In such cases, it is explicitly stated in the *.mk file by setting
> <PKG>_MAKE = $(MAKE1).
> - the package configuration step:
>    Unfortunately the package configuration is highly serialized and
> depends on the build-system used
>    by the package.
> - building packages in parallel:
>    This point tends to hit some Buildroot's principles:
>    - simplicity;
>    - and build reproducibility (don't know if this word really exists in
> English...).
>    Being able to reproduce build (mainly build failures) implies that
> the sysroot should not change during
>    the configuration/build steps of a package, so the sysroot should be
> copied for each package built in
>    parallel (iow, if 4 packages are built in parallel, there will be 4
> sysroots at the same time, each of them
>    with a content that may be slightly different one from the others).
> This may introduce some complexity
>    in Buildroot and it may also make build time worse due to the copies
> of the sysroot for each package.

... we have a hack around that sort of does separate sysroots. That is, 
before going to global staging we create a per packet staging (in order 
to create ipkgs).
Would it be possible to switch off parallelization just for the 
configure and staging steps?


> The patch series pointed by Maxime introduces support for building
> packages in parallel, but does not
> implement the sysroot copy thing.
> During the Buildroot Developer Days in last November, it has been
> decided that we won't enable building
> packages in parallel (see:
> http://elinux.org/Buildroot:DeveloperDaysELCE2013#Parallel_top-level_make ).
>
>
> @others developers: feel free to correct me, if I speak with my ass ;)
>
> BTW, this could certainly be added in the FAQ.
>
>
> Regards,
>
>
> --
> Samuel

[Buildroot] Antw: build speedup possible?

[Fabio Porcedda]

On Fri, Feb 7, 2014 at 2:35 PM, Andreas Naumann <dev@andin.de> wrote:
> Thanks for all your input!
>
> I understand the issues with the implicit/optional dependencies which may
> vary because of unpredictable execution order during the config step. Since
> determinism is more important to me than speed I wont remove the
> .NOTPARALLEL directive for now.
> However...
>
> Am 07.02.2014 11:31, schrieb Samuel Martin:
>>
>> Andreas, Maxime, Frank, all,
>>
>>
>> On Fri, Feb 7, 2014 at 10:00 AM, Frank Ihle <frank.ihle@hs-offenburg.de
>> <mailto:frank.ihle@hs-offenburg.de>> wrote:
>>  >
>>  > Hi Andreas,
>>  >
>>  > This is not a solution more my experience:
>>  >
>>  > I'm using a 4 Core 3.2 GHz on a Fedora 20 system and for a
>> at91sam9x5ek_defconfig it takes about 7 min from 0 (without an
>> downloaded DLs) to a bootable Image.
>>  >
>>  > During build I'm using GKrellM System Monitor and and shows me that
>> all cores are in use. ATM I'm ok with these results, since i started
>> with a compile time about 45 min.
>>  >
>>  > Regards,
>>  >
>>  > Frank
>>  >
>>  > >>> Andreas Naumann <dev at andin.de <mailto:dev@andin.de>> 07.02.14
>>
>> 9.52 Uhr >>>
>>  >
>>  > Hi guys,
>>  >
>>  > First of all kudos again for this very good tool! I'm using it almsot
>>  > daily and am stoked about the functionality but simple usage at the
>> same
>>  > time.
>>  > Now one issue with build systems is always build time. It seems the
>>  > speed is not impacted much by just spending more horsepower (cpu
>> cores).
>>  > When looking at the load I see very frequently that just one core is
>>  > busy with one package, especially during the configure step. Some
>>  > packages themselves are build with just one core, I guess we cant do
>>  > very much about that.
>>  > However, why are independent packages not build in parallel? Are the
>>  > dependencies too complex to make this work? Thanks for any insight!
>>
>> Well, this is a tricky thing. I see 3 points on this topic:
>> - building a package in parallel:
>>    We already do this, unless the package itself (more precisely its
>> build-system) does not support it.
>>    In such cases, it is explicitly stated in the *.mk file by setting
>> <PKG>_MAKE = $(MAKE1).
>> - the package configuration step:
>>    Unfortunately the package configuration is highly serialized and
>> depends on the build-system used
>>    by the package.
>> - building packages in parallel:
>>    This point tends to hit some Buildroot's principles:
>>    - simplicity;
>>    - and build reproducibility (don't know if this word really exists in
>> English...).
>>    Being able to reproduce build (mainly build failures) implies that
>> the sysroot should not change during
>>    the configuration/build steps of a package, so the sysroot should be
>> copied for each package built in
>>    parallel (iow, if 4 packages are built in parallel, there will be 4
>> sysroots at the same time, each of them
>>    with a content that may be slightly different one from the others).
>> This may introduce some complexity
>>    in Buildroot and it may also make build time worse due to the copies
>> of the sysroot for each package.
>
>
> ... we have a hack around that sort of does separate sysroots. That is,
> before going to global staging we create a per packet staging (in order to
> create ipkgs).
> Would it be possible to switch off parallelization just for the configure
> and staging steps?

IMHO Switching off the parallelization for the configuration phase
does not make any sense.
The problem for the configuration phase is the building order and that
is changed even if you switch off the parallelization only for the
configuration phase.

>> The patch series pointed by Maxime introduces support for building
>> packages in parallel, but does not
>> implement the sysroot copy thing.
>> During the Buildroot Developer Days in last November, it has been
>> decided that we won't enable building
>> packages in parallel (see:
>> http://elinux.org/Buildroot:DeveloperDaysELCE2013#Parallel_top-level_make
>> ).
>>
>>
>> @others developers: feel free to correct me, if I speak with my ass ;)
>>
>> BTW, this could certainly be added in the FAQ.

Best regards
Fabio Porcedda

[Buildroot] Antw: build speedup possible?

[Fabio Porcedda]

On Fri, Feb 7, 2014 at 11:31 AM, Samuel Martin <s.martin49@gmail.com> wrote:
> Andreas, Maxime, Frank, all,
>
>
>
> On Fri, Feb 7, 2014 at 10:00 AM, Frank Ihle <frank.ihle@hs-offenburg.de>
> wrote:
>>
>> Hi Andreas,
>>
>> This is not a solution more my experience:
>>
>> I'm using a 4 Core 3.2 GHz on a Fedora 20 system and for a
>> at91sam9x5ek_defconfig it takes about 7 min from 0 (without an downloaded
>> DLs) to a bootable Image.
>>
>> During build I'm using GKrellM System Monitor and and shows me that all
>> cores are in use. ATM I'm ok with these results, since i started with a
>> compile time about 45 min.
>>
>> Regards,
>>
>> Frank
>>
>> >>> Andreas Naumann <dev@andin.de> 07.02.14 9.52 Uhr >>>
>>
>> Hi guys,
>>
>> First of all kudos again for this very good tool! I'm using it almsot
>> daily and am stoked about the functionality but simple usage at the same
>> time.
>> Now one issue with build systems is always build time. It seems the
>> speed is not impacted much by just spending more horsepower (cpu cores).
>> When looking at the load I see very frequently that just one core is
>> busy with one package, especially during the configure step. Some
>> packages themselves are build with just one core, I guess we cant do
>> very much about that.
>> However, why are independent packages not build in parallel? Are the
>> dependencies too complex to make this work? Thanks for any insight!
>
> Well, this is a tricky thing. I see 3 points on this topic:
> - building a package in parallel:
>   We already do this, unless the package itself (more precisely its
> build-system) does not support it.
>   In such cases, it is explicitly stated in the *.mk file by setting
> <PKG>_MAKE = $(MAKE1).
> - the package configuration step:
>   Unfortunately the package configuration is highly serialized and depends
> on the build-system used
>   by the package.
> - building packages in parallel:
>   This point tends to hit some Buildroot's principles:
>   - simplicity;
>   - and build reproducibility (don't know if this word really exists in
> English...).
>   Being able to reproduce build (mainly build failures) implies that the
> sysroot should not change during
>   the configuration/build steps of a package, so the sysroot should be
> copied for each package built in
>   parallel (iow, if 4 packages are built in parallel, there will be 4
> sysroots at the same time, each of them
>   with a content that may be slightly different one from the others). This
> may introduce some complexity
>   in Buildroot and it may also make build time worse due to the copies of
> the sysroot for each package.

The build can be reproduced even when building packages in parallel,
we just need to record the order of the steps and replay that order if
needed.

The sysroot copy is needed only to be sure that even if some package
has an unspecified optional dependency it generates every time the
same binary.

>
> The patch series pointed by Maxime introduces support for building packages
> in parallel, but does not
> implement the sysroot copy thing.
> During the Buildroot Developer Days in last November, it has been decided
> that we won't enable building
> packages in parallel (see:
> http://elinux.org/Buildroot:DeveloperDaysELCE2013#Parallel_top-level_make ).
>
>
> @others developers: feel free to correct me, if I speak with my ass ;)
>
> BTW, this could certainly be added in the FAQ.

I think it's a good idea. Maybe after merging the patch set.

I think that at least when doing development it will be useful to
build packages in parallel.
Maybe it will be best to avoid building packages in parallel just for releases.

Best regards
Fabio Porcedda

[Buildroot] Antw: build speedup possible?

[Fabio Porcedda]

On Fri, Feb 7, 2014 at 3:48 PM, Fabio Porcedda <fabio.porcedda@gmail.com> wrote:
> On Fri, Feb 7, 2014 at 11:31 AM, Samuel Martin <s.martin49@gmail.com> wrote:
>> Andreas, Maxime, Frank, all,
>>
>>
>>
>> On Fri, Feb 7, 2014 at 10:00 AM, Frank Ihle <frank.ihle@hs-offenburg.de>
>> wrote:
>>>
>>> Hi Andreas,
>>>
>>> This is not a solution more my experience:
>>>
>>> I'm using a 4 Core 3.2 GHz on a Fedora 20 system and for a
>>> at91sam9x5ek_defconfig it takes about 7 min from 0 (without an downloaded
>>> DLs) to a bootable Image.
>>>
>>> During build I'm using GKrellM System Monitor and and shows me that all
>>> cores are in use. ATM I'm ok with these results, since i started with a
>>> compile time about 45 min.
>>>
>>> Regards,
>>>
>>> Frank
>>>
>>> >>> Andreas Naumann <dev@andin.de> 07.02.14 9.52 Uhr >>>
>>>
>>> Hi guys,
>>>
>>> First of all kudos again for this very good tool! I'm using it almsot
>>> daily and am stoked about the functionality but simple usage at the same
>>> time.
>>> Now one issue with build systems is always build time. It seems the
>>> speed is not impacted much by just spending more horsepower (cpu cores).
>>> When looking at the load I see very frequently that just one core is
>>> busy with one package, especially during the configure step. Some
>>> packages themselves are build with just one core, I guess we cant do
>>> very much about that.
>>> However, why are independent packages not build in parallel? Are the
>>> dependencies too complex to make this work? Thanks for any insight!
>>
>> Well, this is a tricky thing. I see 3 points on this topic:
>> - building a package in parallel:
>>   We already do this, unless the package itself (more precisely its
>> build-system) does not support it.
>>   In such cases, it is explicitly stated in the *.mk file by setting
>> <PKG>_MAKE = $(MAKE1).
>> - the package configuration step:
>>   Unfortunately the package configuration is highly serialized and depends
>> on the build-system used
>>   by the package.
>> - building packages in parallel:
>>   This point tends to hit some Buildroot's principles:
>>   - simplicity;
>>   - and build reproducibility (don't know if this word really exists in
>> English...).
>>   Being able to reproduce build (mainly build failures) implies that the
>> sysroot should not change during
>>   the configuration/build steps of a package, so the sysroot should be
>> copied for each package built in
>>   parallel (iow, if 4 packages are built in parallel, there will be 4
>> sysroots at the same time, each of them
>>   with a content that may be slightly different one from the others). This
>> may introduce some complexity
>>   in Buildroot and it may also make build time worse due to the copies of
>> the sysroot for each package.
>
> The build can be reproduced even when building packages in parallel,
> we just need to record the order of the steps and replay that order if
> needed.
>
> The sysroot copy is needed only to be sure that even if some package
> has an unspecified optional dependency it generates every time the
> same binary.

I just want to clarify that if you know the unspecified optional
dependency, the problem can be fixed just adding the unspecified
dependency to the <package>.mk file.

It's just that we don't know a easy way to discover such dependencies.
Someone has any idea?

>>
>> The patch series pointed by Maxime introduces support for building packages
>> in parallel, but does not
>> implement the sysroot copy thing.
>> During the Buildroot Developer Days in last November, it has been decided
>> that we won't enable building
>> packages in parallel (see:
>> http://elinux.org/Buildroot:DeveloperDaysELCE2013#Parallel_top-level_make ).
>>
>>
>> @others developers: feel free to correct me, if I speak with my ass ;)
>>
>> BTW, this could certainly be added in the FAQ.
>
> I think it's a good idea. Maybe after merging the patch set.
>
> I think that at least when doing development it will be useful to
> build packages in parallel.
> Maybe it will be best to avoid building packages in parallel just for releases.

Best regards
-- 
Fabio Porcedda

[Buildroot] Antw: build speedup possible?

[Thomas Petazzoni]

Dear Fabio Porcedda,

On Fri, 7 Feb 2014 15:48:35 +0100, Fabio Porcedda wrote:

> The build can be reproduced even when building packages in parallel,
> we just need to record the order of the steps and replay that order if
> needed.

And in practice, how do you achieve this? When building in parallel,
how do you record the fact that the test for libfoo in package A
configure script was run before or after the installation of libfoo in
the global sysroot? It's not only about recording the order of the
build of the package: it's knowing whether each specific test of a
configure script was run before or after the corresponding library was
installed in the global sysroot.

> The sysroot copy is needed only to be sure that even if some package
> has an unspecified optional dependency it generates every time the
> same binary.

Which is *the* very important thing. So saying it's needed "only to be
sure" is quite minimizing the very important consequence of parallel
build with a global sysroot, IMO.

> I think that at least when doing development it will be useful to
> build packages in parallel.
> Maybe it will be best to avoid building packages in parallel just for releases.

I don't agree.

I do a lot of support on #buildroot, and I would *hate* to have people
coming up with bug reports that are not reproducible because they
depend on the specific scheduling of the build. I really don't want new
comers to use this feature and then come back with bug reports that are
basically unfixable because they cannot be reproduced.

I think we should really not minimize the support problem that
parallelization without per-package sysroot is going to cause.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Antw: build speedup possible?

[Fabio Porcedda]

On Fri, Feb 7, 2014 at 4:29 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Fabio Porcedda,
>
> On Fri, 7 Feb 2014 15:48:35 +0100, Fabio Porcedda wrote:
>
>> The build can be reproduced even when building packages in parallel,
>> we just need to record the order of the steps and replay that order if
>> needed.
>
> And in practice, how do you achieve this? When building in parallel,
> how do you record the fact that the test for libfoo in package A
> configure script was run before or after the installation of libfoo in
> the global sysroot? It's not only about recording the order of the
> build of the package: it's knowing whether each specific test of a
> configure script was run before or after the corresponding library was
> installed in the global sysroot.

Well i admit i haven't thought about that corner case, I don't know an
easy solution for that.

>> The sysroot copy is needed only to be sure that even if some package
>> has an unspecified optional dependency it generates every time the
>> same binary.
>
> Which is *the* very important thing. So saying it's needed "only to be
> sure" is quite minimizing the very important consequence of parallel
> build with a global sysroot, IMO.

Well minimizing was not my intention, but you are right to point it out.

>> I think that at least when doing development it will be useful to
>> build packages in parallel.
>> Maybe it will be best to avoid building packages in parallel just for releases.
>
> I don't agree.
>
> I do a lot of support on #buildroot, and I would *hate* to have people
> coming up with bug reports that are not reproducible because they
> depend on the specific scheduling of the build. I really don't want new
> comers to use this feature and then come back with bug reports that are
> basically unfixable because they cannot be reproduced.
>
> I think we should really not minimize the support problem that
> parallelization without per-package sysroot is going to cause.

My suggestion was based on the assumption that there is a way to
reproduce the exact build order.
But if that is not true i agree with you.

Best regards
-- 
Fabio Porcedda

[Buildroot] Antw: build speedup possible?

[Thomas Petazzoni]

Dear Fabio Porcedda,

On Fri, 7 Feb 2014 16:54:20 +0100, Fabio Porcedda wrote:

> > And in practice, how do you achieve this? When building in parallel,
> > how do you record the fact that the test for libfoo in package A
> > configure script was run before or after the installation of libfoo in
> > the global sysroot? It's not only about recording the order of the
> > build of the package: it's knowing whether each specific test of a
> > configure script was run before or after the corresponding library was
> > installed in the global sysroot.
> 
> Well i admit i haven't thought about that corner case, I don't know an
> easy solution for that.

IMO this is not a corner case, this is precisely the problem we have
been pointing out since the beginning when mixing a global sysroot with
top-level parallelization.

> > I think we should really not minimize the support problem that
> > parallelization without per-package sysroot is going to cause.
> 
> My suggestion was based on the assumption that there is a way to
> reproduce the exact build order.
> But if that is not true i agree with you.

I don't really see how to reproduce the exact build order, so the
support problem will be here. That's really why I have been a bit
skeptical and resistant to this patch series from the very beginning. I
am obviously very interested in ways to reduce the build time. But
doing that at the expense of reliability is not a good idea in my
opinion, as users will face spurious failures that we will not be able
to reproduce and fix.

If we want to get this to the next level, someone need to experiment
with per-package sysroot. Of course, this would be a long experiment,
without necessarily at the end the guarantee that we will accept it,
because it may be too complex to match Buildroot simplicity principle.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Antw: build speedup possible?

[Fabio Porcedda]

On Fri, Feb 7, 2014 at 5:43 PM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Dear Fabio Porcedda,
>
> On Fri, 7 Feb 2014 16:54:20 +0100, Fabio Porcedda wrote:
>
>> > And in practice, how do you achieve this? When building in parallel,
>> > how do you record the fact that the test for libfoo in package A
>> > configure script was run before or after the installation of libfoo in
>> > the global sysroot? It's not only about recording the order of the
>> > build of the package: it's knowing whether each specific test of a
>> > configure script was run before or after the corresponding library was
>> > installed in the global sysroot.
>>
>> Well i admit i haven't thought about that corner case, I don't know an
>> easy solution for that.
>
> IMO this is not a corner case, this is precisely the problem we have
> been pointing out since the beginning when mixing a global sysroot with
> top-level parallelization.

With speaking about the corner case i mean the overlapping of the
configuration and installation steps,
because even when configurations and installations steps do not
overlap  there is still the problem of the building order,
the solution of replaying the steps solve only this issue, as you pointed out.

>> > I think we should really not minimize the support problem that
>> > parallelization without per-package sysroot is going to cause.
>>
>> My suggestion was based on the assumption that there is a way to
>> reproduce the exact build order.
>> But if that is not true i agree with you.
>
> I don't really see how to reproduce the exact build order, so the
> support problem will be here. That's really why I have been a bit
> skeptical and resistant to this patch series from the very beginning. I
> am obviously very interested in ways to reduce the build time. But
> doing that at the expense of reliability is not a good idea in my
> opinion, as users will face spurious failures that we will not be able
> to reproduce and fix.

About the spurious failures when building packages in parallel  IMHO
they depends only on the building order of the finished steps.
If a build fails IMHO it depends on something that is not available so
those failures can be reproduced considering only the finished steps
and ignoring the overlapping unfinished steps.
What do you think about that?

> If we want to get this to the next level, someone need to experiment
> with per-package sysroot. Of course, this would be a long experiment,
> without necessarily at the end the guarantee that we will accept it,
> because it may be too complex to match Buildroot simplicity principle.

Best regards
-- 
Fabio Porcedda

[Buildroot] Antw: build speedup possible?

[Thomas Petazzoni]

Dear Fabio Porcedda,

On Fri, 7 Feb 2014 19:05:34 +0100, Fabio Porcedda wrote:

> > I don't really see how to reproduce the exact build order, so the
> > support problem will be here. That's really why I have been a bit
> > skeptical and resistant to this patch series from the very beginning. I
> > am obviously very interested in ways to reduce the build time. But
> > doing that at the expense of reliability is not a good idea in my
> > opinion, as users will face spurious failures that we will not be able
> > to reproduce and fix.
> 
> About the spurious failures when building packages in parallel  IMHO
> they depends only on the building order of the finished steps.
> If a build fails IMHO it depends on something that is not available so
> those failures can be reproduced considering only the finished steps
> and ignoring the overlapping unfinished steps.
> What do you think about that?

Of course even with parallel build it is *possible* to understand what
happened. But it is much more complicated because we cannot simply take
the user .config file, replay the build, and reproduce the issue. We
would need to have access to the user's complete build log to see in
which order things happened, and from this get an understanding of what
the issue could be. And the thing is that 99% of the time, users do not
save their build log into a file, so when they come on
IRC/bugtracker/mailing list for support, they only have the last few
lines of build output.

We're not talking about theory here, but practice. In theory, all
parallel build problems can be solved. In practice, it will make
supporting users much more complicated, as we will not be able to
reproduce their builds, and we won't have enough informations about the
order in which things happened.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Antw: build speedup possible?

[Andreas Naumann]

Hi,

Am 07.02.2014 17:43, schrieb Thomas Petazzoni:

>
> If we want to get this to the next level, someone need to experiment
> with per-package sysroot. Of course, this would be a long experiment,
> without necessarily at the end the guarantee that we will accept it,
> because it may be too complex to match Buildroot simplicity principle.
>

as I said, we have hacked br to a per-package sysroot in the past. This 
was in order to create isolated ipkgs and did its job. Even if we dont 
solve any of the dependency issues with that, it could be a starting 
point. If this is of  interest I can try to go further with it.

And for the depency issue, how does OE solve it?


cheers,
Andreas



> Best regards,
>
> Thomas
>

[Buildroot] Antw: build speedup possible?

[Thomas Petazzoni]

Dear Andreas Naumann,

On Mon, 10 Feb 2014 12:31:08 +0100, Andreas Naumann wrote:

> > If we want to get this to the next level, someone need to experiment
> > with per-package sysroot. Of course, this would be a long experiment,
> > without necessarily at the end the guarantee that we will accept it,
> > because it may be too complex to match Buildroot simplicity principle.
> 
> as I said, we have hacked br to a per-package sysroot in the past. This 
> was in order to create isolated ipkgs and did its job. Even if we dont 
> solve any of the dependency issues with that, it could be a starting 
> point. If this is of  interest I can try to go further with it.

Investigation in this area are definitely welcome. But as I said, we
cannot guarantee that it will be merged: it will highly depends on how
much additional complexity it will add.

> And for the depency issue, how does OE solve it?

I don't know exactly. The author of oe-lite (http://oe-lite.org/) told
me that they don't do per-package sysroot (which in his opinion is
fundamentally wrong when you do parallel build of multiple packages),
and it indeed seems to be the case from a quick look at what Yocto is
doing. That being said, my knowledge of OE/Yocto internals is almost
non-existent, so I don't know how they solve this problem.

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com