* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-22 8:22 ` André Draszik
@ 2013-07-22 9:48 ` Phil Blundell
2013-07-24 2:19 ` Kang Kai
2013-07-25 5:53 ` Khem Raj
2 siblings, 0 replies; 10+ messages in thread
From: Phil Blundell @ 2013-07-22 9:48 UTC (permalink / raw)
To: André Draszik; +Cc: openembedded-core
Agreed, I think this sort of thing belongs in site.conf or some such
place, not in the recipe.
p.
On Mon, 2013-07-22 at 09:22 +0100, André Draszik wrote:
> Hi,
>
> Instead of penalising everybody, would it be possible to make this
> change dependent on the build machine (ram + swap)? Or to only add
> -Wl,--no-keep-memory if the build failed without it?
>
> Not sure if it would make any difference, but do you see this only with
> full debug enabled (-ggdb)? If yes, can the debug flags be changed, or
> that linker flag only be added, if full debug is enabled?
>
>
> Cheers,
> Andre'
>
> On Mon, 2013-07-22 at 15:51 +0800, Kai Kang wrote:
> > Build webkit-gtk occupies lot of memory and fails on some host:
> > | ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
> > failed to set dynamic section sizes: Memory exhausted
> > | collect2: error: ld returned 1 exit status
> > | make[1]: *** [libwebkitgtk-1.0.la] Error 1
> >
> > Add ld option to fix it.
> >
> > Refer to:
> > http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078
> >
> > Signed-off-by: Kai Kang <kai.kang@windriver.com>
> > ---
> > meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb b/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
> > index 5691d3f..90625a3 100644
> > --- a/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
> > +++ b/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
> > @@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \
> >
> > EXTRA_AUTORECONF = " -I Source/autotools "
> >
> > +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted"
> > +LDFLAGS += " -Wl,--no-keep-memory"
> >
> > #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]':
> > #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-22 8:22 ` André Draszik
2013-07-22 9:48 ` Phil Blundell
@ 2013-07-24 2:19 ` Kang Kai
2013-07-24 7:49 ` Burton, Ross
2013-07-25 5:53 ` Khem Raj
2 siblings, 1 reply; 10+ messages in thread
From: Kang Kai @ 2013-07-24 2:19 UTC (permalink / raw)
To: André Draszik; +Cc: openembedded-core
On 2013年07月22日 16:22, André Draszik wrote:
> Hi,
>
> Instead of penalising everybody, would it be possible to make this
> change dependent on the build machine (ram + swap)? Or to only add
> -Wl,--no-keep-memory if the build failed without it?
Yes, it doesn't fail on every build machine. But it seems that build
fails randomly on different build machines.
>
> Not sure if it would make any difference, but do you see this only with
> full debug enabled (-ggdb)? If yes, can the debug flags be changed, or
> that linker flag only be added, if full debug is enabled?
webkit-gtk set configure option "--enable-debug=no" to turn off debug.
And no such "-ggdb" flags.
Thanks,
Kai
>
>
> Cheers,
> Andre'
>
> On Mon, 2013-07-22 at 15:51 +0800, Kai Kang wrote:
>> Build webkit-gtk occupies lot of memory and fails on some host:
>> | ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
>> failed to set dynamic section sizes: Memory exhausted
>> | collect2: error: ld returned 1 exit status
>> | make[1]: *** [libwebkitgtk-1.0.la] Error 1
>>
>> Add ld option to fix it.
>>
>> Refer to:
>> http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078
>>
>> Signed-off-by: Kai Kang <kai.kang@windriver.com>
>> ---
>> meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb b/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
>> index 5691d3f..90625a3 100644
>> --- a/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
>> +++ b/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
>> @@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \
>>
>> EXTRA_AUTORECONF = " -I Source/autotools "
>>
>> +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted"
>> +LDFLAGS += " -Wl,--no-keep-memory"
>>
>> #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]':
>> #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
--
Regards,
Neil | Kai Kang
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-24 2:19 ` Kang Kai
@ 2013-07-24 7:49 ` Burton, Ross
2013-07-24 8:31 ` Paul Barker
0 siblings, 1 reply; 10+ messages in thread
From: Burton, Ross @ 2013-07-24 7:49 UTC (permalink / raw)
To: Kang Kai; +Cc: openembedded-core
On 24 July 2013 03:19, Kang Kai <Kai.Kang@windriver.com> wrote:
>> Instead of penalising everybody, would it be possible to make this
>> change dependent on the build machine (ram + swap)? Or to only add
>> -Wl,--no-keep-memory if the build failed without it?
>
> Yes, it doesn't fail on every build machine. But it seems that build fails
> randomly on different build machines.
I'm in agreement with the others - this is how individual build
machine admins make a webkit-gtk build work on their machine if it
doesn't have enough memory available. We should probably document how
to do this in local.conf in the recipe for reference, but not make it
a default.
Ross
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-24 7:49 ` Burton, Ross
@ 2013-07-24 8:31 ` Paul Barker
2013-07-24 9:47 ` Phil Blundell
0 siblings, 1 reply; 10+ messages in thread
From: Paul Barker @ 2013-07-24 8:31 UTC (permalink / raw)
To: Burton, Ross; +Cc: openembedded-core
On 24 July 2013 08:49, Burton, Ross <ross.burton@intel.com> wrote:
> On 24 July 2013 03:19, Kang Kai <Kai.Kang@windriver.com> wrote:
>>> Instead of penalising everybody, would it be possible to make this
>>> change dependent on the build machine (ram + swap)? Or to only add
>>> -Wl,--no-keep-memory if the build failed without it?
>>
>> Yes, it doesn't fail on every build machine. But it seems that build fails
>> randomly on different build machines.
>
> I'm in agreement with the others - this is how individual build
> machine admins make a webkit-gtk build work on their machine if it
> doesn't have enough memory available. We should probably document how
> to do this in local.conf in the recipe for reference, but not make it
> a default.
>
Perhaps create a bbclass for this that can be globally inherited in
local.conf? I ran into a similar thing with gnuradio and had to drop
PARALLEL_MAKE to get the build to complete with my amount of RAM+swap.
Something like "lean-build.bbclass" that handles known issues for
systems with limited memory.
Would variables like LDFLAGS_webkit-gtk and PARALLEL_MAKE_gnuradio be
applied to the appropriate recipes if they were set in local.conf or a
globally inherited bbclass, or am I misunderstanding how bitbake
parses variables?
--
Paul Barker
Email: paul@paulbarker.me.uk
http://www.paulbarker.me.uk
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-24 8:31 ` Paul Barker
@ 2013-07-24 9:47 ` Phil Blundell
2013-07-25 4:58 ` Randy MacLeod
0 siblings, 1 reply; 10+ messages in thread
From: Phil Blundell @ 2013-07-24 9:47 UTC (permalink / raw)
To: Paul Barker; +Cc: openembedded-core
On Wed, 2013-07-24 at 09:31 +0100, Paul Barker wrote:
> Would variables like LDFLAGS_webkit-gtk and PARALLEL_MAKE_gnuradio be
> applied to the appropriate recipes if they were set in local.conf or a
> globally inherited bbclass, or am I misunderstanding how bitbake
> parses variables?
LDFLAGS_pn-webkit-gtk and PARALLEL_MAKE_pn-gnuradio, yes. Any of that
stuff can go in site.conf and/or local.conf, and this is where this sort
of build-environment-specific customisation belongs.
p.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-24 9:47 ` Phil Blundell
@ 2013-07-25 4:58 ` Randy MacLeod
0 siblings, 0 replies; 10+ messages in thread
From: Randy MacLeod @ 2013-07-25 4:58 UTC (permalink / raw)
To: Phil Blundell; +Cc: openembedded-core
On 13-07-24 05:47 AM, Phil Blundell wrote:
> On Wed, 2013-07-24 at 09:31 +0100, Paul Barker wrote:
>> Would variables like LDFLAGS_webkit-gtk and PARALLEL_MAKE_gnuradio be
>> applied to the appropriate recipes if they were set in local.conf or a
>> globally inherited bbclass, or am I misunderstanding how bitbake
>> parses variables?
>
> LDFLAGS_pn-webkit-gtk and PARALLEL_MAKE_pn-gnuradio, yes. Any of that
> stuff can go in site.conf and/or local.conf, and this is where this sort
> of build-environment-specific customisation belongs.
I disagree.
The default behaviour should be that the system builds as long
as the hosts has some minimum RAM. If this means that we pay
a < ~5% build time penalty, we should not flinch. I'm not sure
how much extra time I'm willing to accept but for a given package
even a factor of two could be acceptable as long as the overall
build did not take twice as long.
Kai, when you have a chance to test this, can you tell us
what sort of build time differences we are talking about for
the full package as well as just for the link phase if that's easy
to extract? You could use /usr/bin/time to get the cpu and memory
usage.
Are we really expecting all system builders to track all
these limitations and put them in place on low-end machines?
I realize that we need, and most of us have, a beefy box for builds
but a few people are still trying to build using 32 bit, 4 GB systems
and many people likely expect that 8 GB of RAM is plenty, in fact
the system where this problem occurred had 8 GB and was running
with:
BB_NUMBER_THREADS ?= "2"
PARALLEL_MAKE ?= "-j 4"
so there shouldn't have been much memory pressure.
Do we have minimum build system specs?
Regardless of what default behaviour we pick, switching a build to be
tuned for a high/low end machine should be easy to do at
set-up time as Paul suggested. Embedding such limitations in
individual recipes and activating them by setting a single
variable in local.conf is appealing.
// Randy
> p.
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
--
# Randy MacLeod. SMTS, Linux, Wind River
Direct: 613.963.1350
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/1] webkit-gtk: fix 'Memory exhausted' error
2013-07-22 8:22 ` André Draszik
2013-07-22 9:48 ` Phil Blundell
2013-07-24 2:19 ` Kang Kai
@ 2013-07-25 5:53 ` Khem Raj
2 siblings, 0 replies; 10+ messages in thread
From: Khem Raj @ 2013-07-25 5:53 UTC (permalink / raw)
To: André Draszik; +Cc: openembedded-core
On Jul 22, 2013, at 1:22 AM, André Draszik <andre.draszik@linaro.org> wrote:
> Hi,
>
> Instead of penalising everybody, would it be possible to make this
> change dependent on the build machine (ram + swap)? Or to only add
> -Wl,--no-keep-memory if the build failed without it?
penalty is not as much when you do parallel builds. However it will be good
to measure how much extra built time is needed when this options is active.
if its regressing the build time considerably then we need to make a choice.
>
> Not sure if it would make any difference, but do you see this only with
> full debug enabled (-ggdb)? If yes, can the debug flags be changed, or
> that linker flag only be added, if full debug is enabled?
>
>
> Cheers,
> Andre'
>
> On Mon, 2013-07-22 at 15:51 +0800, Kai Kang wrote:
>> Build webkit-gtk occupies lot of memory and fails on some host:
>> | ...tmp/sysroots/i686-linux/usr/libexec/x86_64-wrs-linux/gcc/x86_64-wrs-linux/4.8.1/ld:
>> failed to set dynamic section sizes: Memory exhausted
>> | collect2: error: ld returned 1 exit status
>> | make[1]: *** [libwebkitgtk-1.0.la] Error 1
>>
>> Add ld option to fix it.
>>
>> Refer to:
>> http://comments.gmane.org/gmane.os.opendarwin.webkit.devel/20078
>>
>> Signed-off-by: Kai Kang <kai.kang@windriver.com>
>> ---
>> meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb b/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
>> index 5691d3f..90625a3 100644
>> --- a/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
>> +++ b/meta/recipes-sato/webkit/webkit-gtk_1.8.3.bb
>> @@ -60,6 +60,8 @@ CPPFLAGS_append_powerpc = " -I${STAGING_INCDIR}/pango-1.0 \
>>
>> EXTRA_AUTORECONF = " -I Source/autotools "
>>
>> +# Fix compile error: "ld: failed to set dynamic section sizes: Memory exhausted"
>> +LDFLAGS += " -Wl,--no-keep-memory"
>>
>> #| ./Source/JavaScriptCore/heap/HandleTypes.h: In static member function 'static T* JSC::HandleTypes<T>::getFromSlot(JSC::HandleSlot) [with T = JSC::Structure, JSC::HandleTypes<T>::ExternalType = JSC::Structure*, JSC::HandleSlot = JSC::JSValue*]':
>> #| ./Source/JavaScriptCore/heap/Handle.h:141:79: instantiated from 'JSC::Handle<T>::ExternalType JSC::Handle<T>::get() const [with T = JSC::Structure, JSC::Handle<T>::ExternalType = JSC::Structure*]'
>
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 10+ messages in thread