bitbake weirdness during -k world

bitbake weirdness during -k world

[Graham Gower]

I have a 1gb world.log file after running:
bitbake -k world | tee world.log

The build appears to have gotten about half way through (compared with
previous builds) and then begun infinitely repeating the following message.

Traceback (most recent call last):
  File "/home/grg/oe2/bitbake/lib/bb/server/none.py", line 128, in idle_commands
    retval = function(self, data, False)
  File "/home/grg/oe2/bitbake/lib/bb/cooker.py", line 716, in buildTargetsIdle
    retval = rq.execute_runqueue()
  File "/home/grg/oe2/bitbake/lib/bb/runqueue.py", line 877, in execute_runqueue
    self.execute_runqueue_internal()
  File "/home/grg/oe2/bitbake/lib/bb/runqueue.py", line 992, in execute_runqueue_internal
    self.build_pipes[pipe].read()
  File "/home/grg/oe2/bitbake/lib/bb/runqueue.py", line 1227, in read
    bb.event.fire_from_worker(self.queue[:index+8], self.d)
  File "/home/grg/oe2/bitbake/lib/bb/event.py", line 137, in fire_from_worker
    event = pickle.loads(event[7:-8])
  File "/usr/lib64/python2.6/pickle.py", line 1374, in loads
    return Unpickler(file).load()
  File "/usr/lib64/python2.6/pickle.py", line 858, in load
    dispatch[key](self)
  File "/usr/lib64/python2.6/pickle.py", line 966, in load_string
    raise ValueError, "insecure string pickle"
ValueError: insecure string pickle


I identified the location where the message begins and it is not possible
to link it to any other errors nor to a particular recipe. Context looks like this:

NOTE: Unpacking ../oe/sources/libAppleWM-1.4.0.tar.bz2 to /mnt/oe/tmp/work/mipsel-oe-linux/libapplewm-1_1.4.0-r9.0/
NOTE: package dhcpcd-1.3.22-pl4-r1: task do_install: Succeeded
NOTE:   /usr/share/ti/ti-biosutils-tree/packages/xdoc/external.png
NOTE:   /usr/share/ti/ti-biosutils-tree/packages/xdoc/Bullet.png
Traceback (most recent call last):
  File "/home/grg/oe2/bitbake/lib/bb/server/none.py", line 128, in idle_commands
    retval = function(self, data, False)
  File "/home/grg/oe2/bitbake/lib/bb/cooker.py", line 716, in buildTargetsIdle
    retval = rq.execute_runqueue()



FYI, i do have this stupidity in my local.conf:
PARALLEL_MAKE = "-j 12"
BB_NUMBER_THREADS = "12"


-Graham

Re: bitbake weirdness during -k world

[Khem Raj]

On Sun, Nov 7, 2010 at 3:09 PM, Graham Gower <graham.gower@gmail.com> wrote:
> I have a 1gb world.log file after running:
> bitbake -k world | tee world.log
>

I have this very same problem too and its recent enough. We should be
able to find which commit is causing it (bisect bitbake)
I am using bitbake master are you using same too ?

> The build appears to have gotten about half way through (compared with
> previous builds) and then begun infinitely repeating the following message.
>
> Traceback (most recent call last):
>  File "/home/grg/oe2/bitbake/lib/bb/server/none.py", line 128, in idle_commands
>    retval = function(self, data, False)
>  File "/home/grg/oe2/bitbake/lib/bb/cooker.py", line 716, in buildTargetsIdle
>    retval = rq.execute_runqueue()
>  File "/home/grg/oe2/bitbake/lib/bb/runqueue.py", line 877, in execute_runqueue
>    self.execute_runqueue_internal()
>  File "/home/grg/oe2/bitbake/lib/bb/runqueue.py", line 992, in execute_runqueue_internal
>    self.build_pipes[pipe].read()
>  File "/home/grg/oe2/bitbake/lib/bb/runqueue.py", line 1227, in read
>    bb.event.fire_from_worker(self.queue[:index+8], self.d)
>  File "/home/grg/oe2/bitbake/lib/bb/event.py", line 137, in fire_from_worker
>    event = pickle.loads(event[7:-8])
>  File "/usr/lib64/python2.6/pickle.py", line 1374, in loads
>    return Unpickler(file).load()
>  File "/usr/lib64/python2.6/pickle.py", line 858, in load
>    dispatch[key](self)
>  File "/usr/lib64/python2.6/pickle.py", line 966, in load_string
>    raise ValueError, "insecure string pickle"
> ValueError: insecure string pickle
>
>
> I identified the location where the message begins and it is not possible
> to link it to any other errors nor to a particular recipe. Context looks like this:
>
> NOTE: Unpacking ../oe/sources/libAppleWM-1.4.0.tar.bz2 to /mnt/oe/tmp/work/mipsel-oe-linux/libapplewm-1_1.4.0-r9.0/
> NOTE: package dhcpcd-1.3.22-pl4-r1: task do_install: Succeeded
> NOTE:   /usr/share/ti/ti-biosutils-tree/packages/xdoc/external.png
> NOTE:   /usr/share/ti/ti-biosutils-tree/packages/xdoc/Bullet.png
> Traceback (most recent call last):
>  File "/home/grg/oe2/bitbake/lib/bb/server/none.py", line 128, in idle_commands
>    retval = function(self, data, False)
>  File "/home/grg/oe2/bitbake/lib/bb/cooker.py", line 716, in buildTargetsIdle
>    retval = rq.execute_runqueue()
>
>
>
> FYI, i do have this stupidity in my local.conf:
> PARALLEL_MAKE = "-j 12"
> BB_NUMBER_THREADS = "12"
>
>
> -Graham
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>

Re: bitbake weirdness during -k world

[Chris Larson]

On Mon, Nov 8, 2010 at 12:22 PM, Khem Raj <raj.khem@gmail.com> wrote:

> On Sun, Nov 7, 2010 at 3:09 PM, Graham Gower <graham.gower@gmail.com>
> wrote:
> > I have a 1gb world.log file after running:
> > bitbake -k world | tee world.log
> >
>
> I have this very same problem too and its recent enough. We should be
> able to find which commit is causing it (bisect bitbake)
>

Yes, please do a bisect to isolate the problem if possible.  Haven't run
into this one yet.
-- 
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics

Re: bitbake weirdness during -k world

[Graham Gower]

On 9 November 2010 05:52, Khem Raj <raj.khem@gmail.com> wrote:
> On Sun, Nov 7, 2010 at 3:09 PM, Graham Gower <graham.gower@gmail.com> wrote:
>> I have a 1gb world.log file after running:
>> bitbake -k world | tee world.log
>>
>
> I have this very same problem too and its recent enough. We should be
> able to find which commit is causing it (bisect bitbake)
> I am using bitbake master are you using same too ?
>

Yep, bitbake master.

Re: bitbake weirdness during -k world

[Graham Gower]

Problem(s) fixed by these commits.
http://cgit.openembedded.org/cgit.cgi/openembedded/commit/?id=b6d7a41d8ae0c916b217264b135998ac2d8bce4c
http://cgit.openembedded.org/cgit.cgi/openembedded/commit/?id=fe0b4102c33f7300aaf8b139ee2d73820e12d635

Thanks to Chris for pointing it out once I'd narrowed it down to a
particular recipe.

-Graham

Re: bitbake weirdness during -k world

[Khem Raj]

On Tue, Nov 9, 2010 at 3:16 PM, Graham Gower <graham.gower@gmail.com> wrote:
> Problem(s) fixed by these commits.
> http://cgit.openembedded.org/cgit.cgi/openembedded/commit/?id=b6d7a41d8ae0c916b217264b135998ac2d8bce4c
> http://cgit.openembedded.org/cgit.cgi/openembedded/commit/?id=fe0b4102c33f7300aaf8b139ee2d73820e12d635
>
> Thanks to Chris for pointing it out once I'd narrowed it down to a
> particular recipe.

hmmm I think we need to fix recipes/kaeilos/kaeilos-version.bb and
recipes/distro-version/distro-version.bb
too. Care to create a patch ?

>
> -Graham
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>