Re: [bitbake-devel] ValueError: insecure string pickle with latest bitbake

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Sun, 2014-02-23 at 10:06 +0100, Martin Jansa wrote:
> On Sat, Feb 22, 2014 at 01:07:36PM +0100, Martin Jansa wrote:
> > Hi,
> > 
> > last couple of days many of my jenkins builds failed with exception like
> > this:
> > 
> > ERROR: An uncaught exception occured in runqueue, please see the failure below:
> > ERROR: Running idle function
> > Traceback (most recent call last):
> >   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/cooker.py", line 1182, in buildTargetsIdle(server=<ProcessServer(ProcessServer-2, started)>, rq=<bb.runqueue.RunQueue instance at 0x16a95d40>, abort=False):
> >                  try:
> >     >                retval = rq.execute_runqueue()
> >                  except runqueue.TaskFailure as exc:
> >   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 1093, in RunQueue.execute_runqueue():
> >                  try:
> >     >                self.teardown_workers()
> >                  except:
> >   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 925, in RunQueue.teardown_workers():
> >              self.workerpipe = None
> >     >        self._teardown_worker(self.fakeworker, self.fakeworkerpipe)
> >              self.fakeworker = None
> >   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 906, in RunQueue._teardown_worker(worker=<subprocess.Popen object at 0xcb4e690>, workerpipe=<bb.runqueue.runQueuePipe instance at 0x11c17710>):
> >              while worker.returncode is None:
> >     >            workerpipe.read()
> >                  worker.poll()
> >   File "/home/jenkins/oe/shr-core-branches/shr-core/bitbake/lib/bb/runqueue.py", line 2095, in runQueuePipe.read():
> >                  while index != -1 and self.queue.startswith("<event>"):
> >     >                event = pickle.loads(self.queue[7:index])
> >                      bb.event.fire_from_worker(event, self.d)
> > ValueError: insecure string pickle
> > 
> > and it repeats the same exception very quickly (generated 3GB log in ~ hour)
> > 
> > It doesn't happen always, it seems to be triggered in different places in build.
> > 
> > Anyone else seeing this as well?
> > 
> > I'll add some debug output around it and wait for it to happen again.
> 
> I got one example now (not sure if this is the only one):
> 
> Because it shows runrpmdeps and I'm the only one so far seeing this, I guess it's caused by:
> http://lists.openembedded.org/pipermail/openembedded-core/2014-February/089632.html
> I'll try to revert that one from my builds

Ah, it that function is called as:

processed = list(pool.imap(oe.package.filedeprunner, pkglist))

i.e. we have multiple processes each running this function. If we print
output, the output stream has no synchronisation and can hence be
corrupted which would explain the value error. You can see an <event>
appear half way through your output...

Cheers,

Richard