Re: [bitbake-devel] regression of 'world' performance?

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Fri, 2024-02-09 at 21:44 +0100, Alexander Kanavin wrote:
> On Fri, 9 Feb 2024 at 18:02, Alexander Kanavin via
> lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org>
> wrote:
> > I worked around it by running 'bitbake world' without -P so it could
> > parse recipes successfully, then re-running with -P against pre-parsed
> > data. So profiles are being collected now.
> 
> Here it is: https://drive.google.com/file/d/1bM1MGJWU5uWRf6kE6AaENQLQKIbopnGW/view?usp=sharing
> 
> I pressed ctrl-c after some 20 minutes - I took a look into it then
> but somehow couldn't find anything related to execute(). The two
> collected profiles seemingly are just waiting on select().

This didn't make much sense to me so I ended up going digging.

The traceback you showed is reproducible if you trigger parsing while
using -P so a bug needs fixing there. You can avoid it with something
like:

bitbake -p
bitbake world -nP

However what I've then realised is that when we added the second thread
to cooker, we only dump the performance data for one of the threads,
not the other.

So I suspect we need to fix the profiling to dump the stats on the
second thread, then we'll see what the real issue is.

I can confirm that bitbake world -nP is running much slower than it
should, even just for oe-core.

Cheers,

Richard