Re: [bitbake-devel] regression of 'world' performance?

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Fri, 2024-02-09 at 16:59 +0100, Alexander Kanavin wrote:
> On Fri, 9 Feb 2024 at 16:43, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> 
> > There is effectively a "main loop" within bitbake, so execute() is
> > meant to do one thing, then return, on the assumption it will get
> > called straight back.
> > 
> > Since the "continue" makes it faster, it suggests something else
> > somewhere is being slow and as you mention, there are a load of other
> > code paths which could also have a similar issue.
> > 
> > Could you find out where the actual delay is, since it should just hget
> > a call straight back. Or is it the loop iteration itself which is slow?
> > Even with 20k entries, you'd think python would handle that.
> 
> It's a combination of having 20k entries, and that the loop stops at
> each of those entries, and returns from execute(), then execute()
> starts again and the loop is restarted from the beginning. So the
> result is processing 20000 entries to the power of two. I inserted the
> timings, iteration counters and additional debug calls into execute()
> specifically to confirm this, and they did.

Iterating 28k entries isn't an issue for python. Yes, it will be "slow"
when you put logging around it but I doubt it is the real problem.

What is more likely is one of the calls like:

self.check_dependencies(xxx)

getting called once per entry and that call being slow.

Can you try the profile -P data please. If it doesn't make sense, share
it somewhere and I'll take a look.

Cheers,

Richard