Re: Build profiling, now in pictures

[Richard Purdie <richard.purdie@linuxfoundation.org>]

This is a chart of pseudo-native as of master before the last few
bitbake commits:

http://tim.rpsys.net/bootchart-pseudo1.png

and after the recent bitbake commits, this is the result:

http://tim.rpsys.net/bootchart-pseudo1.png

Whats interesting in the gradient of the rapid fire tasks at the start
of the graph. There is now much less "free" space between the tasks and
task execution and throughput was therefore improved and the gradient
much sharper.

Looking at the graphs, the new task execution order is suboptimal
though, mainly as I changed something there that before realising there
was another problem that was making this hurt more than it should.

I think reverting:

http://git.pokylinux.org/cgit.cgi/poky/commit/?id=974ea1a190167dcfd831ba1fc5f733e0dc9a6fda

should help parallelism a little further.

I'm running a full build to see what this means in read world numbers.
It certainly executed all the non-critical path tasks faster with these
changes. I pulled the graph of the first 1000 tasks and they executed in
a faster time period, its difficult to say what the effect on the
overall build will be at this point though.

Interesting what some numbers can do to point out problems though.

Just for the record, I used a dummy recipe of empty tasks to test how
fast the execution code was combined with bitbake profiling and some
suitable tweaks to the looping delays to make it clear when bitbake was
sleeping and shouldn't be. These changes dropped the test time from 9
seconds to 5 (where the timing includes bitbake startup and cache
loading overheads).

Cheers,

Richard