poky.lists.yoctoproject.org archive mirror
 help / color / mirror / Atom feed
From: Darren Hart <dvhart@linux.intel.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: poky <poky@yoctoproject.org>
Subject: Re: Build profiling, now in pictures
Date: Wed, 16 Feb 2011 11:47:26 -0800	[thread overview]
Message-ID: <4D5C29CE.9020605@linux.intel.com> (raw)
In-Reply-To: <1297791165.4945.281.camel@rex>

On 02/15/2011 09:32 AM, Richard Purdie wrote:
> We have just merged Beth's initial buildstats logging work. I was
> sitting wondering how to actually evaluate the numbers as I wanted to
> know "where are we spending the time?".
>
> It occurred to me that I wanted a graph very similar to that generated
> by bootchart. I looked around and found pyboootchartgui and then hacked
> it around a bit and coerced it to start producing charts like:
>
> http://tim.rpsys.net/bootchart.png
>
> which is the initial "pseudo-native" part of the build. This was simple
> enough to test with.
>
> I then tried graphing a poky-image-sato. To get a graph I could actually
> read, I stripped out any task taking less than 8 seconds and scaled the
> x axis from 25 units per second to one unit per second. The result was:
>
> http://tim.rpsys.net/bootchart2.png
> (warning this is a 2.7MB png)
>
> I also added in a little bit of colour coding for the second chart.
> Interestingly it looks like there is more yellow than green meaning
> configure is a bigger drain on the build time not that its
> unexpected :/.
>
> I quite enjoyed playing with this and on a serious note, the gradient of
> the task graph makes me a little suspicious of whether the overhead of
> launching tasks in bitbake itself is having some effect on build time.
> Certainly on the first graph there are some interesting latencies
> showing up.
>
> Anyhow, I think this is the first time bitbake's task execution has been
> visualised and there are some interesting things we can learn from it.
> I'm hoping this is a start of a much more detailed understanding of the
> build process with respect to performance.
>

Per Richard's request, I ran this on a large Westmere machine with 40 
physical cores (10 per socket).

BB_NUMBER_THREADS = "40"
PARALLEL_MAKE = "-j 40"
MACHINE=qemux86

Here's the poky-image-sato "buildchart" (1.6M):

http://dvhart.com/darren/yocto/buildchart-sato.png

For reference, total build time:

real    68m57.268s
user    424m23.566s
sys     211m30.272s

The higher core count results in a steeper gradient and makes the 
serialized build sections a bit more prominent. The two obvious 
serialization points being "eglibc" and the 
"openssl,ncurses,gettext,readline,sqlite,augeas,python,boost,perl" sequence.

--
Darren

> Cheers,
>
> Richard
>
>
>
> _______________________________________________
> poky mailing list
> poky@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/poky


-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel


  parent reply	other threads:[~2011-02-16 19:47 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-02-15 17:32 Build profiling, now in pictures Richard Purdie
2011-02-16  0:07 ` Richard Purdie
2011-02-16 19:47 ` Darren Hart [this message]
2011-02-17  2:23   ` Darren Hart

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4D5C29CE.9020605@linux.intel.com \
    --to=dvhart@linux.intel.com \
    --cc=poky@yoctoproject.org \
    --cc=richard.purdie@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).