From: Richard Purdie <rpurdie@linux.intel.com>
To: "Xu, Dongxiao" <dongxiao.xu@intel.com>
Cc: "poky@yoctoproject.org" <poky@yoctoproject.org>
Subject: Re: About the operator "??="
Date: Wed, 08 Dec 2010 14:37:47 +0000 [thread overview]
Message-ID: <1291819067.1554.606.camel@rex> (raw)
In-Reply-To: <D5AB6E638E5A3E4B8F4406B113A5A19A314F2BAE@shsmsx501.ccr.corp.intel.com>
On Wed, 2010-12-08 at 19:59 +0800, Xu, Dongxiao wrote:
> Here I did a profile, see following results:
>
> Total time 39.296 secs
> Here I ranked the result according to "cumtime" item.
> See "finalize" (33.150 secs) and "finalise" (20.597 secs), there are 13s difference. A lot of time is cost on the following code:
>
> for lazykey in bb.data.getVar("__lazy_assigned", d) or ():
> if bb.data.getVar(lazykey, d) is None:
> val = bb.data.getVarFlag(lazykey, "defaultval", d)
> bb.data.setVar(lazykey, val, d)
>
>
> Ordered by: cumulative time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 1 0.020 0.020 39.419 39.419 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/ui/knotty.py:33(init)
> 1689 0.003 0.000 39.374 0.023 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:95(waitEvent)
> 765 0.002 0.000 39.368 0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:121(idle_commands)
> 765 0.001 0.000 39.115 0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:159(runCommands)
> 765 0.002 0.000 39.114 0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/command.py:78(runAsyncCommand)
> 765 0.002 0.000 39.110 0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:766(updateCache)
> 764 0.054 0.000 39.058 0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:995(parse_next)
> 764 0.007 0.000 38.166 0.050 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:181(loadData)
> 764 0.010 0.000 36.523 0.048 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:462(load_bbfile)
> 4302/764 0.016 0.000 36.435 0.048 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/__init__.py:71(handle)
> 4306/764 0.068 0.000 36.429 0.048 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/parse_py/BBHandler.py:109(handle)
> 764 0.008 0.000 33.951 0.044 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:365(multi_finalize)
> 957 0.308 0.000 33.150 0.035 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:303(finalize)
> 923 0.086 0.000 20.597 0.022 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:92(finalise)
> 923 1.310 0.001 19.847 0.022 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:60(_build_data)
> 923 1.593 0.002 16.898 0.018 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:299(generate_dependencies)
> 287898 2.043 0.000 12.702 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:271(build_dependencies)
> 928560/459862 1.961 0.000 11.680 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:86(expandWithRefs)
> 1517574/1198288 0.990 0.000 11.241 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:246(getVar)
> 525304/269546 1.332 0.000 10.041 0.000 {built-in method sub}
> 640662/347252 0.373 0.000 8.484 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:116(expand)
> 523659/318900 0.838 0.000 7.917 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:51(var_sub)
> 305565 1.620 0.000 7.658 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:212(setVar)
> 3652690 2.389 0.000 6.970 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:285(getVarFlag)
> 225169 0.129 0.000 6.047 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:81(setVar)
> 41959/34572 0.307 0.000 4.718 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:63(python_sub)
> 360953/338884 0.206 0.000 3.933 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:86(getVar)
>
>
>
> If replace all the "??=" with "?=", and re-run the profile, the time for finalize and finalise decrease a lot and the difference between the two functions is about 6 secs.
>
> Total time: 32.708 secs. (20% time saving)
>
> Ordered by: cumulative time
>
> ncalls tottime percall cumtime percall filename:lineno(function)
> 1 0.019 0.019 32.828 32.828 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/ui/knotty.py:33(init)
> 1689 0.003 0.000 32.785 0.019 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:95(waitEvent)
> 765 0.002 0.000 32.779 0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:121(idle_commands)
> 765 0.001 0.000 32.526 0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:159(runCommands)
> 765 0.002 0.000 32.525 0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/command.py:78(runAsyncCommand)
> 765 0.002 0.000 32.522 0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:766(updateCache)
> 764 0.053 0.000 32.488 0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:995(parse_next)
> 764 0.007 0.000 31.610 0.041 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:181(loadData)
> 764 0.009 0.000 29.994 0.039 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:462(load_bbfile)
> 4302/764 0.015 0.000 29.907 0.039 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/__init__.py:71(handle)
> 4306/764 0.066 0.000 29.901 0.039 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/parse_py/BBHandler.py:109(handle)
> 764 0.008 0.000 27.475 0.036 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:365(multi_finalize)
> 957 0.056 0.000 26.675 0.028 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:303(finalize)
> 923 0.085 0.000 20.561 0.022 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:92(finalise)
> 923 1.290 0.001 19.810 0.021 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:60(_build_data)
> 923 1.601 0.002 16.901 0.018 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:299(generate_dependencies)
> 287898 1.845 0.000 12.768 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:271(build_dependencies)
> 928560/459862 1.913 0.000 11.589 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:86(expandWithRefs)
> 1358764/1039478 0.937 0.000 10.748 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:246(getVar)
> 525304/269546 1.328 0.000 9.979 0.000 {built-in method sub}
> 640662/347252 0.366 0.000 8.370 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:116(expand)
> 523659/318900 0.833 0.000 7.802 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:51(var_sub)
> 3337869 2.138 0.000 6.174 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:285(getVarFlag)
> 41959/34572 0.271 0.000 4.643 0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:63(python_sub)
Thanks, that is quite a significant difference.
Could you try working out how often getVar returns a "None" value? I'm
wondering if we should add something to getVar that if it is about to
return None, it checks for the default value flag and returns that if
set. Originally ??= wasn't implemented that way do to performance
concerns but I think we need to recheck that!
Cheers,
Richard
next prev parent reply other threads:[~2010-12-08 14:38 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-01 1:57 About the operator "??=" Xu, Dongxiao
2010-12-08 10:30 ` Richard Purdie
2010-12-08 11:35 ` Tian, Kevin
2010-12-08 11:59 ` Xu, Dongxiao
2010-12-08 14:37 ` Richard Purdie [this message]
2010-12-09 3:32 ` Xu, Dongxiao
2010-12-09 4:57 ` Tian, Kevin
2010-12-09 7:59 ` Xu, Dongxiao
2010-12-09 11:24 ` Richard Purdie
2010-12-09 12:13 ` Xu, Dongxiao
2010-12-09 12:45 ` Richard Purdie
2010-12-09 14:44 ` Xu, Dongxiao
2010-12-10 23:42 ` Richard Purdie
2010-12-10 23:58 ` Chris Larson
2010-12-13 16:24 ` Scott Garman
2010-12-11 2:48 ` Xu, Dongxiao
2010-12-08 12:49 ` Tian, Kevin
2010-12-08 13:02 ` Xu, Dongxiao
-- strict thread matches above, loose matches on Subject: below --
2010-12-08 3:00 Xu, Dongxiao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1291819067.1554.606.camel@rex \
--to=rpurdie@linux.intel.com \
--cc=dongxiao.xu@intel.com \
--cc=poky@yoctoproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.