Poky Project Discussions
 help / color / mirror / Atom feed
From: Richard Purdie <rpurdie@linux.intel.com>
To: "Xu, Dongxiao" <dongxiao.xu@intel.com>
Cc: "poky@yoctoproject.org" <poky@yoctoproject.org>
Subject: Re: About the operator "??="
Date: Wed, 08 Dec 2010 14:37:47 +0000	[thread overview]
Message-ID: <1291819067.1554.606.camel@rex> (raw)
In-Reply-To: <D5AB6E638E5A3E4B8F4406B113A5A19A314F2BAE@shsmsx501.ccr.corp.intel.com>

On Wed, 2010-12-08 at 19:59 +0800, Xu, Dongxiao wrote:
> Here I did a profile, see following results:
> 
> Total time 39.296 secs
> Here I ranked the result according to "cumtime" item.
> See "finalize" (33.150 secs) and "finalise" (20.597 secs), there are 13s difference. A lot of time is cost on the following code:
> 
>     for lazykey in bb.data.getVar("__lazy_assigned", d) or ():
>         if bb.data.getVar(lazykey, d) is None:
>             val = bb.data.getVarFlag(lazykey, "defaultval", d)
>             bb.data.setVar(lazykey, val, d)
> 
> 
>    Ordered by: cumulative time
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.020    0.020   39.419   39.419 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/ui/knotty.py:33(init)
>      1689    0.003    0.000   39.374    0.023 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:95(waitEvent)
>       765    0.002    0.000   39.368    0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:121(idle_commands)
>       765    0.001    0.000   39.115    0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:159(runCommands)
>       765    0.002    0.000   39.114    0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/command.py:78(runAsyncCommand)
>       765    0.002    0.000   39.110    0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:766(updateCache)
>       764    0.054    0.000   39.058    0.051 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:995(parse_next)
>       764    0.007    0.000   38.166    0.050 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:181(loadData)
>       764    0.010    0.000   36.523    0.048 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:462(load_bbfile)
>  4302/764    0.016    0.000   36.435    0.048 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/__init__.py:71(handle)
>  4306/764    0.068    0.000   36.429    0.048 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/parse_py/BBHandler.py:109(handle)
>       764    0.008    0.000   33.951    0.044 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:365(multi_finalize)
>       957    0.308    0.000   33.150    0.035 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:303(finalize)
>       923    0.086    0.000   20.597    0.022 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:92(finalise)
>       923    1.310    0.001   19.847    0.022 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:60(_build_data)
>       923    1.593    0.002   16.898    0.018 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:299(generate_dependencies)
>    287898    2.043    0.000   12.702    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:271(build_dependencies)
> 928560/459862    1.961    0.000   11.680    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:86(expandWithRefs)
> 1517574/1198288    0.990    0.000   11.241    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:246(getVar)
> 525304/269546    1.332    0.000   10.041    0.000 {built-in method sub}
> 640662/347252    0.373    0.000    8.484    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:116(expand)
> 523659/318900    0.838    0.000    7.917    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:51(var_sub)
>    305565    1.620    0.000    7.658    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:212(setVar)
>   3652690    2.389    0.000    6.970    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:285(getVarFlag)
>    225169    0.129    0.000    6.047    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:81(setVar)
> 41959/34572    0.307    0.000    4.718    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:63(python_sub)
> 360953/338884    0.206    0.000    3.933    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:86(getVar)
> 
> 
> 
> If replace all the "??=" with "?=", and re-run the profile, the time for finalize and finalise decrease a lot and the difference between the two functions is about 6 secs.
> 
> Total time: 32.708 secs. (20% time saving)
> 
>    Ordered by: cumulative time
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.019    0.019   32.828   32.828 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/ui/knotty.py:33(init)
>      1689    0.003    0.000   32.785    0.019 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:95(waitEvent)
>       765    0.002    0.000   32.779    0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/server/none.py:121(idle_commands)
>       765    0.001    0.000   32.526    0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:159(runCommands)
>       765    0.002    0.000   32.525    0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/command.py:78(runAsyncCommand)
>       765    0.002    0.000   32.522    0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:766(updateCache)
>       764    0.053    0.000   32.488    0.043 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cooker.py:995(parse_next)
>       764    0.007    0.000   31.610    0.041 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:181(loadData)
>       764    0.009    0.000   29.994    0.039 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/cache.py:462(load_bbfile)
>  4302/764    0.015    0.000   29.907    0.039 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/__init__.py:71(handle)
>  4306/764    0.066    0.000   29.901    0.039 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/parse_py/BBHandler.py:109(handle)
>       764    0.008    0.000   27.475    0.036 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:365(multi_finalize)
>       957    0.056    0.000   26.675    0.028 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/parse/ast.py:303(finalize)
>       923    0.085    0.000   20.561    0.022 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:92(finalise)
>       923    1.290    0.001   19.810    0.021 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/siggen.py:60(_build_data)
>       923    1.601    0.002   16.901    0.018 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:299(generate_dependencies)
>    287898    1.845    0.000   12.768    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data.py:271(build_dependencies)
> 928560/459862    1.913    0.000   11.589    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:86(expandWithRefs)
> 1358764/1039478    0.937    0.000   10.748    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:246(getVar)
> 525304/269546    1.328    0.000    9.979    0.000 {built-in method sub}
> 640662/347252    0.366    0.000    8.370    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:116(expand)
> 523659/318900    0.833    0.000    7.802    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:51(var_sub)
>   3337869    2.138    0.000    6.174    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:285(getVarFlag)
> 41959/34572    0.271    0.000    4.643    0.000 /home/dongxiao/poky/scripts/..//bitbake/lib/bb/data_smart.py:63(python_sub)

Thanks, that is quite a significant difference.

Could you try working out how often getVar returns a "None" value? I'm
wondering if we should add something to getVar that if it is about to
return None, it checks for the default value flag and returns that if
set. Originally ??= wasn't implemented that way do to performance
concerns but I think we need to recheck that!

Cheers,

Richard



  reply	other threads:[~2010-12-08 14:38 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-12-01  1:57 About the operator "??=" Xu, Dongxiao
2010-12-08 10:30 ` Richard Purdie
2010-12-08 11:35   ` Tian, Kevin
2010-12-08 11:59     ` Xu, Dongxiao
2010-12-08 14:37       ` Richard Purdie [this message]
2010-12-09  3:32         ` Xu, Dongxiao
2010-12-09  4:57           ` Tian, Kevin
2010-12-09  7:59             ` Xu, Dongxiao
2010-12-09 11:24               ` Richard Purdie
2010-12-09 12:13                 ` Xu, Dongxiao
2010-12-09 12:45                   ` Richard Purdie
2010-12-09 14:44                     ` Xu, Dongxiao
2010-12-10 23:42                       ` Richard Purdie
2010-12-10 23:58                         ` Chris Larson
2010-12-13 16:24                           ` Scott Garman
2010-12-11  2:48                         ` Xu, Dongxiao
2010-12-08 12:49     ` Tian, Kevin
2010-12-08 13:02       ` Xu, Dongxiao
  -- strict thread matches above, loose matches on Subject: below --
2010-12-08  3:00 Xu, Dongxiao

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1291819067.1554.606.camel@rex \
    --to=rpurdie@linux.intel.com \
    --cc=dongxiao.xu@intel.com \
    --cc=poky@yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox