From: Markus Lehtonen <markus.lehtonen@linux.intel.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: bitbake-devel@lists.openembedded.org
Subject: Re: [PATCH 0/6] Improve caching task file dependency checksum caching
Date: Tue, 02 Feb 2016 18:23:41 +0200 [thread overview]
Message-ID: <1454430221.13214.72.camel@linux.intel.com> (raw)
In-Reply-To: <1454087371.10340.73.camel@linuxfoundation.org>
[-- Attachment #1: Type: text/plain, Size: 6178 bytes --]
Hi Richard,
On Fri, 2016-01-29 at 17:09 +0000, Richard Purdie wrote:
> On Tue, 2016-01-26 at 15:34 +0200, Markus Lehtonen wrote:
> > This patchset improves the caching of task file dependency checksums in two
> > ways. First, the patchset enables writing of the cache onto disk so that the
> > checksums can be re-used on subsequent runs of bitbake. Previously the task
> > signature checksums were calculated and cached in memory but never really
> > saved.
> >
> > Second, this patchset makes the task file dependency checksum cache file
> > configurable. This is intended to be used e.g. with a further patchset for
> > oe-core that will utilize task file dependencies heavily.
> >
> > [YOCTO #8853]
>
> Before I'd take this, I'd like to understand what this code does with a
> large codebase, for example a kernel source tree.
First, this patchset should improve the performance on "normal" builds
(i.e. where my accompaniying externalsrc patch to oe-core is not
enabled). For current oe-core master I get 1211 files that get checked -
currently they are MD5'd every time bitbake gets run. With this patchset
enabled only the mtime of each of the files need to be checked for most
of the time and we get the checksum from the cache.
I think my externalsrc patchset for oe-core is then slightly another
matter:
http://lists.openembedded.org/pipermail/openembedded-core/2016-January/116296.html
This patchset takes heavy use of the task file dependencies and I fully
understand your concern there. I should've provided some data to look
at.
> I'd really like some performance measurements for the file counts
> involved and the time it takes to stat trees like that. What is the
> command overhead in that fairly pathological case?
I did some profiling on my desktop machine (i7-3770K, 16GB of RAM, 2TB
HDD). I basically ran "bitbake core-image-sato" and profiled the end of
bb.runqueue.RunQueueData.prepare() which does the task hashing.
The test cases I used were
- bb master: bitbake master (only profiling code added), oe-core
master
- bb patched: bitbake with my patchset applied, oe-core master
- bb patched + linux-yocto: bitbake with my patchset applied, oe-core
with my other patchset applied and "devtool modify linux-yocto"
enabled where all files of the kernel source are added as deps
First, the results with kernel caches hot which is probably closer to
the usual use case.
# OF FILE CHECKSUM CUMTIME OF
# TESTCASE FILES CACHE STATE get_taskhash
------------------------------------------------------------------------
1 bb master 1211 cold 0.602
2 bb patched 1211 cold 0.599
3 bb patched 1211 warm 0.487
4 bb patched + linux-yocto 53519 cold 15.06
5 bb patched + linux-yocto 53519 warm 1.218
We can see that the improvement in a "normal" build is about 20% (#2 vs.
#3) but in absolute time this is neglible. Tests #1 and #2 are about the
same as they should as all the same files are hashed. In the "kernel
case" the first run takes quite long as the whole kernel source tree
(over 52000 files) is hashed. After this, the hashes are cached on disk
and on subsequent runs the overhead is quite small. The time taken by
get_taskhash drops to about one tenth.
Next, I made some tests with the kernel caches dropped before every test
run which represents certain worst case scenario and should give more
coherent results.
# OF FILE CHECKSUM CUMTIME OF
# TESTCASE FILES CACHE STATE get_taskhash
------------------------------------------------------------------------
6 bb master 1211 cold 2.222
7 bb patched 1211 cold 2.178
8 bb patched 1211 warm 0.576
9 bb patched + linux-yocto 53519 cold 80.039
10 bb patched + linux-yocto 53519 warm 8.496
Here the "kernel case" is quite rude as the whole source tree is read
and hashed from disk. After the on-disk file checksum cache has been
populated the numbers look better. Again, the time taken by get_taskhash
drops to about one tenth.
Loading of the cache file takes some time, too. On my machine loading
(pickle.load()) a "normal" cache with 1211 entries takes about 0.02
seconds. Loading the big linux-yocto polluted cache file (with 53519
entries) takes about 0.4 seconds. So, I think that should be bearable.
Note that all these results are from one run only, and thus, they are
not statistically reliable as such. But, they should give a ballpark
figure. All the cprofile results are found as attachments of this email,
if you want to take a closer look.
With these results I think that the bitbake patchset could be merged.
Normally, you would get slight performance improvement as the cache file
would contain a reasonable amount of entries. As said before, I think
that the "linux-yocto" case is somewhat separate from this patchset: in
that case you will get the same hashing penalty whether you have this
bitbake patchset applied or not. The difference is that with this
patchset applied the hashing penalty drops dramatically after the
checksums have been cached on disk.
> Would we be better off requiring memory resident bitbake and using
> inotify watches for example?
I don't know :) How would you "require" that?
When taking a closer look at the code I realized that the existing
caching code is partly dead. The checksum cache file written to disk
(local_file_checksum_cache.dat) is basically always empty.
FileChecksumCache class is effectively only used by the task hashing
code but the data gets never written onto disk. It doesn't make much
sense reading/writing an empty local_file_checksum_cache.dat every time.
I'll fix this in the next version of this patchset.
Cheers,
Markus
[-- Attachment #2: profile_01.txt --]
[-- Type: text/plain, Size: 5141 bytes --]
827014 function calls (827009 primitive calls) in 0.612 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4917 0.009 0.000 0.602 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4917 0.069 0.000 0.593 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:177(get_taskhash)
416 0.007 0.000 0.206 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/fetch2/__init__.py:1104(get_file_checksums)
23313 0.016 0.000 0.191 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
1211 0.001 0.000 0.188 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/fetch2/__init__.py:1112(checksum_file)
1211 0.003 0.000 0.187 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:69(get_checksum)
1211 0.124 0.000 0.181 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:512(md5_file)
23313 0.054 0.000 0.175 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
14820 0.013 0.000 0.061 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
4917 0.014 0.000 0.052 0.000 {sorted}
216557 0.052 0.000 0.052 0.000 {method 'update' of '_hashlib.HASH' objects}
29841 0.039 0.000 0.039 0.000 {method 'find' of 'str' objects}
23313 0.020 0.000 0.038 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:316(clean_basepath)
4917 0.011 0.000 0.035 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:168(read_taint)
15732 0.025 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
6128 0.028 0.000 0.028 0.000 {open}
23313 0.023 0.000 0.023 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
66338 0.019 0.000 0.019 0.000 {method 'rsplit' of 'str' objects}
32340 0.019 0.000 0.019 0.000 {method 'join' of 'str' objects}
16542 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
39065 0.007 0.000 0.007 0.000 {method 'startswith' of 'str' objects}
6128 0.007 0.000 0.007 0.000 {_hashlib.openssl_md5}
15732 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
16623 0.006 0.000 0.006 0.000 {method 'split' of 'str' objects}
105812 0.005 0.000 0.005 0.000 {len}
1214 0.001 0.000 0.005 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
2174 0.005 0.000 0.005 0.000 {posix.stat}
46451 0.005 0.000 0.005 0.000 {method 'append' of 'list' objects}
23313 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
16562 0.004 0.000 0.004 0.000 {method 'endswith' of 'str' objects}
6128 0.004 0.000 0.004 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
1211 0.001 0.000 0.003 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:39(cached_mtime)
15760 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
1214 0.000 0.000 0.001 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
4917 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
4917 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
416 0.000 0.000 0.000 0.000 {method 'sort' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/fetch2/__init__.py:1120(checksum_dir)
1211 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}
1216 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
9/4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:209(walk)
20 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
3 0.000 0.000 0.000 0.000 {posix.listdir}
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
2 0.000 0.000 0.000 0.000 {posix.lstat}
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
1 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #3: profile_02.txt --]
[-- Type: text/plain, Size: 7432 bytes --]
827145 function calls (827140 primitive calls) in 0.613 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4917 0.008 0.000 0.599 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4917 0.068 0.000 0.590 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
416 0.007 0.000 0.203 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
23313 0.016 0.000 0.192 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
1211 0.001 0.000 0.185 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
1211 0.003 0.000 0.184 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
1211 0.122 0.000 0.179 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:512(md5_file)
23313 0.055 0.000 0.176 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
14820 0.014 0.000 0.061 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
216557 0.051 0.000 0.051 0.000 {method 'update' of '_hashlib.HASH' objects}
4917 0.014 0.000 0.051 0.000 {sorted}
29841 0.038 0.000 0.038 0.000 {method 'find' of 'str' objects}
23313 0.020 0.000 0.037 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
4917 0.012 0.000 0.036 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
15732 0.025 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
6134 0.029 0.000 0.029 0.000 {open}
23313 0.024 0.000 0.024 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
66338 0.019 0.000 0.019 0.000 {method 'rsplit' of 'str' objects}
32340 0.019 0.000 0.019 0.000 {method 'join' of 'str' objects}
16542 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
15732 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
6128 0.007 0.000 0.007 0.000 {_hashlib.openssl_md5}
39071 0.007 0.000 0.007 0.000 {method 'startswith' of 'str' objects}
16623 0.006 0.000 0.006 0.000 {method 'split' of 'str' objects}
105820 0.005 0.000 0.005 0.000 {len}
1214 0.001 0.000 0.005 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
2184 0.005 0.000 0.005 0.000 {posix.stat}
23313 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
46451 0.004 0.000 0.004 0.000 {method 'append' of 'list' objects}
16563 0.004 0.000 0.004 0.000 {method 'endswith' of 'str' objects}
1 0.000 0.000 0.004 0.004 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
6128 0.004 0.000 0.004 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
1211 0.001 0.000 0.003 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
15760 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
1 0.000 0.000 0.002 0.002 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
2 0.002 0.001 0.002 0.001 {method 'dump' of 'cPickle.Pickler' objects}
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
1 0.000 0.000 0.001 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
1 0.001 0.001 0.001 0.001 {built-in method load}
1214 0.000 0.000 0.001 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
4917 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
4917 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
416 0.000 0.000 0.000 0.000 {method 'sort' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
1211 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}
1216 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
9/4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:209(walk)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
4 0.000 0.000 0.000 0.000 {posix.unlink}
4 0.000 0.000 0.000 0.000 {posix.listdir}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
21 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
9 0.000 0.000 0.000 0.000 {fcntl.flock}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
3 0.000 0.000 0.000 0.000 {posix.access}
8 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
2 0.000 0.000 0.000 0.000 {posix.lstat}
13 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
3 0.000 0.000 0.000 0.000 {posix.fstat}
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {posix.getpid}
1 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #4: profile_03.txt --]
[-- Type: text/plain, Size: 7219 bytes --]
605747 function calls (605742 primitive calls) in 0.498 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4917 0.008 0.000 0.487 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4917 0.069 0.000 0.479 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
23313 0.016 0.000 0.262 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23313 0.061 0.000 0.246 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
66338 0.085 0.000 0.085 0.000 {method 'rsplit' of 'str' objects}
14820 0.013 0.000 0.060 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
4917 0.014 0.000 0.052 0.000 {sorted}
23313 0.020 0.000 0.039 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
29841 0.038 0.000 0.038 0.000 {method 'find' of 'str' objects}
4917 0.011 0.000 0.035 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
15732 0.024 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
4923 0.024 0.000 0.024 0.000 {open}
23313 0.023 0.000 0.023 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
416 0.006 0.000 0.021 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
32340 0.018 0.000 0.018 0.000 {method 'join' of 'str' objects}
16542 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
15732 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
39072 0.007 0.000 0.007 0.000 {method 'startswith' of 'str' objects}
4917 0.006 0.000 0.006 0.000 {_hashlib.openssl_md5}
16623 0.006 0.000 0.006 0.000 {method 'split' of 'str' objects}
105820 0.005 0.000 0.005 0.000 {len}
1211 0.000 0.000 0.005 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
23313 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
46451 0.004 0.000 0.004 0.000 {method 'append' of 'list' objects}
1211 0.001 0.000 0.004 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
16563 0.004 0.000 0.004 0.000 {method 'endswith' of 'str' objects}
2184 0.004 0.000 0.004 0.000 {posix.stat}
1214 0.001 0.000 0.004 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
4917 0.003 0.000 0.003 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
1211 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
15760 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
1 0.000 0.000 0.001 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
1 0.000 0.000 0.001 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
2 0.001 0.000 0.001 0.000 {method 'dump' of 'cPickle.Pickler' objects}
4917 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
1214 0.000 0.000 0.001 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
4917 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
1211 0.000 0.000 0.000 0.000 {method 'get' of 'dict' objects}
416 0.000 0.000 0.000 0.000 {method 'sort' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
1216 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
9/4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:209(walk)
4 0.000 0.000 0.000 0.000 {posix.listdir}
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
4 0.000 0.000 0.000 0.000 {posix.unlink}
21 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
9 0.000 0.000 0.000 0.000 {fcntl.flock}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
1 0.000 0.000 0.000 0.000 {built-in method load}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
3 0.000 0.000 0.000 0.000 {posix.access}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
6 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
2 0.000 0.000 0.000 0.000 {posix.lstat}
8 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
3 0.000 0.000 0.000 0.000 {posix.fstat}
14 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {posix.getpid}
1 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #5: profile_04.txt --]
[-- Type: text/plain, Size: 12185 bytes --]
26278622 function calls (26266186 primitive calls) in 15.331 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4911 0.014 0.000 15.060 0.003 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4911 0.094 0.000 15.046 0.003 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
417 0.008 0.000 14.622 0.035 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
23 0.088 0.004 14.532 0.632 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
53519 0.032 0.000 13.886 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
53519 0.216 0.000 13.854 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
52308 5.352 0.000 10.975 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:512(md5_file)
20893617 5.259 0.000 5.259 0.000 {method 'update' of '_hashlib.HASH' objects}
52308 0.070 0.000 2.353 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/__init__.py:84(debug)
52308 0.052 0.000 2.244 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/__init__.py:57(bbdebug)
52308 0.094 0.000 2.192 0.000 /usr/lib64/python2.7/logging/__init__.py:1209(log)
52308 0.075 0.000 2.046 0.000 /usr/lib64/python2.7/logging/__init__.py:1260(_log)
52308 0.043 0.000 0.969 0.000 /usr/lib64/python2.7/logging/__init__.py:1281(handle)
52308 0.093 0.000 0.906 0.000 /usr/lib64/python2.7/logging/__init__.py:1313(callHandlers)
52308 0.150 0.000 0.859 0.000 /usr/lib64/python2.7/logging/__init__.py:1247(makeRecord)
104616 0.142 0.000 0.813 0.000 /usr/lib64/python2.7/logging/__init__.py:744(handle)
52308 0.411 0.000 0.709 0.000 /usr/lib64/python2.7/logging/__init__.py:243(__init__)
15955/3519 0.069 0.000 0.467 0.000 /usr/lib64/python2.7/os.py:209(walk)
110299 0.357 0.000 0.357 0.000 {posix.stat}
57225 0.319 0.000 0.319 0.000 {open}
53519 0.078 0.000 0.276 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
1 0.022 0.022 0.261 0.261 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
104616 0.054 0.000 0.255 0.000 /usr/lib64/python2.7/logging/__init__.py:701(acquire)
57021 0.038 0.000 0.225 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
104616 0.130 0.000 0.202 0.000 /usr/lib64/python2.7/threading.py:146(acquire)
104616 0.045 0.000 0.199 0.000 /usr/lib64/python2.7/logging/__init__.py:708(release)
23297 0.016 0.000 0.192 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
52308 0.025 0.000 0.181 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/event.py:602(emit)
111605 0.120 0.000 0.177 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
23297 0.056 0.000 0.176 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
1 0.004 0.004 0.160 0.160 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
52308 0.035 0.000 0.157 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/event.py:163(fire)
104616 0.118 0.000 0.154 0.000 /usr/lib64/python2.7/threading.py:186(release)
52308 0.093 0.000 0.144 0.000 /usr/lib64/python2.7/logging/__init__.py:1226(findCaller)
2 0.143 0.072 0.143 0.072 {method 'dump' of 'cPickle.Pickler' objects}
52308 0.026 0.000 0.099 0.000 /usr/lib64/python2.7/posixpath.py:104(splitext)
52308 0.057 0.000 0.093 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/event.py:139(fire_ui_handlers)
3498 0.087 0.000 0.087 0.000 {posix.listdir}
1 0.000 0.000 0.078 0.078 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
52308 0.055 0.000 0.073 0.000 /usr/lib64/python2.7/genericpath.py:85(_splitext)
1 0.066 0.066 0.066 0.066 {built-in method load}
14819 0.013 0.000 0.061 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
4911 0.014 0.000 0.052 0.000 {sorted}
57219 0.050 0.000 0.050 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
158135 0.049 0.000 0.049 0.000 {method 'get' of 'dict' objects}
52314 0.030 0.000 0.049 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
52308 0.030 0.000 0.047 0.000 /usr/lib64/python2.7/logging/__init__.py:1353(isEnabledFor)
417 0.045 0.000 0.045 0.000 {method 'sort' of 'list' objects}
209233 0.045 0.000 0.045 0.000 {isinstance}
52308 0.037 0.000 0.045 0.000 /usr/lib64/python2.7/logging/__init__.py:154(getLevelName)
104616 0.045 0.000 0.045 0.000 {method 'acquire' of 'thread.lock' objects}
150639 0.044 0.000 0.044 0.000 {method 'startswith' of 'str' objects}
4911 0.012 0.000 0.043 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
29839 0.038 0.000 0.038 0.000 {method 'find' of 'str' objects}
23297 0.020 0.000 0.038 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
57219 0.037 0.000 0.037 0.000 {_hashlib.openssl_md5}
52308 0.023 0.000 0.037 0.000 /usr/lib64/python2.7/logging/__init__.py:82(<lambda>)
156939 0.036 0.000 0.036 0.000 {method 'rfind' of 'str' objects}
52308 0.032 0.000 0.036 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/event.py:257(filter)
104616 0.036 0.000 0.036 0.000 /usr/lib64/python2.7/logging/__init__.py:605(filter)
209232 0.034 0.000 0.034 0.000 /usr/lib64/python2.7/threading.py:63(_note)
15731 0.025 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
52308 0.026 0.000 0.029 0.000 /usr/lib64/python2.7/threading.py:1152(currentThread)
52308 0.022 0.000 0.028 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/event.py:99(fire_class_handlers)
313848 0.028 0.000 0.028 0.000 {thread.get_ident}
84646 0.028 0.000 0.028 0.000 {method 'join' of 'str' objects}
57021 0.018 0.000 0.027 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
128146 0.024 0.000 0.024 0.000 {method 'endswith' of 'str' objects}
23300 0.024 0.000 0.024 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
1 0.023 0.023 0.023 0.023 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
3479 0.004 0.000 0.020 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
66305 0.018 0.000 0.018 0.000 {method 'rsplit' of 'str' objects}
52308 0.017 0.000 0.017 0.000 /usr/lib64/python2.7/logging/__init__.py:1339(getEffectiveLevel)
104616 0.016 0.000 0.016 0.000 {hasattr}
154535 0.016 0.000 0.016 0.000 {method 'append' of 'list' objects}
52308 0.014 0.000 0.014 0.000 /usr/lib64/python2.7/threading.py:967(name)
52308 0.014 0.000 0.014 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/event.py:613(filter)
52308 0.014 0.000 0.014 0.000 {sys._getframe}
52308 0.013 0.000 0.013 0.000 /usr/lib64/python2.7/multiprocessing/process.py:161(name)
3479 0.013 0.000 0.013 0.000 {posix.lstat}
104616 0.013 0.000 0.013 0.000 {method 'release' of 'thread.lock' objects}
60500 0.010 0.000 0.010 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
52308 0.010 0.000 0.010 0.000 /usr/lib64/python2.7/multiprocessing/process.py:59(current_process)
52308 0.010 0.000 0.010 0.000 {time.time}
16541 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
15731 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
52309 0.006 0.000 0.006 0.000 /usr/lib64/python2.7/posixpath.py:51(normcase)
52308 0.006 0.000 0.006 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/__init__.py:47(emit)
52309 0.005 0.000 0.005 0.000 {posix.getpid}
16626 0.005 0.000 0.005 0.000 {method 'split' of 'str' objects}
105673 0.005 0.000 0.005 0.000 {len}
23297 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
3479 0.002 0.000 0.003 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
15759 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
4 0.001 0.000 0.001 0.000 {posix.unlink}
4911 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
23 0.001 0.000 0.001 0.000 {method 'extend' of 'list' objects}
4911 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:18(glob)
31 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:29(iglob)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:66(glob1)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/fnmatch.py:45(filter)
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
9 0.000 0.000 0.000 0.000 {fcntl.flock}
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
1 0.000 0.000 0.000 0.000 {filter}
3 0.000 0.000 0.000 0.000 {posix.access}
30 0.000 0.000 0.000 0.000 {method 'match' of '_sre.SRE_Pattern' objects}
38 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:77(<lambda>)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:94(has_magic)
9 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
3 0.000 0.000 0.000 0.000 {posix.fstat}
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #6: profile_05.txt --]
[-- Type: text/plain, Size: 8042 bytes --]
1566521 function calls (1554085 primitive calls) in 1.286 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4911 0.013 0.000 1.218 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4911 0.089 0.000 1.204 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
417 0.007 0.000 0.801 0.002 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
23 0.047 0.002 0.736 0.032 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
15955/3519 0.056 0.000 0.325 0.000 /usr/lib64/python2.7/os.py:209(walk)
53519 0.018 0.000 0.238 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
53519 0.055 0.000 0.220 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
110299 0.192 0.000 0.192 0.000 {posix.stat}
111605 0.151 0.000 0.191 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
23297 0.015 0.000 0.188 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23297 0.054 0.000 0.173 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
57021 0.032 0.000 0.156 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
53519 0.048 0.000 0.139 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
14819 0.014 0.000 0.061 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
1 0.006 0.006 0.059 0.059 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
1 0.001 0.001 0.053 0.053 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
4911 0.013 0.000 0.051 0.000 {sorted}
2 0.051 0.025 0.051 0.025 {method 'dump' of 'cPickle.Pickler' objects}
417 0.043 0.000 0.043 0.000 {method 'sort' of 'list' objects}
3498 0.040 0.000 0.040 0.000 {posix.listdir}
29839 0.038 0.000 0.038 0.000 {method 'find' of 'str' objects}
23297 0.020 0.000 0.038 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
4911 0.011 0.000 0.035 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
15731 0.024 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
150639 0.030 0.000 0.030 0.000 {method 'startswith' of 'str' objects}
53519 0.026 0.000 0.026 0.000 {method 'get' of 'dict' objects}
4917 0.025 0.000 0.025 0.000 {open}
57021 0.015 0.000 0.024 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
23300 0.022 0.000 0.022 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
128146 0.020 0.000 0.020 0.000 {method 'endswith' of 'str' objects}
66305 0.018 0.000 0.018 0.000 {method 'rsplit' of 'str' objects}
32338 0.018 0.000 0.018 0.000 {method 'join' of 'str' objects}
154535 0.013 0.000 0.013 0.000 {method 'append' of 'list' objects}
3479 0.003 0.000 0.011 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
60500 0.009 0.000 0.009 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
16541 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
4911 0.009 0.000 0.009 0.000 {_hashlib.openssl_md5}
15731 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
3479 0.006 0.000 0.006 0.000 {posix.lstat}
16626 0.005 0.000 0.005 0.000 {method 'split' of 'str' objects}
105673 0.005 0.000 0.005 0.000 {len}
23297 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
4911 0.003 0.000 0.003 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
15759 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
3479 0.001 0.000 0.002 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
4911 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
4911 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
23 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:18(glob)
31 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:29(iglob)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:66(glob1)
4 0.000 0.000 0.000 0.000 {posix.unlink}
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/fnmatch.py:45(filter)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
9 0.000 0.000 0.000 0.000 {fcntl.flock}
1 0.000 0.000 0.000 0.000 {filter}
4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
30 0.000 0.000 0.000 0.000 {method 'match' of '_sre.SRE_Pattern' objects}
1 0.000 0.000 0.000 0.000 {built-in method load}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
3 0.000 0.000 0.000 0.000 {posix.access}
38 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:77(<lambda>)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:94(has_magic)
6 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
15 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
3 0.000 0.000 0.000 0.000 {posix.fstat}
9 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {isinstance}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:51(normcase)
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {posix.getpid}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #7: profile_06.txt --]
[-- Type: text/plain, Size: 5141 bytes --]
827014 function calls (827009 primitive calls) in 2.235 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4917 0.012 0.000 2.222 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4917 0.084 0.000 2.210 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:177(get_taskhash)
416 0.018 0.000 1.749 0.004 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/fetch2/__init__.py:1104(get_file_checksums)
1211 0.002 0.000 1.690 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/fetch2/__init__.py:1112(checksum_file)
1211 0.008 0.000 1.688 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:69(get_checksum)
1211 1.534 0.001 1.672 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:512(md5_file)
23313 0.018 0.000 0.196 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23313 0.057 0.000 0.178 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
216557 0.121 0.000 0.121 0.000 {method 'update' of '_hashlib.HASH' objects}
4917 0.016 0.000 0.075 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:168(read_taint)
6128 0.071 0.000 0.071 0.000 {open}
14820 0.014 0.000 0.062 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
4917 0.017 0.000 0.060 0.000 {sorted}
23313 0.023 0.000 0.043 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:316(clean_basepath)
29841 0.039 0.000 0.039 0.000 {method 'find' of 'str' objects}
15732 0.025 0.000 0.034 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
23313 0.026 0.000 0.026 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
66338 0.021 0.000 0.021 0.000 {method 'rsplit' of 'str' objects}
32340 0.018 0.000 0.018 0.000 {method 'join' of 'str' objects}
1214 0.003 0.000 0.016 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
2174 0.015 0.000 0.015 0.000 {posix.stat}
1 0.000 0.000 0.014 0.014 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/fetch2/__init__.py:1120(checksum_dir)
16623 0.013 0.000 0.013 0.000 {method 'split' of 'str' objects}
9/4 0.000 0.000 0.011 0.003 /usr/lib64/python2.7/os.py:209(walk)
3 0.010 0.003 0.010 0.003 {posix.listdir}
6128 0.010 0.000 0.010 0.000 {_hashlib.openssl_md5}
16542 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
39065 0.007 0.000 0.007 0.000 {method 'startswith' of 'str' objects}
1211 0.003 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:39(cached_mtime)
6128 0.007 0.000 0.007 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
105812 0.007 0.000 0.007 0.000 {len}
15732 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
23313 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
46451 0.005 0.000 0.005 0.000 {method 'append' of 'list' objects}
16562 0.004 0.000 0.004 0.000 {method 'endswith' of 'str' objects}
15760 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
1214 0.001 0.000 0.002 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
416 0.001 0.000 0.001 0.000 {method 'sort' of 'list' objects}
4917 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
4917 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
1211 0.001 0.000 0.001 0.000 {method 'get' of 'dict' objects}
1216 0.001 0.000 0.001 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
20 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
2 0.000 0.000 0.000 0.000 {posix.lstat}
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
1 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #8: profile_07.txt --]
[-- Type: text/plain, Size: 7432 bytes --]
827142 function calls (827137 primitive calls) in 2.194 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4917 0.012 0.000 2.178 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4917 0.084 0.000 2.166 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
416 0.019 0.000 1.644 0.004 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
1211 0.002 0.000 1.585 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
1211 0.008 0.000 1.583 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
1211 1.423 0.001 1.568 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:512(md5_file)
23313 0.018 0.000 0.262 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23313 0.124 0.000 0.245 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
216557 0.128 0.000 0.128 0.000 {method 'update' of '_hashlib.HASH' objects}
4917 0.016 0.000 0.072 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
6134 0.068 0.000 0.068 0.000 {open}
14820 0.013 0.000 0.062 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
4917 0.017 0.000 0.058 0.000 {sorted}
23313 0.022 0.000 0.042 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
29841 0.039 0.000 0.039 0.000 {method 'find' of 'str' objects}
15732 0.025 0.000 0.034 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
23313 0.025 0.000 0.025 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
66338 0.020 0.000 0.020 0.000 {method 'rsplit' of 'str' objects}
32340 0.019 0.000 0.019 0.000 {method 'join' of 'str' objects}
1214 0.003 0.000 0.015 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
2184 0.014 0.000 0.014 0.000 {posix.stat}
1 0.000 0.000 0.014 0.014 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
16623 0.013 0.000 0.013 0.000 {method 'split' of 'str' objects}
9/4 0.000 0.000 0.011 0.003 /usr/lib64/python2.7/os.py:209(walk)
4 0.010 0.003 0.010 0.003 {posix.listdir}
6128 0.009 0.000 0.009 0.000 {_hashlib.openssl_md5}
16542 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
6128 0.007 0.000 0.007 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
39070 0.007 0.000 0.007 0.000 {method 'startswith' of 'str' objects}
105820 0.007 0.000 0.007 0.000 {len}
1211 0.003 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
15732 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
46451 0.005 0.000 0.005 0.000 {method 'append' of 'list' objects}
23313 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
16563 0.004 0.000 0.004 0.000 {method 'endswith' of 'str' objects}
1 0.000 0.000 0.004 0.004 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
15760 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
1 0.000 0.000 0.002 0.002 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
2 0.002 0.001 0.002 0.001 {method 'dump' of 'cPickle.Pickler' objects}
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
1214 0.001 0.000 0.002 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
1 0.000 0.000 0.001 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
416 0.001 0.000 0.001 0.000 {method 'sort' of 'list' objects}
4917 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
1 0.001 0.001 0.001 0.001 {built-in method load}
4917 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
1216 0.001 0.000 0.001 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
1211 0.001 0.000 0.001 0.000 {method 'get' of 'dict' objects}
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
21 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
4 0.000 0.000 0.000 0.000 {posix.unlink}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
2 0.000 0.000 0.000 0.000 {posix.lstat}
9 0.000 0.000 0.000 0.000 {fcntl.flock}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
3 0.000 0.000 0.000 0.000 {posix.access}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
3 0.000 0.000 0.000 0.000 {posix.fstat}
12 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
8 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {posix.getpid}
1 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #9: profile_08.txt --]
[-- Type: text/plain, Size: 7219 bytes --]
605744 function calls (605739 primitive calls) in 0.617 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4917 0.011 0.000 0.576 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4917 0.082 0.000 0.565 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
23313 0.017 0.000 0.268 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23313 0.063 0.000 0.250 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
66338 0.086 0.000 0.086 0.000 {method 'rsplit' of 'str' objects}
4917 0.015 0.000 0.066 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
14820 0.013 0.000 0.061 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
4923 0.061 0.000 0.061 0.000 {open}
4917 0.016 0.000 0.059 0.000 {sorted}
416 0.013 0.000 0.044 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
23313 0.023 0.000 0.043 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
29841 0.038 0.000 0.038 0.000 {method 'find' of 'str' objects}
15732 0.025 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
1 0.000 0.000 0.029 0.029 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
1 0.018 0.018 0.029 0.029 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
23313 0.025 0.000 0.025 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
32340 0.018 0.000 0.018 0.000 {method 'join' of 'str' objects}
16623 0.011 0.000 0.011 0.000 {method 'split' of 'str' objects}
1211 0.001 0.000 0.010 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
16542 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
1211 0.003 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
4917 0.008 0.000 0.008 0.000 {_hashlib.openssl_md5}
2184 0.008 0.000 0.008 0.000 {posix.stat}
1214 0.002 0.000 0.008 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
39071 0.007 0.000 0.007 0.000 {method 'startswith' of 'str' objects}
15732 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
105820 0.007 0.000 0.007 0.000 {len}
23313 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
46451 0.005 0.000 0.005 0.000 {method 'append' of 'list' objects}
1211 0.002 0.000 0.005 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
16563 0.004 0.000 0.004 0.000 {method 'endswith' of 'str' objects}
4917 0.004 0.000 0.004 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
15760 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
1214 0.001 0.000 0.001 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
1 0.000 0.000 0.001 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
9/4 0.000 0.000 0.001 0.000 /usr/lib64/python2.7/os.py:209(walk)
1211 0.001 0.000 0.001 0.000 {method 'get' of 'dict' objects}
4917 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
2 0.001 0.000 0.001 0.000 {method 'dump' of 'cPickle.Pickler' objects}
416 0.001 0.000 0.001 0.000 {method 'sort' of 'list' objects}
4917 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
4 0.001 0.000 0.001 0.000 {posix.listdir}
1216 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
21 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
4 0.000 0.000 0.000 0.000 {posix.unlink}
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
2 0.000 0.000 0.000 0.000 {posix.lstat}
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
9 0.000 0.000 0.000 0.000 {fcntl.flock}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
1 0.000 0.000 0.000 0.000 {built-in method load}
3 0.000 0.000 0.000 0.000 {posix.access}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
3 0.000 0.000 0.000 0.000 {posix.fstat}
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
2 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
8 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
13 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {posix.getpid}
1 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
[-- Attachment #10: profile_09.txt --]
[-- Type: text/plain, Size: 8258 bytes --]
22669370 function calls (22656934 primitive calls) in 80.332 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4911 0.016 0.000 80.039 0.016 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4911 0.107 0.000 80.023 0.016 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
417 0.015 0.000 79.538 0.191 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
23 0.229 0.010 79.169 3.442 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
53519 0.075 0.000 70.268 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
53519 0.338 0.000 70.193 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
52308 57.299 0.001 69.200 0.001 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:512(md5_file)
20893617 11.202 0.000 11.202 0.000 {method 'update' of '_hashlib.HASH' objects}
15955/3519 0.160 0.000 8.463 0.002 /usr/lib64/python2.7/os.py:209(walk)
3498 5.004 0.001 5.004 0.001 {posix.listdir}
110299 3.558 0.000 3.558 0.000 {posix.stat}
57021 0.088 0.000 3.268 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
53519 0.175 0.000 0.613 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
57225 0.577 0.000 0.577 0.000 {open}
111605 0.297 0.000 0.457 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
1 0.028 0.028 0.282 0.282 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
23297 0.017 0.000 0.201 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23297 0.059 0.000 0.184 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
1 0.004 0.004 0.176 0.176 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
2 0.164 0.082 0.164 0.082 {method 'dump' of 'cPickle.Pickler' objects}
57219 0.131 0.000 0.131 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
150639 0.117 0.000 0.117 0.000 {method 'startswith' of 'str' objects}
1 0.000 0.000 0.113 0.113 /usr/lib64/python2.7/glob.py:18(glob)
31 0.000 0.000 0.113 0.004 /usr/lib64/python2.7/glob.py:29(iglob)
1 0.000 0.000 0.113 0.113 /usr/lib64/python2.7/glob.py:66(glob1)
1 0.000 0.000 0.078 0.078 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
4911 0.014 0.000 0.073 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
1 0.069 0.069 0.069 0.069 {built-in method load}
57219 0.064 0.000 0.064 0.000 {_hashlib.openssl_md5}
14819 0.014 0.000 0.063 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
417 0.061 0.000 0.061 0.000 {method 'sort' of 'list' objects}
57021 0.038 0.000 0.059 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
4911 0.015 0.000 0.057 0.000 {sorted}
128146 0.056 0.000 0.056 0.000 {method 'endswith' of 'str' objects}
53519 0.043 0.000 0.043 0.000 {method 'get' of 'dict' objects}
3479 0.009 0.000 0.042 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
23297 0.022 0.000 0.042 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
29839 0.040 0.000 0.040 0.000 {method 'find' of 'str' objects}
15731 0.025 0.000 0.034 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
154535 0.028 0.000 0.028 0.000 {method 'append' of 'list' objects}
3479 0.027 0.000 0.027 0.000 {posix.lstat}
23300 0.025 0.000 0.025 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
60500 0.023 0.000 0.023 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
66305 0.019 0.000 0.019 0.000 {method 'rsplit' of 'str' objects}
32338 0.019 0.000 0.019 0.000 {method 'join' of 'str' objects}
1 0.016 0.016 0.016 0.016 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
16541 0.005 0.000 0.010 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
16626 0.010 0.000 0.010 0.000 {method 'split' of 'str' objects}
15731 0.005 0.000 0.008 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
105673 0.006 0.000 0.006 0.000 {len}
3479 0.004 0.000 0.006 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
23297 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
15759 0.003 0.000 0.003 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
4 0.001 0.000 0.001 0.000 {posix.unlink}
4911 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
23 0.001 0.000 0.001 0.000 {method 'extend' of 'list' objects}
4911 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/fnmatch.py:45(filter)
1 0.000 0.000 0.000 0.000 {filter}
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
38 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:77(<lambda>)
9 0.000 0.000 0.000 0.000 {fcntl.flock}
30 0.000 0.000 0.000 0.000 {method 'match' of '_sre.SRE_Pattern' objects}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
3 0.000 0.000 0.000 0.000 {posix.access}
4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
15 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:94(has_magic)
3 0.000 0.000 0.000 0.000 {posix.fstat}
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
9 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
6 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {isinstance}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:51(normcase)
1 0.000 0.000 0.000 0.000 {posix.getpid}
[-- Attachment #11: profile_10.txt --]
[-- Type: text/plain, Size: 8042 bytes --]
1566521 function calls (1554085 primitive calls) in 8.581 seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
4911 0.015 0.000 8.496 0.002 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:132(get_taskhash)
4911 0.101 0.000 8.481 0.002 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:188(get_taskhash)
417 0.012 0.000 8.031 0.019 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:98(get_checksums)
23 0.096 0.004 7.838 0.341 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:109(checksum_dir)
15955/3519 0.224 0.000 7.168 0.002 /usr/lib64/python2.7/os.py:209(walk)
3498 3.700 0.001 3.700 0.001 {posix.listdir}
110299 3.213 0.000 3.213 0.000 {posix.stat}
57021 0.077 0.000 3.165 0.000 /usr/lib64/python2.7/genericpath.py:38(isdir)
53519 0.036 0.000 0.455 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:101(checksum_file)
53519 0.105 0.000 0.420 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:71(get_checksum)
53519 0.093 0.000 0.270 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:41(cached_mtime)
111605 0.182 0.000 0.267 0.000 /usr/lib64/python2.7/posixpath.py:68(join)
23297 0.016 0.000 0.192 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:117(rundep_check)
23297 0.055 0.000 0.175 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:3(sstate_rundepfilter)
1 0.006 0.006 0.073 0.073 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:238(writeout_file_checksum_cache)
1 0.001 0.001 0.067 0.067 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:819(save_merge)
14819 0.013 0.000 0.061 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:11(isKernel)
150639 0.061 0.000 0.061 0.000 {method 'startswith' of 'str' objects}
417 0.060 0.000 0.060 0.000 {method 'sort' of 'list' objects}
4917 0.060 0.000 0.060 0.000 {open}
4911 0.014 0.000 0.058 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:179(read_taint)
4911 0.015 0.000 0.055 0.000 {sorted}
57021 0.034 0.000 0.053 0.000 /usr/lib64/python2.7/stat.py:40(S_ISDIR)
2 0.051 0.025 0.051 0.025 {method 'dump' of 'cPickle.Pickler' objects}
53519 0.044 0.000 0.044 0.000 {method 'get' of 'dict' objects}
23297 0.021 0.000 0.040 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/siggen.py:342(clean_basepath)
29839 0.039 0.000 0.039 0.000 {method 'find' of 'str' objects}
128146 0.037 0.000 0.037 0.000 {method 'endswith' of 'str' objects}
15731 0.025 0.000 0.033 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:14(isPackageGroup)
1 0.000 0.000 0.024 0.024 /usr/lib64/python2.7/glob.py:18(glob)
31 0.000 0.000 0.024 0.001 /usr/lib64/python2.7/glob.py:29(iglob)
1 0.000 0.000 0.024 0.024 /usr/lib64/python2.7/glob.py:66(glob1)
23300 0.024 0.000 0.024 0.000 {method 'search' of '_sre.SRE_Pattern' objects}
154535 0.023 0.000 0.023 0.000 {method 'append' of 'list' objects}
3479 0.005 0.000 0.021 0.000 /usr/lib64/python2.7/posixpath.py:139(islink)
60500 0.020 0.000 0.020 0.000 /usr/lib64/python2.7/stat.py:24(S_IFMT)
66305 0.019 0.000 0.019 0.000 {method 'rsplit' of 'str' objects}
32338 0.018 0.000 0.018 0.000 {method 'join' of 'str' objects}
3479 0.012 0.000 0.012 0.000 {posix.lstat}
16626 0.011 0.000 0.011 0.000 {method 'split' of 'str' objects}
4911 0.010 0.000 0.010 0.000 {_hashlib.openssl_md5}
16541 0.005 0.000 0.009 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:5(isNative)
15731 0.004 0.000 0.007 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:9(isNativeSDK)
105673 0.006 0.000 0.006 0.000 {len}
23297 0.005 0.000 0.005 0.000 {method 'group' of '_sre.SRE_Match' objects}
3479 0.002 0.000 0.004 0.000 /usr/lib64/python2.7/stat.py:55(S_ISLNK)
4911 0.003 0.000 0.003 0.000 {method 'hexdigest' of '_hashlib.HASH' objects}
15759 0.002 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:7(isCross)
55 0.002 0.000 0.002 0.000 {method 'copy' of 'set' objects}
651 0.001 0.000 0.002 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:20(isImage)
936 0.001 0.000 0.001 0.000 /home/marquiz/yocto/openembedded-core/meta/lib/oe/sstatesig.py:17(isAllArch)
4911 0.001 0.000 0.001 0.000 {method 'add' of 'set' objects}
4911 0.001 0.000 0.001 0.000 {method 'remove' of 'set' objects}
23 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/cache.py:789(save_extras)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:443(lockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:498(unlockfile)
3 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/utils.py:699(mkdirhier)
4 0.000 0.000 0.000 0.000 {posix.unlink}
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/os.py:136(makedirs)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/fnmatch.py:45(filter)
7 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/genericpath.py:15(exists)
3 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
1 0.000 0.000 0.000 0.000 {filter}
9 0.000 0.000 0.000 0.000 {fcntl.flock}
5 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:127(dirname)
4 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:89(split)
1 0.000 0.000 0.000 0.000 {built-in method load}
3 0.000 0.000 0.000 0.000 {posix.mkdir}
2 0.000 0.000 0.000 0.000 {cPickle.Pickler}
30 0.000 0.000 0.000 0.000 {method 'match' of '_sre.SRE_Pattern' objects}
38 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:77(<lambda>)
3 0.000 0.000 0.000 0.000 {posix.access}
6 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:119(basename)
3 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/glob.py:94(has_magic)
15 0.000 0.000 0.000 0.000 {method 'rfind' of 'str' objects}
9 0.000 0.000 0.000 0.000 {method 'rstrip' of 'str' objects}
3 0.000 0.000 0.000 0.000 {posix.fstat}
9 0.000 0.000 0.000 0.000 {method 'fileno' of 'file' objects}
1 0.000 0.000 0.000 0.000 {cPickle.Unpickler}
1 0.000 0.000 0.000 0.000 {isinstance}
1 0.000 0.000 0.000 0.000 /home/marquiz/yocto/openembedded-core/bitbake/lib/bb/checksum.py:88(merge_data)
1 0.000 0.000 0.000 0.000 /usr/lib64/python2.7/posixpath.py:51(normcase)
1 0.000 0.000 0.000 0.000 {posix.getpid}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
prev parent reply other threads:[~2016-02-02 16:23 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-26 13:34 [PATCH 0/6] Improve caching task file dependency checksum caching Markus Lehtonen
2016-01-26 13:34 ` [PATCH 1/6] bb/cache: drop some unused arguments Markus Lehtonen
2016-01-26 13:34 ` [PATCH 2/6] SignatureGenerator: add method for saving the file checksum cache Markus Lehtonen
2016-01-26 13:34 ` [PATCH 3/6] bb/runqueue: save task file dependency cache onto disk Markus Lehtonen
2016-01-26 13:34 ` [PATCH 4/6] FileChecksumCache: add get_checksums() method Markus Lehtonen
2016-01-26 13:34 ` [PATCH 5/6] MultiProcessCache: make cache filename configurable Markus Lehtonen
2016-01-26 13:34 ` [PATCH 6/6] SignatureGeneratorBasic: make checksum cache file configurable Markus Lehtonen
2016-01-29 17:09 ` [PATCH 0/6] Improve caching task file dependency checksum caching Richard Purdie
2016-02-02 16:23 ` Markus Lehtonen [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1454430221.13214.72.camel@linux.intel.com \
--to=markus.lehtonen@linux.intel.com \
--cc=bitbake-devel@lists.openembedded.org \
--cc=richard.purdie@linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.