From: Gary Thomas <gary@mlbassoc.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Paul Eggleton <paul.eggleton@linux.intel.com>,
"poky@yoctoproject.org" <poky@yoctoproject.org>
Subject: Re: sstate status
Date: Sun, 05 Dec 2010 05:02:55 -0700 [thread overview]
Message-ID: <4CFB7F6F.60407@mlbassoc.com> (raw)
In-Reply-To: <625BA99ED14B2D499DC4E29D8138F1504D475430F6@shsmsx502.ccr.corp.intel.com>
On 12/05/2010 04:19 AM, Tian, Kevin wrote:
>> From: Gary Thomas
>> Sent: Saturday, December 04, 2010 8:48 PM
>>
>> On 12/03/2010 04:31 PM, Gary Thomas wrote:
>>> On 12/03/2010 09:34 AM, Paul Eggleton wrote:
>>>> Hi all,
>>>>
>>>> I've been doing some tests with sstate, and I've managed to get to a
>>>> stage where I can share the SSTATE_DIR between my two machines (one
>>>> running Fedora 14 and the other Kubuntu 10.10) and almost get a
>>>> successful build from a clean TMPDIR. My last test got to
>>>> poky-image-minimal.do_rootfs and then failed due to some kind of
>>>> package dependency issue:
>>>>
>>>> --------- snip ---------
>>>> | Processing rpm...
>>>> | Processing zypper...
>>>> | error: Failed dependencies:
>>>> | elfutils>= 0.148 is needed by rpm-5.1.10-r7.i586
>>>> | libaugeas0>= 0.7.3 is needed by
>>>> zypper-1.4.7+git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r1.i586
>>>> | ERROR: Task failed: ('function do_rootfs failed',
>>>>
>> '/home/pokystuff/tmp/work/qemux86-poky-linux/poky-image-minimal-1.0-r0/temp/log.
>> do_rootfs.6874')
>>>>
>>>> --------- snip ---------
>>>>
>>>> A "bitbake -c clean augeas elfutils" and then "bitbake
>>>> poky-image-minimal" again went through OK (and built the
>>>> aforementioned recipes from sstate) , so I'm not sure what went wrong
>>>> here.
>>>>
>>>> My current state of work is in the paule/sstate contrib branch, which
>>>> is based on Kevin's tk/sstate branch. Kevin, I think the sooner we can
>>>> move this towards master the better, then we can get some wider testing.
>>>>
>>>> FYI Joshua has made bug #507 into a tracking bug for current sstate
>>>> issues:
>>>> http://bugzilla.pokylinux.org/show_bug.cgi?id=507
>>>
>>> I gave this a simple try and it failed miserably. Working from a copy of
>>> Paul's branch in poky-contrib, here's what I did:
>>>
>>> * On machine A (x86 Fedora 11)
>>> % bitbake poky-image-minimal
>>>
>>> * On machine B (x86 Fedora 13)
>>> -- Copy sstate-cache from machine A to /tmp/sstate-cache (rsync with
>>> all perms& timestamps preserved)
>>> -- Using the identical conf/local.conf file from machine A, except
>>> I set up SSTATE_MIRRORS like this
>>> SSTATE_MIRRORS ?= "\
>>> file://.* file:///tmp/sstate-cache/"
>>> % bitbake poky-image-minimal
>>>
>>> Fell apart pretty badly. Look at the output
>>> http://www.mlbassoc.com/poky/poky_with_sstate.out
>>>
>>> Also, it seemed to be rebuilding a lot of packages. IMO, there's
>>> no reason it should have to rebuild _any_
>>>
>>
>> One thing that seems to be significant is the base build directory path.
>> In the case above, they did not match, i.e.
>> * On machine A, BUILDDIR=/tmp/sstate_testing
>> * On machine B, BUILDDIR=/home/local/my_sstate_test
>> I tried it this way as it matches my [customer's] expected usage.
>
> I think in usual case the directory paths would mismatch. So let's stick to that
> configuration to expose more issues. In my previous debug, I have made it
> in a good shape but would now turn to RP's branch to try more experiments.
>
> It would be good if you also try that one:
> http://git.pokylinux.org/cgit/cgit.cgi/poky-contrib/log/?h=rpurdie/tweaks2
>
>>
>> I've rerun the experiment again where I used identical paths on both
>> machines and the results were vastly improved. It still takes a horrible
>> length of time (64 minutes on a fast 4 processor box) and rebuilds way to
>> many packages for just replaying what's already been done though (IMO).
>> Results are at
>> http://www.mlbassoc.com/poky/poky_with_sstate_same_paths.out
>>
>
> In my run ~25min are consumed just for sstate check/installation on a NHM 4-core box.
> then several minutes for rootfs generation if two builds totally match. the slow sstate
> phase comes from same reason as a slow build - the 'exec' overhead. I use 'sato' as
> the target.
>
> In your case, however there're still some packages rebuilt, such as gzip-native,
> unifdef-native, eglibc-initial, ncurses, etc. and also gcc related packages are rebuilt
> which just consumes time. Has your source tree changed between the two builds?
Do, identical source trees for all three runs.
I'll try it again later today with Richard's tree.
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
next prev parent reply other threads:[~2010-12-05 12:02 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-03 16:34 sstate status Paul Eggleton
2010-12-03 22:11 ` Richard Purdie
2010-12-05 11:10 ` Tian, Kevin
2010-12-06 1:29 ` Richard Purdie
2010-12-07 7:46 ` Tian, Kevin
2010-12-07 15:02 ` Richard Purdie
2010-12-08 0:40 ` Tian, Kevin
2010-12-06 16:06 ` Paul Eggleton
2010-12-07 12:46 ` Richard Purdie
2010-12-07 16:36 ` Paul Eggleton
2010-12-08 11:22 ` Tian, Kevin
2010-12-08 11:31 ` Paul Eggleton
2010-12-08 11:41 ` Tian, Kevin
2010-12-03 23:31 ` Gary Thomas
2010-12-04 12:47 ` Gary Thomas
2010-12-05 11:19 ` Tian, Kevin
2010-12-05 12:02 ` Gary Thomas [this message]
2010-12-06 11:19 ` Gary Thomas
[not found] ` <4CFCC607.9040803@mlbassoc.com>
2010-12-07 7:17 ` Tian, Kevin
2010-12-05 11:07 ` Tian, Kevin
-- strict thread matches above, loose matches on Subject: below --
2010-12-08 14:02 Paul Eggleton
2010-12-08 14:06 ` Paul Eggleton
2010-12-08 14:15 ` Tian, Kevin
2010-12-08 14:50 ` Paul Eggleton
2010-12-08 14:52 ` Tian, Kevin
2010-12-08 14:41 ` Gary Thomas
2010-12-08 15:31 ` Gary Thomas
2010-12-08 15:55 ` Paul Eggleton
2010-12-08 17:14 ` Paul Eggleton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4CFB7F6F.60407@mlbassoc.com \
--to=gary@mlbassoc.com \
--cc=kevin.tian@intel.com \
--cc=paul.eggleton@linux.intel.com \
--cc=poky@yoctoproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox