From: Gary Thomas <gary@mlbassoc.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Poky <poky@lists.pokylinux.org>
Subject: Re: Is sstate broken
Date: Wed, 09 Mar 2011 13:26:37 -0700 [thread overview]
Message-ID: <4D77E27D.7020901@mlbassoc.com> (raw)
In-Reply-To: <1299697497.602.866.camel@rex>
On 03/09/11 12:04, Richard Purdie wrote:
> On Wed, 2011-03-09 at 11:42 -0700, Gary Thomas wrote:
>> I updated to master (fab742bd4693ed3092690a20dd32d53fe27c3d4c) and tried again.
>> I don't see any difference - the run using the sstate cache as a mirror
>> seems to do all the same work as without. Here's how I tested it.
>>
>> * Build original tree
>> % POKYCONF=meta-cobra3530p60/conf . /tmp/poky-amltd/poky-init-build-env /local/p60_step1
>> ... adjust conf/local.conf
>> % bitbake amltd-console-image
>>
>> * Rebuild, using previous result for SSTATE_MIRRORS
>> % POKYCONF=meta-cobra3530p60/conf . /tmp/poky-amltd/poky-init-build-env /local/p60_step2
>> ... adjust conf/local.conf
>> % bitbake amltd-console-image
>>
>> The only difference between the two runs is enabling SSTATE_MIRRORS in local.conf
>> % diff -u /local/p60_step?/conf/local.conf
>> --- /local/p60_step1/conf/local.conf 2011-03-09 08:28:18.266933061 -0700
>> +++ /local/p60_step2/conf/local.conf 2011-03-09 09:57:51.365932951 -0700
>> @@ -53,4 +53,7 @@
>> IMAGE_LINGUAS ?= "en-us"
>>
>> # Minimize feature set
>> DISTRO_FEATURES ?= "alsa"
>> +SSTATE_MIRRORS ?= "\
>> +file://.* file:///local/p60_step1/sstate-cache/"
>>
>> The results seem to have gone through all the same steps (or nearly so). The output
>> from the runs is at
>> http://www.mlbassoc.com/poky/build.step1
>> http://www.mlbassoc.com/poky/build.step2
>>
>> Comparing the two build trees:
>> % ls /local/p60_step1/tmp/work/*/*/temp/log.do_compile | wc
>> 144 144 12521
>> % ls /local/p60_step2/tmp/work/*/*/temp/log.do_compile | wc
>> 143 143 12427
>> % du -s /local/p60_step?
>> 15229296 /local/p60_step1
>> 15162760 /local/p60_step2
>>
>> I know this procedure used to work (or at least close). Am I doing
>> something wrong?
>
> You're not doing anything wrong and this is the same scenario I've been
> testing with. After I'd fixed the origin problem it created a problem
> with file urls containing globing. I managed to break the original patch
> with the globing fix. The good news is the problem is simple and I've
> pushed a fix.
>
> It should work *much* better than you've seen above, not needing to
> rerun many tasks so if its working it will be very obvious.
Yes, indeed it does! I just reran step2 from scratch and it finished
in only 13 minutes (with a lot of other things on the build machine so
this is quite good). There's still quite a lot of stuff in the build
tree:
% du -s /local/p60_step2
7285332 /local/p60_step2
but that's only about 1/2 of before. The results seem very close:
% ls -l /local/p60_step2/tmp/deploy/images/
total 10748
-rwxrw-r-- 1 gthomas gthomas 5446150 Mar 9 13:15 amltd-console-image-cobra3530p60-20110309200339.rootfs.tar.bz2
lrwxrwxrwx 1 gthomas gthomas 62 Mar 9 13:15 amltd-console-image-cobra3530p60.tar.bz2 -> amltd-console-image-cobra3530p60-20110309200339.rootfs.tar.bz2
drwxrwxr-x 91 gthomas gthomas 4096 Mar 9 13:08 licenses
-rw-rw-r-- 1 gthomas gthomas 3077911 Mar 9 09:24 modules-2.6.37-r0-cobra3530p60.tgz
-rw-r--r-- 1 gthomas gthomas 2468564 Mar 9 09:24 uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
lrwxrwxrwx 1 gthomas gthomas 48 Mar 9 09:24 uImage-cobra3530p60.bin -> uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
% ls -l /local/p60_step1/tmp/deploy/images/
total 10748
-rwxrw-r-- 1 gthomas gthomas 5446783 Mar 9 09:53 amltd-console-image-cobra3530p60-20110309153435.rootfs.tar.bz2
lrwxrwxrwx 1 gthomas gthomas 62 Mar 9 09:53 amltd-console-image-cobra3530p60.tar.bz2 -> amltd-console-image-cobra3530p60-20110309153435.rootfs.tar.bz2
drwxrwxr-x 92 gthomas gthomas 4096 Mar 9 09:06 licenses
-rw-rw-r-- 1 gthomas gthomas 3077911 Mar 9 09:24 modules-2.6.37-r0-cobra3530p60.tgz
-rw-r--r-- 1 gthomas gthomas 2468564 Mar 9 09:24 uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
lrwxrwxrwx 1 gthomas gthomas 48 Mar 9 09:24 uImage-cobra3530p60.bin -> uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
I unpacked the resulting image tarballs and they match, modulo the
package install times in the opkg database.
Looks like it's working correctly now :-)
Can you explain what the sstate files are and how I might make use of them?
In particular, I'd like to be able to hand my customers a sstate cache which
holds the majority of the tools they might need to speed startup. Sending
the whole thing is a bit problematic as it's quite large, so if I can winnow
it a bit it would help.
Thanks for getting this back in order
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
prev parent reply other threads:[~2011-03-09 20:26 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-24 15:14 Is sstate broken Gary Thomas
2011-02-28 6:07 ` Zhai, Edwin
2011-03-03 12:56 ` Gary Thomas
2011-03-04 1:59 ` Zhai, Edwin
2011-03-04 11:34 ` Gary Thomas
2011-03-09 1:29 ` Richard Purdie
2011-03-09 18:42 ` Gary Thomas
2011-03-09 19:04 ` Richard Purdie
2011-03-09 19:30 ` Koen Kooi
2011-03-09 21:51 ` Khem Raj
2011-03-09 23:04 ` Richard Purdie
2011-03-09 20:26 ` Gary Thomas [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D77E27D.7020901@mlbassoc.com \
--to=gary@mlbassoc.com \
--cc=poky@lists.pokylinux.org \
--cc=richard.purdie@linuxfoundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox