Poky Project Discussions
 help / color / mirror / Atom feed
From: Gary Thomas <gary@mlbassoc.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Poky <poky@lists.pokylinux.org>
Subject: Re: Is sstate broken
Date: Wed, 09 Mar 2011 13:26:37 -0700	[thread overview]
Message-ID: <4D77E27D.7020901@mlbassoc.com> (raw)
In-Reply-To: <1299697497.602.866.camel@rex>

On 03/09/11 12:04, Richard Purdie wrote:
> On Wed, 2011-03-09 at 11:42 -0700, Gary Thomas wrote:
>> I updated to master (fab742bd4693ed3092690a20dd32d53fe27c3d4c) and tried again.
>> I don't see any difference - the run using the sstate cache as a mirror
>> seems to do all the same work as without.  Here's how I tested it.
>>
>> * Build original tree
>>     % POKYCONF=meta-cobra3530p60/conf . /tmp/poky-amltd/poky-init-build-env /local/p60_step1
>>     ... adjust conf/local.conf
>>     % bitbake amltd-console-image
>>
>> * Rebuild, using previous result for SSTATE_MIRRORS
>>     % POKYCONF=meta-cobra3530p60/conf . /tmp/poky-amltd/poky-init-build-env /local/p60_step2
>>     ... adjust conf/local.conf
>>     % bitbake amltd-console-image
>>
>> The only difference between the two runs is enabling SSTATE_MIRRORS in local.conf
>>     %  diff -u /local/p60_step?/conf/local.conf
>>      --- /local/p60_step1/conf/local.conf    2011-03-09 08:28:18.266933061 -0700
>>      +++ /local/p60_step2/conf/local.conf    2011-03-09 09:57:51.365932951 -0700
>>      @@ -53,4 +53,7 @@
>>       IMAGE_LINGUAS ?= "en-us"
>>
>>       # Minimize feature set
>>       DISTRO_FEATURES ?= "alsa"
>>      +SSTATE_MIRRORS ?= "\
>>      +file://.* file:///local/p60_step1/sstate-cache/"
>>
>> The results seem to have gone through all the same steps (or nearly so).  The output
>> from the runs is at
>>     http://www.mlbassoc.com/poky/build.step1
>>     http://www.mlbassoc.com/poky/build.step2
>>
>> Comparing the two build trees:
>>     % ls /local/p60_step1/tmp/work/*/*/temp/log.do_compile | wc
>>       144     144   12521
>>     % ls /local/p60_step2/tmp/work/*/*/temp/log.do_compile | wc
>>       143     143   12427
>>     % du -s /local/p60_step?
>>     15229296        /local/p60_step1
>>     15162760        /local/p60_step2
>>
>> I know this procedure used to work (or at least close).  Am I doing
>> something wrong?
>
> You're not doing anything wrong and this is the same scenario I've been
> testing with. After I'd fixed the origin problem it created a problem
> with file urls containing globing. I managed to break the original patch
> with the globing fix. The good news is the problem is simple and I've
> pushed a fix.
>
> It should work *much* better than you've seen above, not needing to
> rerun many tasks so if its working it will be very obvious.

Yes, indeed it does!  I just reran step2 from scratch and it finished
in only 13 minutes (with a lot of other things on the build machine so
this is quite good).  There's still quite a lot of stuff in the build
tree:
   % du -s /local/p60_step2
   7285332 /local/p60_step2
but that's only about 1/2 of before.  The results seem very close:
   % ls -l /local/p60_step2/tmp/deploy/images/
   total 10748
   -rwxrw-r--  1 gthomas gthomas 5446150 Mar  9 13:15 amltd-console-image-cobra3530p60-20110309200339.rootfs.tar.bz2
   lrwxrwxrwx  1 gthomas gthomas      62 Mar  9 13:15 amltd-console-image-cobra3530p60.tar.bz2 -> amltd-console-image-cobra3530p60-20110309200339.rootfs.tar.bz2
   drwxrwxr-x 91 gthomas gthomas    4096 Mar  9 13:08 licenses
   -rw-rw-r--  1 gthomas gthomas 3077911 Mar  9 09:24 modules-2.6.37-r0-cobra3530p60.tgz
   -rw-r--r--  1 gthomas gthomas 2468564 Mar  9 09:24 uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
   lrwxrwxrwx  1 gthomas gthomas      48 Mar  9 09:24 uImage-cobra3530p60.bin -> uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
   % ls -l /local/p60_step1/tmp/deploy/images/
   total 10748
   -rwxrw-r--  1 gthomas gthomas 5446783 Mar  9 09:53 amltd-console-image-cobra3530p60-20110309153435.rootfs.tar.bz2
   lrwxrwxrwx  1 gthomas gthomas      62 Mar  9 09:53 amltd-console-image-cobra3530p60.tar.bz2 -> amltd-console-image-cobra3530p60-20110309153435.rootfs.tar.bz2
   drwxrwxr-x 92 gthomas gthomas    4096 Mar  9 09:06 licenses
   -rw-rw-r--  1 gthomas gthomas 3077911 Mar  9 09:24 modules-2.6.37-r0-cobra3530p60.tgz
   -rw-r--r--  1 gthomas gthomas 2468564 Mar  9 09:24 uImage-2.6.37-r0-cobra3530p60-20110309153435.bin
   lrwxrwxrwx  1 gthomas gthomas      48 Mar  9 09:24 uImage-cobra3530p60.bin -> uImage-2.6.37-r0-cobra3530p60-20110309153435.bin

I unpacked the resulting image tarballs and they match, modulo the
package install times in the opkg database.

Looks like it's working correctly now :-)

Can you explain what the sstate files are and how I might make use of them?
In particular, I'd like to be able to hand my customers a sstate cache which
holds the majority of the tools they might need to speed startup.  Sending
the whole thing is a bit problematic as it's quite large, so if I can winnow
it a bit it would help.

Thanks for getting this back in order

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------


      parent reply	other threads:[~2011-03-09 20:26 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-02-24 15:14 Is sstate broken Gary Thomas
2011-02-28  6:07 ` Zhai, Edwin
2011-03-03 12:56   ` Gary Thomas
2011-03-04  1:59     ` Zhai, Edwin
2011-03-04 11:34       ` Gary Thomas
2011-03-09  1:29         ` Richard Purdie
2011-03-09 18:42           ` Gary Thomas
2011-03-09 19:04             ` Richard Purdie
2011-03-09 19:30               ` Koen Kooi
2011-03-09 21:51                 ` Khem Raj
2011-03-09 23:04                   ` Richard Purdie
2011-03-09 20:26               ` Gary Thomas [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4D77E27D.7020901@mlbassoc.com \
    --to=gary@mlbassoc.com \
    --cc=poky@lists.pokylinux.org \
    --cc=richard.purdie@linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox