Poky Project Discussions
 help / color / mirror / Atom feed
From: Gary Thomas <gary@mlbassoc.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: "paul.eggleton@linux.intel.com" <paul.eggleton@linux.intel.com>,
	Chris Larson <clarson@kergoth.com>,
	"poky@pokylinux.org" <poky@pokylinux.org>
Subject: Re: perl binary hard-code path hurts sstate
Date: Tue, 21 Dec 2010 08:14:26 -0700	[thread overview]
Message-ID: <4D10C452.1000405@mlbassoc.com> (raw)
In-Reply-To: <4D109DE4.2070901@mlbassoc.com>

On 12/21/2010 05:30 AM, Gary Thomas wrote:
> On 12/21/2010 04:51 AM, Tian, Kevin wrote:
>>> From: Tian, Kevin
>>> Sent: Tuesday, December 21, 2010 9:10 AM
>>>
>>>> From: Gary Thomas
>>>> Sent: Friday, December 17, 2010 11:09 PM
>>>>
>>>> On 12/17/2010 06:58 AM, Richard Purdie wrote:
>>>>> On Fri, 2010-12-17 at 06:49 -0700, Chris Larson wrote:
>>>>>> These sorts of issues aren't new, or fun, sadly. We've run into many
>>>>>> like this using packaged staging. We had to use the variable to
>>>>>> disable packaged-staging for exactly this reason, so something similar
>>>>>> for sstate would indeed be useful for this.
>>>>>
>>>>> In cases like this I'd suggest adding the encoded path into the checksum
>>>>> via vardeps. This means the sstate package will be reused when the path
>>>>> matches but not otherwise.
>>>>>
>>>>> We can then work on removing the dependencies which I agree is the goal
>>>>> but we should get them identified first.
>>>>
>>>> Sadly, this problem is not confined to perl. I just tried this experiment:
>>
>> Hi, Gary, I can't reproduce in my side, though I use only one machine.
>>
>>>> * Build Poky on MachineA to some directory $NEW
>>>> * Copy Poky ($POKYBASE) and $NEW/sstate-cache to MachineB, in this case
>>>> the path for $POKYBASE is the same on both machines but $NEW differs.
>>>> MachineA:$NEW does not exist on MachineB
>>>> * Try to build on Machine B, pointing SSTATE_MIRRORS to $NEW/sstate-cache
>>>> This build is in $NEW2
>>
>> compare to your steps, mine is:
>>
>> * build Poky in $DIR1, with sstate generated in ${SSTATE-CACHE} (a global one)
>> * then with same $POKYBASE, create a new build $DIR2, with SSTATE_MIRRORS
>> pointing to ${SSTATE-CACHE}
>> * move $DIR1 to $DIR1.bak
>>
>> This way if there's reference to $DIR1, I should be able to observe failure
>>
>>>> The process fails for me building the kernel:
>>>> | arm-poky-linux-gnueabi-ld: libgcc.a: No such file: No such file or directory
>>>> Looking carefully, I can see that the compiler is looking for libgcc.a in
>>>> the directory path $NEW that is only on MachineA.
>>
>> However I didn't see above failure and most sstate packages could be reused
>> successfully except several ones depending on perl-native...
>>
>>>>
>>>> So this is the same problem Paul had, just for the cross compiler
>>>> instead of perl.
>>>>
>>>> n.b. Is there a bug # I should use to be tracking this?
>>>>
>>>
>>> I'll see whether I can reproduce this after RP's new fix about perl-native.
>>
>> so Gary, could you try latest poky master or verify whether my single-machine steps
>> may expose same failure in your side? Or is there any step not equivalent between
>> our cases?
>
> Yes, I'll try it now, thanks
>

Still fails.  There is one step I wasn't clear on - if you
just follow the steps above, you'll succeed and get a working
result, using only the staged (sstate cached) packages.  However,
if you try and build some other package, or in my case I forced
a build of the kernel, it fails.

Amended steps to reproduce this failure:
   * Build Poky on MachineA, results in MachineA:NEW1
   * Copy MachineA:NEW1/sstate-cache to MachineB:NEW1_CACHE
   * Build Poky on MachineB, results in MachineB:NEW2, using
     MachineB:NEW1_CACHE for SSTATE_MIRRORS
   * Disable SSTATE_MIRRORS in conf/local.conf on MachineB:NEW2
   * Build a package from scratch.  In my case, I rebuilt the kernel:
     - bitbake virtual/kernel -c clean
     - rm sstate-cache/sstate-linux-am*
     - bitbake virtual/kernel
     My kernel is based on a local recipe linux-am_X.Y.Z

     Note: I tried to simplify this process by using 'cleanall'
     but ran into a fetch problem (reported separately)

Notes:
   * For my testing, I used a simple package set, e.g. poky-image-minimal
   * My Poky tree is based on today's master 4cd40dbaa8aef7b6b9ff9ce892a576b802f4b1ec
   * I notice that this process now rebuilds some 20 packages from scratch
     on MachineB.  These include the perl-native, etc, that this thread has
     been discussing.  Is this expected?

Thanks for working on this; it's getting close and will be
a big help to my customers.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------


  reply	other threads:[~2010-12-21 15:14 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-12-17  1:38 Shared state prebuild copy bug report (bug #602) Scott Garman
2010-12-17  2:06 ` Tian, Kevin
2010-12-17  2:15   ` Scott Garman
2010-12-17  2:19     ` Tian, Kevin
2010-12-17  2:30       ` Scott Garman
2010-12-17  2:35         ` Tian, Kevin
2010-12-17  4:47         ` Tian, Kevin
2010-12-17  5:29           ` perl binary hard-code path hurts sstate Tian, Kevin
2010-12-17 13:49             ` Chris Larson
2010-12-17 13:58               ` Richard Purdie
2010-12-17 15:08                 ` Gary Thomas
2010-12-21  1:09                   ` Tian, Kevin
2010-12-21 11:51                     ` Tian, Kevin
2010-12-21 12:30                       ` Gary Thomas
2010-12-21 15:14                         ` Gary Thomas [this message]
2010-12-22  1:25                           ` Tian, Kevin
2010-12-22 12:07                             ` Tian, Kevin
2010-12-22 12:35                               ` Gary Thomas
2010-12-22 13:38                                 ` Tian, Kevin
2010-12-22 13:42                                   ` Tian, Kevin
2010-12-23  4:52                           ` Tian, Kevin
2010-12-21 11:43                 ` Tian, Kevin
2010-12-17  5:43           ` Shared state prebuild copy bug report (bug #602) Scott Garman
2010-12-20 22:58             ` Richard Purdie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4D10C452.1000405@mlbassoc.com \
    --to=gary@mlbassoc.com \
    --cc=clarson@kergoth.com \
    --cc=kevin.tian@intel.com \
    --cc=paul.eggleton@linux.intel.com \
    --cc=poky@pokylinux.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox