From: Gary Thomas <gary@mlbassoc.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: "paul.eggleton@linux.intel.com" <paul.eggleton@linux.intel.com>,
Chris Larson <clarson@kergoth.com>,
"poky@pokylinux.org" <poky@pokylinux.org>
Subject: Re: perl binary hard-code path hurts sstate
Date: Tue, 21 Dec 2010 08:14:26 -0700 [thread overview]
Message-ID: <4D10C452.1000405@mlbassoc.com> (raw)
In-Reply-To: <4D109DE4.2070901@mlbassoc.com>
On 12/21/2010 05:30 AM, Gary Thomas wrote:
> On 12/21/2010 04:51 AM, Tian, Kevin wrote:
>>> From: Tian, Kevin
>>> Sent: Tuesday, December 21, 2010 9:10 AM
>>>
>>>> From: Gary Thomas
>>>> Sent: Friday, December 17, 2010 11:09 PM
>>>>
>>>> On 12/17/2010 06:58 AM, Richard Purdie wrote:
>>>>> On Fri, 2010-12-17 at 06:49 -0700, Chris Larson wrote:
>>>>>> These sorts of issues aren't new, or fun, sadly. We've run into many
>>>>>> like this using packaged staging. We had to use the variable to
>>>>>> disable packaged-staging for exactly this reason, so something similar
>>>>>> for sstate would indeed be useful for this.
>>>>>
>>>>> In cases like this I'd suggest adding the encoded path into the checksum
>>>>> via vardeps. This means the sstate package will be reused when the path
>>>>> matches but not otherwise.
>>>>>
>>>>> We can then work on removing the dependencies which I agree is the goal
>>>>> but we should get them identified first.
>>>>
>>>> Sadly, this problem is not confined to perl. I just tried this experiment:
>>
>> Hi, Gary, I can't reproduce in my side, though I use only one machine.
>>
>>>> * Build Poky on MachineA to some directory $NEW
>>>> * Copy Poky ($POKYBASE) and $NEW/sstate-cache to MachineB, in this case
>>>> the path for $POKYBASE is the same on both machines but $NEW differs.
>>>> MachineA:$NEW does not exist on MachineB
>>>> * Try to build on Machine B, pointing SSTATE_MIRRORS to $NEW/sstate-cache
>>>> This build is in $NEW2
>>
>> compare to your steps, mine is:
>>
>> * build Poky in $DIR1, with sstate generated in ${SSTATE-CACHE} (a global one)
>> * then with same $POKYBASE, create a new build $DIR2, with SSTATE_MIRRORS
>> pointing to ${SSTATE-CACHE}
>> * move $DIR1 to $DIR1.bak
>>
>> This way if there's reference to $DIR1, I should be able to observe failure
>>
>>>> The process fails for me building the kernel:
>>>> | arm-poky-linux-gnueabi-ld: libgcc.a: No such file: No such file or directory
>>>> Looking carefully, I can see that the compiler is looking for libgcc.a in
>>>> the directory path $NEW that is only on MachineA.
>>
>> However I didn't see above failure and most sstate packages could be reused
>> successfully except several ones depending on perl-native...
>>
>>>>
>>>> So this is the same problem Paul had, just for the cross compiler
>>>> instead of perl.
>>>>
>>>> n.b. Is there a bug # I should use to be tracking this?
>>>>
>>>
>>> I'll see whether I can reproduce this after RP's new fix about perl-native.
>>
>> so Gary, could you try latest poky master or verify whether my single-machine steps
>> may expose same failure in your side? Or is there any step not equivalent between
>> our cases?
>
> Yes, I'll try it now, thanks
>
Still fails. There is one step I wasn't clear on - if you
just follow the steps above, you'll succeed and get a working
result, using only the staged (sstate cached) packages. However,
if you try and build some other package, or in my case I forced
a build of the kernel, it fails.
Amended steps to reproduce this failure:
* Build Poky on MachineA, results in MachineA:NEW1
* Copy MachineA:NEW1/sstate-cache to MachineB:NEW1_CACHE
* Build Poky on MachineB, results in MachineB:NEW2, using
MachineB:NEW1_CACHE for SSTATE_MIRRORS
* Disable SSTATE_MIRRORS in conf/local.conf on MachineB:NEW2
* Build a package from scratch. In my case, I rebuilt the kernel:
- bitbake virtual/kernel -c clean
- rm sstate-cache/sstate-linux-am*
- bitbake virtual/kernel
My kernel is based on a local recipe linux-am_X.Y.Z
Note: I tried to simplify this process by using 'cleanall'
but ran into a fetch problem (reported separately)
Notes:
* For my testing, I used a simple package set, e.g. poky-image-minimal
* My Poky tree is based on today's master 4cd40dbaa8aef7b6b9ff9ce892a576b802f4b1ec
* I notice that this process now rebuilds some 20 packages from scratch
on MachineB. These include the perl-native, etc, that this thread has
been discussing. Is this expected?
Thanks for working on this; it's getting close and will be
a big help to my customers.
--
------------------------------------------------------------
Gary Thomas | Consulting for the
MLB Associates | Embedded world
------------------------------------------------------------
next prev parent reply other threads:[~2010-12-21 15:14 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-17 1:38 Shared state prebuild copy bug report (bug #602) Scott Garman
2010-12-17 2:06 ` Tian, Kevin
2010-12-17 2:15 ` Scott Garman
2010-12-17 2:19 ` Tian, Kevin
2010-12-17 2:30 ` Scott Garman
2010-12-17 2:35 ` Tian, Kevin
2010-12-17 4:47 ` Tian, Kevin
2010-12-17 5:29 ` perl binary hard-code path hurts sstate Tian, Kevin
2010-12-17 13:49 ` Chris Larson
2010-12-17 13:58 ` Richard Purdie
2010-12-17 15:08 ` Gary Thomas
2010-12-21 1:09 ` Tian, Kevin
2010-12-21 11:51 ` Tian, Kevin
2010-12-21 12:30 ` Gary Thomas
2010-12-21 15:14 ` Gary Thomas [this message]
2010-12-22 1:25 ` Tian, Kevin
2010-12-22 12:07 ` Tian, Kevin
2010-12-22 12:35 ` Gary Thomas
2010-12-22 13:38 ` Tian, Kevin
2010-12-22 13:42 ` Tian, Kevin
2010-12-23 4:52 ` Tian, Kevin
2010-12-21 11:43 ` Tian, Kevin
2010-12-17 5:43 ` Shared state prebuild copy bug report (bug #602) Scott Garman
2010-12-20 22:58 ` Richard Purdie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D10C452.1000405@mlbassoc.com \
--to=gary@mlbassoc.com \
--cc=clarson@kergoth.com \
--cc=kevin.tian@intel.com \
--cc=paul.eggleton@linux.intel.com \
--cc=poky@pokylinux.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox