* Using SSTATE_MIRRORS with sstate subdirectories
@ 2012-10-19 15:38 Mike Crowe
2012-10-19 15:46 ` Chris Larson
0 siblings, 1 reply; 10+ messages in thread
From: Mike Crowe @ 2012-10-19 15:38 UTC (permalink / raw)
To: openembedded-core
I'm having trouble using SSTATE_MIRRORS as suggested at
https://wiki.yoctoproject.org/wiki/Enable_sstate_cache :
SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/"
Since commit 5d45c7b025f6635b2232d7bf92b7c1aba350396b my sstate-cache
files are stored in two-letter subdirectories (and
9360fc318cec5c873b17f59b817dd6312c81ee27 adds the LSB prefix too.)
Although these components are present in the original URL they don't
make it into the rewritten one:
DEBUG: SState using premirror of: file://.* file:///private/sstate-cache/
DEBUG: SState: Attempting to fetch file://Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz
DEBUG: Searching for Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz in paths:
DEBUG: Searching for Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz in path: /vg0disk/mac/src/oe2/build-bcm7425/conf/.
DEBUG: Defaulting to /vg0disk/mac/src/oe2/build-bcm7425/sstate-cache/Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz for Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz
DEBUG: Testing URL file://Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz
DEBUG: For url ['file', '', 'Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz', '', '', {}] comparing ['file', '', '.*', '', '', {}] to ['file', '', '/private/sstate-cache/', '', '', {}]
DEBUG: For url file://Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz returning file:///private/sstate-cache/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz
DEBUG: For url ['file', '', '/private/sstate-cache/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz', '', '', {}] comparing ['file', '', '.*', '', '', {}] to ['file', '', '/private/sstate-cache/', '', '', {}]
DEBUG: SState: Unsuccessful fetch test for file://Debian-testing/b0/sstate-quilt-native-x86_64-linux-0.60-r0-x86_64-2-b0869ac9e0e74960018909e56a6822a1_populate-sysroot.tgz
Inspired by 9360fc318cec5c873b17f59b817dd6312c81ee27 I tried:
SSTATE_MIRRORS ?= "file://(.*) file:///private/sstate-cache/\1"
But this just resulted in an infinite loop with progressively more
private/sstate-cache/ components added to the path name.
I've had a poke at fetch2's uri_replace but it isn't obvious to me how
to change this behaviour without risking breaking something else.
Any ideas?
Thanks.
Mike.
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 15:38 Using SSTATE_MIRRORS with sstate subdirectories Mike Crowe
@ 2012-10-19 15:46 ` Chris Larson
2012-10-19 15:52 ` Paul Eggleton
2012-10-19 16:57 ` Mike Crowe
0 siblings, 2 replies; 10+ messages in thread
From: Chris Larson @ 2012-10-19 15:46 UTC (permalink / raw)
To: Mike Crowe; +Cc: openembedded-core
On Fri, Oct 19, 2012 at 8:38 AM, Mike Crowe <mac@mcrowe.com> wrote:
> I'm having trouble using SSTATE_MIRRORS as suggested at
> https://wiki.yoctoproject.org/wiki/Enable_sstate_cache :
>
> SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/"
SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/PATH"
--
Christopher Larson
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 15:46 ` Chris Larson
@ 2012-10-19 15:52 ` Paul Eggleton
2012-10-19 16:02 ` Mike Crowe
2012-10-19 16:57 ` Mike Crowe
1 sibling, 1 reply; 10+ messages in thread
From: Paul Eggleton @ 2012-10-19 15:52 UTC (permalink / raw)
To: openembedded-core; +Cc: Chris Larson, Mike Crowe
On Friday 19 October 2012 08:46:23 Chris Larson wrote:
> On Fri, Oct 19, 2012 at 8:38 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > I'm having trouble using SSTATE_MIRRORS as suggested at
> > https://wiki.yoctoproject.org/wiki/Enable_sstate_cache :
> >
> > SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/"
>
> SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/PATH"
Note: with master / danny, but not denzil or earlier.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 15:52 ` Paul Eggleton
@ 2012-10-19 16:02 ` Mike Crowe
0 siblings, 0 replies; 10+ messages in thread
From: Mike Crowe @ 2012-10-19 16:02 UTC (permalink / raw)
To: Paul Eggleton
On Fri, Oct 19, 2012 at 04:52:25PM +0100, Paul Eggleton wrote:
> On Friday 19 October 2012 08:46:23 Chris Larson wrote:
> > On Fri, Oct 19, 2012 at 8:38 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > > I'm having trouble using SSTATE_MIRRORS as suggested at
> > > https://wiki.yoctoproject.org/wiki/Enable_sstate_cache :
> > >
> > > SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/"
> >
> > SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/PATH"
Thanks Christopher. That works.
> Note: with master / danny, but not denzil or earlier.
I'm wary of attempting to update the wiki page for that reason.
Mike.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 15:46 ` Chris Larson
2012-10-19 15:52 ` Paul Eggleton
@ 2012-10-19 16:57 ` Mike Crowe
2012-10-19 17:41 ` Chris Larson
2012-10-19 18:24 ` McClintock Matthew-B29882
1 sibling, 2 replies; 10+ messages in thread
From: Mike Crowe @ 2012-10-19 16:57 UTC (permalink / raw)
To: Chris Larson, openembedded-core
On Fri, Oct 19, 2012 at 08:46:23AM -0700, Chris Larson wrote:
> On Fri, Oct 19, 2012 at 8:38 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > I'm having trouble using SSTATE_MIRRORS as suggested at
> > https://wiki.yoctoproject.org/wiki/Enable_sstate_cache :
> >
> > SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/"
>
> SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/PATH"
Thanks for your reply.
Although that works for the simple case, if I try and do something
more adventurous like:
SSTATE_MIRRORS = "\
file://Debian-testing/.* file:///private/sstate-cache/Debian-6.0.6/PATH \n \
file://.* file:///private/sstate-cache/PATH \n \
"
Then I get paths like:
DEBUG: For url file://Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz returning file:///private/sstate-cache/Debian-6.0.6/Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz
so I really would like to be able to say:
"file://Debian-testing/(.*) file:///private/sstate-cache/Debian-6.0.6/\1"
Thanks.
Mike.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 16:57 ` Mike Crowe
@ 2012-10-19 17:41 ` Chris Larson
2012-10-19 18:33 ` Mike Crowe
2012-10-19 18:24 ` McClintock Matthew-B29882
1 sibling, 1 reply; 10+ messages in thread
From: Chris Larson @ 2012-10-19 17:41 UTC (permalink / raw)
To: Mike Crowe; +Cc: openembedded-core
On Fri, Oct 19, 2012 at 9:57 AM, Mike Crowe <mac@mcrowe.com> wrote:
> SSTATE_MIRRORS = "\
> file://Debian-testing/.* file:///private/sstate-cache/Debian-6.0.6/PATH \n \
> file://.* file:///private/sstate-cache/PATH \n \
> "
>
> Then I get paths like:
>
> DEBUG: For url file://Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz returning file:///private/sstate-cache/Debian-6.0.6/Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz
SSTATE_MIRRORS = "\
file://${LSBNATIVESTRING} file:///private/sstate-cache/Debian-6.0.6 \n \
file://.* file:///private/sstate-cache/PATH \n \
"
We do this at mentor with a class to simplify the setup. See
http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor/tree/classes/sstate-reuse.bbclass.
Then you'd do this:
INHERIT += "sstate-reuse"
SSTATE_MIRROR_DISTROS = "Debian-testing"
SSTATE_MIRROR_SITES = "file:///private/sstate-cache"
We do most of our automated builds on 32 bit and 64 bit centos 5.4
hosts, then those native/cross sstates can then be reused just about
everywhere, since that distro is so old.
--
Christopher Larson
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 17:41 ` Chris Larson
@ 2012-10-19 18:33 ` Mike Crowe
2012-10-19 19:03 ` Chris Larson
0 siblings, 1 reply; 10+ messages in thread
From: Mike Crowe @ 2012-10-19 18:33 UTC (permalink / raw)
To: Chris Larson, openembedded-core
On Fri, Oct 19, 2012 at 10:41:17AM -0700, Chris Larson wrote:
> On Fri, Oct 19, 2012 at 9:57 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > SSTATE_MIRRORS = "\
> > file://Debian-testing/.* file:///private/sstate-cache/Debian-6.0.6/PATH \n \
> > file://.* file:///private/sstate-cache/PATH \n \
> > "
> >
> > Then I get paths like:
> >
> > DEBUG: For url file://Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz returning file:///private/sstate-cache/Debian-6.0.6/Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz
>
> SSTATE_MIRRORS = "\
> file://${LSBNATIVESTRING} file:///private/sstate-cache/Debian-6.0.6 \n \
> file://.* file:///private/sstate-cache/PATH \n \
> "
Hi Chris,
Thanks for your reply.
After correcting ${LSBNATIVESTRING} to ${NATIVELSBSTRING} that worked!
I'm still somewhat baffled as to why that one doesn't require PATH but
the general one does but that no longer matters to me.
> We do this at mentor with a class to simplify the setup. See
> http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor/tree/classes/sstate-reuse.bbclass.
> Then you'd do this:
>
> INHERIT += "sstate-reuse"
> SSTATE_MIRROR_DISTROS = "Debian-testing"
> SSTATE_MIRROR_SITES = "file:///private/sstate-cache"
>
> We do most of our automated builds on 32 bit and 64 bit centos 5.4
> hosts, then those native/cross sstates can then be reused just about
> everywhere, since that distro is so old.
Thanks. That looks interesting. I'm trying to do something similar,
except our main automated build machine is Debian stable and the
"clients" are Debian stable, testing and Ubuntu recent. I'd made the
same assumption that binaries compiledon Debian stable should run on
all the others.
Thanks again.
Mike.
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 18:33 ` Mike Crowe
@ 2012-10-19 19:03 ` Chris Larson
2012-10-19 21:01 ` Mike Crowe
0 siblings, 1 reply; 10+ messages in thread
From: Chris Larson @ 2012-10-19 19:03 UTC (permalink / raw)
To: Mike Crowe; +Cc: openembedded-core
On Fri, Oct 19, 2012 at 11:33 AM, Mike Crowe <mac@mcrowe.com> wrote:
> Thanks for your reply.
>
> After correcting ${LSBNATIVESTRING} to ${NATIVELSBSTRING} that worked!
>
> I'm still somewhat baffled as to why that one doesn't require PATH but
> the general one does but that no longer matters to me.
The file://.* in the other one dose a replacement of the full url
contents, so you have to add back the directory component. The other
one I showed you just does a search/replace on the leading component,
leaving the rest alone.
--
Christopher Larson
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 19:03 ` Chris Larson
@ 2012-10-19 21:01 ` Mike Crowe
0 siblings, 0 replies; 10+ messages in thread
From: Mike Crowe @ 2012-10-19 21:01 UTC (permalink / raw)
To: Chris Larson, openembedded-core
On Fri, Oct 19, 2012 at 12:03:09PM -0700, Chris Larson wrote:
> On Fri, Oct 19, 2012 at 11:33 AM, Mike Crowe <mac@mcrowe.com> wrote:
> > Thanks for your reply.
> >
> > After correcting ${LSBNATIVESTRING} to ${NATIVELSBSTRING} that worked!
> >
> > I'm still somewhat baffled as to why that one doesn't require PATH but
> > the general one does but that no longer matters to me.
>
> The file://.* in the other one dose a replacement of the full url
> contents, so you have to add back the directory component. The other
> one I showed you just does a search/replace on the leading component,
> leaving the rest alone.
Thanks again Chris. That makes sense now. I would now expect:
file:// file:///private/sstate-cache
to work just as well as:
file://.* file:///private/sstate-cache/PATH
I'll try that later.
Anyway, I ended up with:
SSTATE_MIRRORS = "\
file://Debian-[^/]* file:///private/sstate-cache/Debian-6 \n \
file://Ubuntu-[^/]* file:///private/sstate-cache/Debian-6 \n \
file://.* file:///private/sstate-cache/PATH \n \
along with a tweaked version of your lsb_distro_adjust hook from
http://git.yoctoproject.org/cgit/cgit.cgi/meta-mentor/tree/conf/distro/include/sstate.inc
to collapse Debian minor versions.
This seems to do exactly what I want.
Thanks to everyone who responded.
Mike.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Using SSTATE_MIRRORS with sstate subdirectories
2012-10-19 16:57 ` Mike Crowe
2012-10-19 17:41 ` Chris Larson
@ 2012-10-19 18:24 ` McClintock Matthew-B29882
1 sibling, 0 replies; 10+ messages in thread
From: McClintock Matthew-B29882 @ 2012-10-19 18:24 UTC (permalink / raw)
To: Mike Crowe; +Cc: Chris Larson, openembedded-core@lists.openembedded.org
On Fri, Oct 19, 2012 at 11:57 AM, Mike Crowe <mac@mcrowe.com> wrote:
> On Fri, Oct 19, 2012 at 08:46:23AM -0700, Chris Larson wrote:
>> On Fri, Oct 19, 2012 at 8:38 AM, Mike Crowe <mac@mcrowe.com> wrote:
>> > I'm having trouble using SSTATE_MIRRORS as suggested at
>> > https://wiki.yoctoproject.org/wiki/Enable_sstate_cache :
>> >
>> > SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/"
>>
>> SSTATE_MIRRORS ?= "file://.* file:///private/sstate-cache/PATH"
>
> Thanks for your reply.
>
> Although that works for the simple case, if I try and do something
> more adventurous like:
>
> SSTATE_MIRRORS = "\
> file://Debian-testing/.* file:///private/sstate-cache/Debian-6.0.6/PATH \n \
> file://.* file:///private/sstate-cache/PATH \n \
> "
>
> Then I get paths like:
>
> DEBUG: For url file://Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz returning file:///private/sstate-cache/Debian-6.0.6/Debian-testing/8c/sstate-tar-replacement-native-x86_64-linux-1.26-r3-x86_64-2-8cc4342260b064ace38e0aa1acf2f618_populate-sysroot.tgz
>
> so I really would like to be able to say:
>
> "file://Debian-testing/(.*) file:///private/sstate-cache/Debian-6.0.6/\1"
Something like this might be what you are looking for:
SSTATE_MIRRORS = "file://.*/(.*)/(.*)
http://linux.freescale.net/yocto/sstate-cache/CentOS-5.8/\1/\2 \n"
This maps all native sstate-cache to my CentOS-5 box.
-M
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2012-10-19 21:14 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-10-19 15:38 Using SSTATE_MIRRORS with sstate subdirectories Mike Crowe
2012-10-19 15:46 ` Chris Larson
2012-10-19 15:52 ` Paul Eggleton
2012-10-19 16:02 ` Mike Crowe
2012-10-19 16:57 ` Mike Crowe
2012-10-19 17:41 ` Chris Larson
2012-10-19 18:33 ` Mike Crowe
2012-10-19 19:03 ` Chris Larson
2012-10-19 21:01 ` Mike Crowe
2012-10-19 18:24 ` McClintock Matthew-B29882
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox