Using meta-toolchain output as prebuilt toolchains

Using meta-toolchain output as prebuilt toolchains

[Koen Kooi]

Hi,

What is the preferred way of 'importing' prebuilt toolchains into OE-core? In oe.dev I can use this: http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.

My actual use case is actually 2 use-cases:

1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.

So any suggestions for #2 as well?

regards,

Koen

Re: Using meta-toolchain output as prebuilt toolchains

[Khem Raj]

On (27/03/11 13:44), Koen Kooi wrote:
> Hi,
> 
> What is the preferred way of 'importing' prebuilt toolchains into OE-core? In oe.dev I can use this: http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.
> 
> My actual use case is actually 2 use-cases:
> 
> 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain

Did you mean "migrate them to OE-core" ? or are there totaly n00b to OE

> 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.
> 
> So any suggestions for #2 as well?
> 
> regards,
> 
> Koen
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

-- 
-Khem

Re: Using meta-toolchain output as prebuilt toolchains

[Koen Kooi]

Op 27 mrt. 2011 om 21:34 heeft Khem Raj <raj.khem@gmail.com> het volgende geschreven:

> On (27/03/11 13:44), Koen Kooi wrote:
>> Hi,
>> 
>> What is the preferred way of 'importing' prebuilt toolchains into OE-core? In oe.dev I can use this: http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.
>> 
>> My actual use case is actually 2 use-cases:
>> 
>> 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
> 
> Did you mean "migrate them to OE-core" ? or are there totaly n00b to OE
> 

Total n00bs :)


>> 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.
>> 
>> So any suggestions for #2 as well?
>> 
>> regards,
>> 
>> Koen
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
> 
> -- 
> -Khem
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Re: Using meta-toolchain output as prebuilt toolchains

[Tom Rini]

On 03/27/2011 04:44 AM, Koen Kooi wrote:
> Hi,
> 
> What is the preferred way of 'importing' prebuilt toolchains into OE-core? In oe.dev I can use this: http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.
> 
> My actual use case is actually 2 use-cases:
> 
> 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
> 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.
> 
> So any suggestions for #2 as well?

sstate is the best way to catch #2 and I know in general there's "look
at this URI for the file you want" bits.  I haven't yet started kicking
the tires on sstate as hard as I have for packaged-staging however.

For #1, I believe external toolchains are to be supported still, but I
don't know off hand if the recipes still work (but there is both an old
CSL one as well as a poky toolchain one in meta/recipes-core/meta/).
Another area on my TODO list...

-- 
Tom Rini
Mentor Graphics Corporation

Re: Using meta-toolchain output as prebuilt toolchains

[Richard Purdie]

On Sun, 2011-03-27 at 13:44 +0200, Koen Kooi wrote:
> What is the preferred way of 'importing' prebuilt toolchains into
> OE-core? In oe.dev I can use this:
> http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.

In theory, something like that recipe should also work against the
output of OECore's "bitbake meta-toolchain" output.

I would however suggest looking at this for inspiration:
http://git.openembedded.net/cgit.cgi/openembedded-core/tree/meta/recipes-core/meta/external-csl-toolchain_2008q3-72.bb

as the:

GLIBC_INTERNAL_USE_BINARY_LOCALE ?= "compile"
inherit libc-package

might save duplicating half the libc packaging code. You also get the
cross locale generation as an added bonus so no qemu! :)

There is a poky-external-toolchain recipe there too but its broken.
Everything that did, sstate now does better so that should really be
removed. Its advantage was not having to repackage the libc locales but
it was so much hacky code to implement, wasn't multiple package manager
safe and sstate replaces it much more neatly. If anyone does want to use
external toolchains like that, I'd suggest the csl example above as the
way to do it now.

> My actual use case is actually 2 use-cases:
> 
> 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
> 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.

sstate mirrors are the way I think things will work well in the future.
They're being used heavily by the yocto development team internally,
support in Yocto 1.0 is looking good for sstate and hence its looking
good for OE-Core.

To use it simply share the sstate-cache directory over http or nfs or
whatever and then set:

SSTATE_MIRRORS ?= "\
file://.* http://someserver.tld/share/sstate/ \n \
file://.* file:///some/local/dir/sstate/"

to point at wherever the files are...

Cheers,

Richard

Re: Using meta-toolchain output as prebuilt toolchains

[Gary Thomas]

On 03/28/2011 12:58 PM, Tom Rini wrote:
> On 03/27/2011 04:44 AM, Koen Kooi wrote:
>> Hi,
>>
>> What is the preferred way of 'importing' prebuilt toolchains into OE-core? In oe.dev I can use this: http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.
>>
>> My actual use case is actually 2 use-cases:
>>
>> 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
>> 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.
>>
>> So any suggestions for #2 as well?
>
> sstate is the best way to catch #2 and I know in general there's "look
> at this URI for the file you want" bits.  I haven't yet started kicking
> the tires on sstate as hard as I have for packaged-staging however.

This is the approach I've been working on to pass on to my customers.  The
idea that I can give them a packaged bundle which includes prebuilt tools
is far more attractive than a simple poky (or oe-core plus layers) tree
which involves many hours of building to produce a simple image.

Hopefully, the sstate method will become more stable over time.  The problem
I've had with it recently is that there are so many variables which are used
to decide if the sstate data is valid (see Richard's description on how this
works) that if the smallest thing changes, the whole lot becomes useless :-(
so it's really only valid for a [fairly] static repository.

>
> For #1, I believe external toolchains are to be supported still, but I
> don't know off hand if the recipes still work (but there is both an old
> CSL one as well as a poky toolchain one in meta/recipes-core/meta/).
> Another area on my TODO list...
>

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: Using meta-toolchain output as prebuilt toolchains

[Richard Purdie]

On Tue, 2011-03-29 at 05:32 -0600, Gary Thomas wrote:
> On 03/28/2011 12:58 PM, Tom Rini wrote:
> > On 03/27/2011 04:44 AM, Koen Kooi wrote:
> >> What is the preferred way of 'importing' prebuilt toolchains into OE-core? In oe.dev I can use this: http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.
> >>
> >> My actual use case is actually 2 use-cases:
> >>
> >> 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
> >> 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.
> >>
> >> So any suggestions for #2 as well?
> >
> > sstate is the best way to catch #2 and I know in general there's "look
> > at this URI for the file you want" bits.  I haven't yet started kicking
> > the tires on sstate as hard as I have for packaged-staging however.
> 
> This is the approach I've been working on to pass on to my customers.  The
> idea that I can give them a packaged bundle which includes prebuilt tools
> is far more attractive than a simple poky (or oe-core plus layers) tree
> which involves many hours of building to produce a simple image.
> 
> Hopefully, the sstate method will become more stable over time.  The problem
> I've had with it recently is that there are so many variables which are used
> to decide if the sstate data is valid (see Richard's description on how this
> works) that if the smallest thing changes, the whole lot becomes useless :-(
> so it's really only valid for a [fairly] static repository.

It depends what changes your customers expect to make and whether with
those changes you consider the prebuilt data still to be valid. I agree
that the defaults do tend towards detecting even small changes. I think
over time we'll get a better feel for the things that should/shouldn't
be included in there. It is almost totally customisable and what we're
missing now is some good "policy" type data.

Cheers,

Richard

Re: Using meta-toolchain output as prebuilt toolchains

[Denys Dmytriyenko]

On Mon, Mar 28, 2011 at 08:25:08PM +0100, Richard Purdie wrote:
> On Sun, 2011-03-27 at 13:44 +0200, Koen Kooi wrote:
> > What is the preferred way of 'importing' prebuilt toolchains into
> > OE-core? In oe.dev I can use this:
> > http://cgit.openembedded.org/cgit.cgi/openembedded/tree/recipes/meta/external-toolchain-angstrom.bb to 'import' an angstrom toolchain built with 'bitbake meta-toolchain', but I'm not sure how to do that in OE-core.
> 
> In theory, something like that recipe should also work against the
> output of OECore's "bitbake meta-toolchain" output.
> 
> I would however suggest looking at this for inspiration:
> http://git.openembedded.net/cgit.cgi/openembedded-core/tree/meta/recipes-core/meta/external-csl-toolchain_2008q3-72.bb
> 
> as the:
> 
> GLIBC_INTERNAL_USE_BINARY_LOCALE ?= "compile"
> inherit libc-package
> 
> might save duplicating half the libc packaging code. You also get the
> cross locale generation as an added bonus so no qemu! :)
> 
> There is a poky-external-toolchain recipe there too but its broken.
> Everything that did, sstate now does better so that should really be
> removed. Its advantage was not having to repackage the libc locales but
> it was so much hacky code to implement, wasn't multiple package manager
> safe and sstate replaces it much more neatly. If anyone does want to use
> external toolchains like that, I'd suggest the csl example above as the
> way to do it now.

Richard,

Those 2 lines indeed make the recipe much smaller and easier to read! I wish 
we had that support in OE when I worked on the original CSL and Angstrom 
external toolchain recipes... Although, they were inspired by and based on the 
old Poky recipes, but evolved quite a lot since then.

Anyway, I'll try to get some time to port the external recipes to oe-core 
using this new libc-package class. Thanks.

-- 
Denys


> > My actual use case is actually 2 use-cases:
> > 
> > 1) Hand people a prebuilt angstrom toolchain and migrate them to OE while keeping the toolchain
> > 2) "Get started in 10 minutes" type of thing, I suspect an sstate mirror might be better.
> 
> sstate mirrors are the way I think things will work well in the future.
> They're being used heavily by the yocto development team internally,
> support in Yocto 1.0 is looking good for sstate and hence its looking
> good for OE-Core.
> 
> To use it simply share the sstate-cache directory over http or nfs or
> whatever and then set:
> 
> SSTATE_MIRRORS ?= "\
> file://.* http://someserver.tld/share/sstate/ \n \
> file://.* file:///some/local/dir/sstate/"
> 
> to point at wherever the files are...
> 
> Cheers,
> 
> Richard
> 
> 
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core