Usage of meta recipe external-toolchain-csl

Usage of meta recipe external-toolchain-csl

[Jerry Jacobs]

Dear all,

I'm very new to openembedded and already have a clear view how it is
structured. My embedded board mini2440 is already in git and want to create
my own distro.
The problem where I bump against is how to use the Codesourcery external
toolchain for my target. I know there is the file in recipes/meta/
external-toolchain-csl.bb
but it not clear how to use it.

This is how my tree looks like:

/opt/arm-2010q1/bin  ==> codesourcery toolchain

/opt/mini2440/oe/
   |----- bitbake              ==> bitbake
   |----- openembedded ==> openembedded overlay
   |----- tunebox             ==> my distro buildir
           | ------ conf        ==> configuration dir for local.conf
           | ------ setenv    ==> script to set the environment

setenv:
export OEBASE=/opt/mini2440/oe/openembedded
export PATH=/opt/mini2440/oe/bitbake/bin:/opt/arm-2010q1/bin:$PATH
export BBPATH=/opt/mini2440/oe/tunebox:$OEBASE
export BB_ENV_EXTRAWHITE="OEBASE"

local.conf:
BBFILES = "/opt/mini2440/oe/openembedded/recipes/*/*.bb"
DISTRO = "micro"
MACHINE = "mini2440"

I found also the arago distro which has a toolchain-csl.conf and includes it
in local.conf, but they do black magic to set the variables:
TOOLCHAIN_SYSPATH
TOOLCHAIN_PATH
SDK_PATH

which can be found in the git repository.

I hope somebody can clear some things up, because its very obsecure if you
start from scratch.

Kind regards,
Jerry Jacobs

Re: Usage of meta recipe external-toolchain-csl

[C Michael Sundius]

This is my understanding:

- the meta-toolchain.bb creates an sdk from a tool chain built by OE.
- the external-toolchain.bb takes the sdk built by meta-toolchain.bb  
and installs it back into oe to be used by rest of the build.

If you are using a CS toolchain that you didn't build yourself w/ the  
meta toolchain recipe then you might have to make some tweeks here and  
there, but in general the external toolchain recipe will pull the  
toolchain into your oe environment:

  -- the compiler into the cross or host staging area
  -- the binutils into the cross or host staging area
  -- the libraries (libc libgcc etc) into the target staging area
  -- the headerfiles into the target staging area

you need to set the PREFERRED_PROVIDER for each of these packages to  
point to the external-toolchain recipe that you are using. This is  
described in the OE user manual.

its something like this:
PREFERRED_PROVIDER_libc-headers = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc = "external-mips-linux- 
toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc-initial = "external- 
mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}gcc-intermediate =  
"external-mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}binutils = "external-mips- 
linux-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}libc-for-gcc = "external- 
mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/libc = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/libintl = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/libiconv = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_glibc-utils = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_libsegfault = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_glibc-thread-db = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_libgcc-dev = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_libgcc = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_libstdc++-dev = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_libstdc++ = "external-mips-linux-toolchain"
PREFERRED_PROVIDER_virtual/${TARGET_PREFIX}libc-initial = "external- 
mips-linux-toolchain"

that's what I use in my distro. Note that we are using a bit of a non- 
standard toolchain forced upon us for legacy reasons and some things  
were not in the exact same place that the external-toolchain recipe  
was expecting so we had to do our own recipe.

I'm by no means an expert, but at least that's how we have things  
working :] good luck

mike

On Apr 30, 2010, at 6:34 AM, Jerry Jacobs wrote:

> Dear all,
>
> I'm very new to openembedded and already have a clear view how it is
> structured. My embedded board mini2440 is already in git and want to  
> create
> my own distro.
> The problem where I bump against is how to use the Codesourcery  
> external
> toolchain for my target. I know there is the file in recipes/meta/
> external-toolchain-csl.bb
> but it not clear how to use it.
>
> This is how my tree looks like:
>
> /opt/arm-2010q1/bin  ==> codesourcery toolchain
>
> /opt/mini2440/oe/
>   |----- bitbake              ==> bitbake
>   |----- openembedded ==> openembedded overlay
>   |----- tunebox             ==> my distro buildir
>           | ------ conf        ==> configuration dir for local.conf
>           | ------ setenv    ==> script to set the environment
>
> setenv:
> export OEBASE=/opt/mini2440/oe/openembedded
> export PATH=/opt/mini2440/oe/bitbake/bin:/opt/arm-2010q1/bin:$PATH
> export BBPATH=/opt/mini2440/oe/tunebox:$OEBASE
> export BB_ENV_EXTRAWHITE="OEBASE"
>
> local.conf:
> BBFILES = "/opt/mini2440/oe/openembedded/recipes/*/*.bb"
> DISTRO = "micro"
> MACHINE = "mini2440"
>
> I found also the arago distro which has a toolchain-csl.conf and  
> includes it
> in local.conf, but they do black magic to set the variables:
> TOOLCHAIN_SYSPATH
> TOOLCHAIN_PATH
> SDK_PATH
>
> which can be found in the git repository.
>
> I hope somebody can clear some things up, because its very obsecure  
> if you
> start from scratch.
>
> Kind regards,
> Jerry Jacobs
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel

Re: Usage of meta recipe external-toolchain-csl

[Tom Rini]

On Fri, 2010-04-30 at 15:34 +0200, Jerry Jacobs wrote:
> Dear all,
> 
> I'm very new to openembedded and already have a clear view how it is
> structured. My embedded board mini2440 is already in git and want to create
> my own distro.
> The problem where I bump against is how to use the Codesourcery external
> toolchain for my target. I know there is the file in recipes/meta/
> external-toolchain-csl.bb
> but it not clear how to use it.

There's 2 ways.  One way is, in local.conf:
### CSL public ARM does <arch>-none-<os>, so fill in the -none
TARGET_VENDOR = "-none"
### Bring in external-toolchain-csl.bb
TOOLCHAIN_BRAND = "csl"
### Where the CSL version is installed.
TOOLCHAIN_PATH = "/full/path/to/arm-2009q3"
### Where inside the CSL install the target libraries exist.
TOOLCHAIN_SYSPATH = "${TOOLCHAIN_PATH}/arm-none-linux-gnueabi"
### Add tools to the PATH automatically.
PATH_prepend = "${TOOLCHAIN_PATH}/bin:"
require conf/distro/include/toolchain-external.inc

The other way has a few less variables in local.conf but assumes PATH is
already updated.  Denys will reply shortly with that, I imagine.

-- 
Tom Rini <tom_rini@mentor.com>
Mentor Graphics Corporation

Re: Usage of meta recipe external-toolchain-csl

[Denys Dmytriyenko]

On Fri, Apr 30, 2010 at 09:53:06AM -0700, Tom Rini wrote:
> On Fri, 2010-04-30 at 15:34 +0200, Jerry Jacobs wrote:
> > Dear all,
> > 
> > I'm very new to openembedded and already have a clear view how it is
> > structured. My embedded board mini2440 is already in git and want to create
> > my own distro.
> > The problem where I bump against is how to use the Codesourcery external
> > toolchain for my target. I know there is the file in recipes/meta/
> > external-toolchain-csl.bb
> > but it not clear how to use it.
> 
> There's 2 ways.  One way is, in local.conf:
> ### CSL public ARM does <arch>-none-<os>, so fill in the -none
> TARGET_VENDOR = "-none"
> ### Bring in external-toolchain-csl.bb
> TOOLCHAIN_BRAND = "csl"
> ### Where the CSL version is installed.
> TOOLCHAIN_PATH = "/full/path/to/arm-2009q3"
> ### Where inside the CSL install the target libraries exist.
> TOOLCHAIN_SYSPATH = "${TOOLCHAIN_PATH}/arm-none-linux-gnueabi"
> ### Add tools to the PATH automatically.
> PATH_prepend = "${TOOLCHAIN_PATH}/bin:"
> require conf/distro/include/toolchain-external.inc
> 
> The other way has a few less variables in local.conf but assumes PATH is
> already updated.  Denys will reply shortly with that, I imagine.

The only difference I have from Tom's setup is I expect the CSL toolchain 
being already in the PATH variable:

export PATH=/full/path/to/arm-2009q1/bin:$PATH

And then set the other several BitBake variables accordingly (in local.conf 
and toolchain-csl.conf):

TARGET_VENDOR = "-none"
# no need to include/require toolchain-external.inc, as TOOLCHAIN_TYPE does it
TOOLCHAIN_TYPE = "external"
TOOLCHAIN_BRAND = "csl"

[There I also set CPPFLAGS and LDFLAGS to point to CSL directly, but it's not 
required as necessary files are staged by the recipe these days.]

The "black magic" in the local.conf is basically a Python code to set 
TOOLCHAIN_PATH and TOOLCHAIN_SYSPATH based on the user's PATH:

TOOLCHAIN_PATH = "${@code-to-extract-csl-path-from-PATH}"
TOOLCHAIN_SYSPATH = ${TOOLCHAIN_PATH}/${TARGET_SYS}

So, with my setup, no modifications to local.conf are required, as long as CSL 
is in your PATH. With Tom's setup, users would need to add the CSL path to 
local.conf.

Hope this helps.

-- 
Denys

Re: Usage of meta recipe external-toolchain-csl

[Jerry Jacobs]

On Fri, Apr 30, 2010 at 7:58 PM, Denys Dmytriyenko <denis@denix.org> wrote:
>
> On Fri, Apr 30, 2010 at 09:53:06AM -0700, Tom Rini wrote:
> > On Fri, 2010-04-30 at 15:34 +0200, Jerry Jacobs wrote:
> > > Dear all,
> > >
> > > I'm very new to openembedded and already have a clear view how it is
> > > structured. My embedded board mini2440 is already in git and want to create
> > > my own distro.
> > > The problem where I bump against is how to use the Codesourcery external
> > > toolchain for my target. I know there is the file in recipes/meta/
> > > external-toolchain-csl.bb
> > > but it not clear how to use it.
> >
> > There's 2 ways.  One way is, in local.conf:
> > ### CSL public ARM does <arch>-none-<os>, so fill in the -none
> > TARGET_VENDOR = "-none"
> > ### Bring in external-toolchain-csl.bb
> > TOOLCHAIN_BRAND = "csl"
> > ### Where the CSL version is installed.
> > TOOLCHAIN_PATH = "/full/path/to/arm-2009q3"
> > ### Where inside the CSL install the target libraries exist.
> > TOOLCHAIN_SYSPATH = "${TOOLCHAIN_PATH}/arm-none-linux-gnueabi"
> > ### Add tools to the PATH automatically.
> > PATH_prepend = "${TOOLCHAIN_PATH}/bin:"
> > require conf/distro/include/toolchain-external.inc
> >
> > The other way has a few less variables in local.conf but assumes PATH is
> > already updated.  Denys will reply shortly with that, I imagine.
>
> The only difference I have from Tom's setup is I expect the CSL toolchain
> being already in the PATH variable:
>
> export PATH=/full/path/to/arm-2009q1/bin:$PATH
>
> And then set the other several BitBake variables accordingly (in local.conf
> and toolchain-csl.conf):
>
> TARGET_VENDOR = "-none"
> # no need to include/require toolchain-external.inc, as TOOLCHAIN_TYPE does it
> TOOLCHAIN_TYPE = "external"
> TOOLCHAIN_BRAND = "csl"
>
> [There I also set CPPFLAGS and LDFLAGS to point to CSL directly, but it's not
> required as necessary files are staged by the recipe these days.]
>
> The "black magic" in the local.conf is basically a Python code to set
> TOOLCHAIN_PATH and TOOLCHAIN_SYSPATH based on the user's PATH:
>
> TOOLCHAIN_PATH = "${@code-to-extract-csl-path-from-PATH}"
> TOOLCHAIN_SYSPATH = ${TOOLCHAIN_PATH}/${TARGET_SYS}
>
> So, with my setup, no modifications to local.conf are required, as long as CSL
> is in your PATH. With Tom's setup, users would need to add the CSL path to
> local.conf.
>
> Hope this helps.
>
> --
> Denys

This clears things up, bit still its not sufficient to get it working.

The DISTRO seems to overwrite the TOOLCHAIN rules. And this while I
set the TOOLCHAIN variables in local.conf

And why compiles it things like coreutils-native? This are the tools
needed to compile software, like make, autoconf?
NOTE: package coreutils-native-7.2-r1: task do_setscene: Started

I have experience with compiling software on a linux machine but
OpenEmbedded is a bit different
than the other frameworks and tools.

Re: Usage of meta recipe external-toolchain-csl

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 30-04-10 21:01, Jerry Jacobs wrote:
> On Fri, Apr 30, 2010 at 7:58 PM, Denys Dmytriyenko <denis@denix.org> wrote:
>>
>> On Fri, Apr 30, 2010 at 09:53:06AM -0700, Tom Rini wrote:
>>> On Fri, 2010-04-30 at 15:34 +0200, Jerry Jacobs wrote:
>>>> Dear all,
>>>>
>>>> I'm very new to openembedded and already have a clear view how it is
>>>> structured. My embedded board mini2440 is already in git and want to create
>>>> my own distro.
>>>> The problem where I bump against is how to use the Codesourcery external
>>>> toolchain for my target. I know there is the file in recipes/meta/
>>>> external-toolchain-csl.bb
>>>> but it not clear how to use it.
>>>
>>> There's 2 ways.  One way is, in local.conf:
>>> ### CSL public ARM does <arch>-none-<os>, so fill in the -none
>>> TARGET_VENDOR = "-none"
>>> ### Bring in external-toolchain-csl.bb
>>> TOOLCHAIN_BRAND = "csl"
>>> ### Where the CSL version is installed.
>>> TOOLCHAIN_PATH = "/full/path/to/arm-2009q3"
>>> ### Where inside the CSL install the target libraries exist.
>>> TOOLCHAIN_SYSPATH = "${TOOLCHAIN_PATH}/arm-none-linux-gnueabi"
>>> ### Add tools to the PATH automatically.
>>> PATH_prepend = "${TOOLCHAIN_PATH}/bin:"
>>> require conf/distro/include/toolchain-external.inc
>>>
>>> The other way has a few less variables in local.conf but assumes PATH is
>>> already updated.  Denys will reply shortly with that, I imagine.
>>
>> The only difference I have from Tom's setup is I expect the CSL toolchain
>> being already in the PATH variable:
>>
>> export PATH=/full/path/to/arm-2009q1/bin:$PATH
>>
>> And then set the other several BitBake variables accordingly (in local.conf
>> and toolchain-csl.conf):
>>
>> TARGET_VENDOR = "-none"
>> # no need to include/require toolchain-external.inc, as TOOLCHAIN_TYPE does it
>> TOOLCHAIN_TYPE = "external"
>> TOOLCHAIN_BRAND = "csl"
>>
>> [There I also set CPPFLAGS and LDFLAGS to point to CSL directly, but it's not
>> required as necessary files are staged by the recipe these days.]
>>
>> The "black magic" in the local.conf is basically a Python code to set
>> TOOLCHAIN_PATH and TOOLCHAIN_SYSPATH based on the user's PATH:
>>
>> TOOLCHAIN_PATH = "${@code-to-extract-csl-path-from-PATH}"
>> TOOLCHAIN_SYSPATH = ${TOOLCHAIN_PATH}/${TARGET_SYS}
>>
>> So, with my setup, no modifications to local.conf are required, as long as CSL
>> is in your PATH. With Tom's setup, users would need to add the CSL path to
>> local.conf.
>>
>> Hope this helps.
>>
>> --
>> Denys
> 
> This clears things up, bit still its not sufficient to get it working.
> 
> The DISTRO seems to overwrite the TOOLCHAIN rules. And this while I
> set the TOOLCHAIN variables in local.conf

DISTRO=micro overwrites the rules, DISTRO=angstrom-2008.1 doesn't.

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFL2yzBMkyGM64RGpERAvKlAJ4745jP9G7hJDuUQuru8Xav9d4JFACglCKR
tdktixAjXdjbh9wqGrxgXc8=
=1vNF
-----END PGP SIGNATURE-----

Re: Usage of meta recipe external-toolchain-csl

[Denys Dmytriyenko]

On Fri, Apr 30, 2010 at 09:01:22PM +0200, Jerry Jacobs wrote:
> On Fri, Apr 30, 2010 at 7:58 PM, Denys Dmytriyenko <denis@denix.org> wrote:
> >
> > On Fri, Apr 30, 2010 at 09:53:06AM -0700, Tom Rini wrote:
> > > On Fri, 2010-04-30 at 15:34 +0200, Jerry Jacobs wrote:
> > > > Dear all,
> > > >
> > > > I'm very new to openembedded and already have a clear view how it is
> > > > structured. My embedded board mini2440 is already in git and want to create
> > > > my own distro.
> > > > The problem where I bump against is how to use the Codesourcery external
> > > > toolchain for my target. I know there is the file in recipes/meta/
> > > > external-toolchain-csl.bb
> > > > but it not clear how to use it.
> > >
> > > There's 2 ways.  One way is, in local.conf:
> > > ### CSL public ARM does <arch>-none-<os>, so fill in the -none
> > > TARGET_VENDOR = "-none"
> > > ### Bring in external-toolchain-csl.bb
> > > TOOLCHAIN_BRAND = "csl"
> > > ### Where the CSL version is installed.
> > > TOOLCHAIN_PATH = "/full/path/to/arm-2009q3"
> > > ### Where inside the CSL install the target libraries exist.
> > > TOOLCHAIN_SYSPATH = "${TOOLCHAIN_PATH}/arm-none-linux-gnueabi"
> > > ### Add tools to the PATH automatically.
> > > PATH_prepend = "${TOOLCHAIN_PATH}/bin:"
> > > require conf/distro/include/toolchain-external.inc
> > >
> > > The other way has a few less variables in local.conf but assumes PATH is
> > > already updated.  Denys will reply shortly with that, I imagine.
> >
> > The only difference I have from Tom's setup is I expect the CSL toolchain
> > being already in the PATH variable:
> >
> > export PATH=/full/path/to/arm-2009q1/bin:$PATH
> >
> > And then set the other several BitBake variables accordingly (in local.conf
> > and toolchain-csl.conf):
> >
> > TARGET_VENDOR = "-none"
> > # no need to include/require toolchain-external.inc, as TOOLCHAIN_TYPE does it
> > TOOLCHAIN_TYPE = "external"
> > TOOLCHAIN_BRAND = "csl"
> >
> > [There I also set CPPFLAGS and LDFLAGS to point to CSL directly, but it's not
> > required as necessary files are staged by the recipe these days.]
> >
> > The "black magic" in the local.conf is basically a Python code to set
> > TOOLCHAIN_PATH and TOOLCHAIN_SYSPATH based on the user's PATH:
> >
> > TOOLCHAIN_PATH = "${@code-to-extract-csl-path-from-PATH}"
> > TOOLCHAIN_SYSPATH = ${TOOLCHAIN_PATH}/${TARGET_SYS}
> >
> > So, with my setup, no modifications to local.conf are required, as long as CSL
> > is in your PATH. With Tom's setup, users would need to add the CSL path to
> > local.conf.
> >
> > Hope this helps.
> >
> > --
> > Denys
> 
> This clears things up, bit still its not sufficient to get it working.
> 
> The DISTRO seems to overwrite the TOOLCHAIN rules. And this while I
> set the TOOLCHAIN variables in local.conf

Can you please be more specific? Grepping conf/ shows all the assignments 
of TOOLCHAIN_* vars are weak.

> And why compiles it things like coreutils-native? This are the tools
> needed to compile software, like make, autoconf?
> NOTE: package coreutils-native-7.2-r1: task do_setscene: Started

coreutils-native provides basic commands like cp, mv, cat, ls, mkdir, mknod, 
install etc. That ensures we have sane versions of those with the expected set 
of supported command line parameters... If you know your host OS/distro 
provides the same, feel free to add coreutils-native to ASSUME_PROVIDED in 
your local.conf...

-- 
Denys