What is glibc-intermediate for?

What is glibc-intermediate for?

[Detlef Vollmann]

Hello,

building a (cross-)toolchain isn't easy and I'm currently trying
to get my mind around it.

My understanding was always that we have the following order:
 linux-libc-headers
 glibc-initial
 gcc-cross-initial
 glibc-intermediate
 gcc-cross
 glibc

But looking at my current build, it turns out that the actual order
is different:
 linux-libc-headers
 glibc-initial
 gcc-cross-initial
 glibc-intermediate
 glibc
 gcc-cross

So, what is glibc-intermediate for (except wasting quite a lot of time)?

  Detlef

-- 
Detlef Vollmann   vollmann engineering gmbh
Linux and C++ for Embedded Systems    http://www.vollmann.ch/

Re: What is glibc-intermediate for?

[pHilipp Zabel]

On 9/9/07, Detlef Vollmann <dv@vollmann.ch> wrote:
> Hello,
>
> building a (cross-)toolchain isn't easy and I'm currently trying
> to get my mind around it.
>
> My understanding was always that we have the following order:
>  linux-libc-headers
>  glibc-initial
>  gcc-cross-initial
>  glibc-intermediate
>  gcc-cross
>  glibc
>
> But looking at my current build, it turns out that the actual order
> is different:
>  linux-libc-headers
>  glibc-initial
>  gcc-cross-initial
>  glibc-intermediate
>  glibc
>  gcc-cross
>
> So, what is glibc-intermediate for (except wasting quite a lot of time)?

glibc-initial is needed to build gcc-cross-initial, which is needed to
build glibc-intermediate, which is needed to build gcc-cross, which is
needed to build glibc. Look at the package recipes, including
glibc.inc to see what's going on.
The seemingly different build order you noticed is probably due to
bitbake's fine-grained task dependencies. Maybe glibc was unpacked
before gcc-cross, but it most certainly wasn't compiled before
gcc-cross was staged.

cheers
Philipp

Re: What is glibc-intermediate for?

[Detlef Vollmann]

pHilipp Zabel wrote:
> glibc-initial is needed to build gcc-cross-initial, which is needed to
> build glibc-intermediate, which is needed to build gcc-cross, which is
> needed to build glibc.
Yes, this is what I expected as well, but what I don't see...

> Look at the package recipes, including
> glibc.inc to see what's going on.
Yes, I looked at all (well, most) of them.
The relevant lines are in glibc.inc:
> DEPENDS = "${@['virtual/${TARGET_PREFIX}gcc-initial', 'virtual/${TARGET_PREFIX}gcc']['nptl' in '${GLIBC_ADDONS}']} linux-libc-headers"
I have to admit that this is some magic for me (Python?), but
I interpret this that the dependency on virtual/gcc is somehow dependent
on NPTL.

> PROVIDES = "virtual/libc ${@['virtual/${TARGET_PREFIX}libc-for-gcc', '']['nptl' in '${GLIBC_ADDONS}']}"
And this looks like glibc provides libc-for-gcc (again somehow dependent
on NPTL), which means it's built first.

> The seemingly different build order you noticed is probably due to
> bitbake's fine-grained task dependencies. Maybe glibc was unpacked
> before gcc-cross, but it most certainly wasn't compiled before
> gcc-cross was staged.
In my case I don't have NPTL but linuxthreads, and interestingly
glibc is compiled before gcc-cross is compiled, but only staged after
gcc-cross is compiled, which is a strange order anyway...

Right now I still have problems with my glibc package, and I'll try
to fix those and then do a clean complete bootstrap again.
Let's see what comes out of that...

  Detlef

-- 
Detlef Vollmann   vollmann engineering gmbh
Linux and C++ for Embedded Systems    http://www.vollmann.ch/

Conditional Expressions in Python

[Dr. Michael Lauer]

Guys,

I vote for rewriting our conditional expressions hack in .bb files:

DEPENDS = "${@['virtual/${TARGET_PREFIX}gcc-initial','virtual/${TARGET_PREFIX}gcc']['nptl' in '${GLIBC_ADDONS}']}"

Since Python 2.5, this can be written as:

DEPENDS = "${@'virtual/${TARGET_PREFIX}gcc' if 'nptl' in '${GLIBC_ADDONS}' else 'virtual/${TARGET_PREFIX}gcc-initial'}

which parses faster and is much more clear and concise. IIRC the
resulting bytecode is also faster since there is no need to construct,
access, and destroy temporary lists.

Regards,

:M:
-- 
Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de

PREFERRED_PROVIDER (was: Conditional Expressions in Python)

[Detlef Vollmann]

Dr. Michael Lauer wrote:
> DEPENDS = "${@['virtual/${TARGET_PREFIX}gcc-initial','virtual/${TARGET_PREFIX}gcc']['nptl' in '${GLIBC_ADDONS}']}"
> 
> Since Python 2.5, this can be written as:
> 
> DEPENDS = "${@'virtual/${TARGET_PREFIX}gcc' if 'nptl' in '${GLIBC_ADDONS}' else 'virtual/${TARGET_PREFIX}gcc-initial'}
Thanks for explaining these lines.
So, in my case (no NPTL), glibc is a provider of libc-for-gcc,
and I decided to put it as preferred provider into my local.conf
(actually it should go into my distro.conf, but that comes later...):
PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}libc-for-gcc:glibc"

But what surprise, it still tried to build glibc-intermediate.
Looking into the log file, I found the following lines:
> DEBUG: providers for virtual/powerpc-angstrom-linux-libc-for-gcc are: ['glibc', 'glibc-intermediate', 'eglibc-intermediate']
> DEBUG: update_data()
> DEBUG: selecting /oe/own/packages/glibc/glibc_2.3.6.bb as PREFERRED_VERSION 2.3.6 of package glibc (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> DEBUG: update_data()
> DEBUG: selecting /oe/rep/org.openembedded.dev/packages/glibc/glibc-intermediate_2.5.bb as PREFERRED_VERSION 2.5 of package glibc-intermediate (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> DEBUG: update_data()
> NOTE: selecting glibc-intermediate to satisfy virtual/powerpc-angstrom-linux-libc-for-gcc due to PREFERRED_PROVIDERS
This last line is simply wrong:  I've searched the whole repository
and my overlay for respective lines, and the only file with such a
preferred provider would be openprotium.conf, and that file isn't read.

My first guess was that the version number 2.5 for glibc-intermediate
is higher than the 2.3.6 for glibc and it was preferred for that.
So I put back another line into my local.conf:
PREFERRED_VERSION_glibc-intermediate = "2.3.6"

But this didn't help.  The relevant lines from the log are now:
> DEBUG: Added dependency virtual/powerpc-angstrom-linux-libc-for-gcc for /oe/own/packages/gcc/gcc-cross_4.1.1.bb
> DEBUG: providers for virtual/powerpc-angstrom-linux-libc-for-gcc are: ['glibc', 'glibc-intermediate', 'eglibc-intermediate']
> DEBUG: update_data()
> DEBUG: selecting /oe/own/packages/glibc/glibc_2.3.6.bb as PREFERRED_VERSION 2.3.6 of package glibc (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> DEBUG: update_data()
> DEBUG: selecting /oe/own/packages/glibc/glibc-intermediate_2.3.6.bb as PREFERRED_VERSION 2.3.6 of package glibc-intermediate (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> DEBUG: update_data()
> NOTE: selecting glibc-intermediate to satisfy virtual/powerpc-angstrom-linux-libc-for-gcc due to PREFERRED_PROVIDERS
> DEBUG: adding /oe/own/packages/glibc/glibc-intermediate_2.3.6.bb to satisfy virtual/powerpc-angstrom-linux-libc-for-gcc

Does anybody have any idea why bitbake still believes that it
should use glibc-intermediate for libc-for-gcc?

  Detlef

-- 
Detlef Vollmann   vollmann engineering gmbh
Linux and C++ for Embedded Systems    http://www.vollmann.ch/

Re: PREFERRED_PROVIDER (was: Conditional Expressions in Python)

[Khem Raj]

which distro are you using. angstrom-2007.1.conf could be your problem

./conf/distro/angstrom-2007.1.conf:335:PREFERRED_PROVIDER_virtual/powerpc-angstrom-linux-libc-for-gcc
       = "glibc-intermediate"

-Khem

On 9/9/07, Detlef Vollmann <dv@vollmann.ch> wrote:
> Dr. Michael Lauer wrote:
> > DEPENDS = "${@['virtual/${TARGET_PREFIX}gcc-initial','virtual/${TARGET_PREFIX}gcc']['nptl' in '${GLIBC_ADDONS}']}"
> >
> > Since Python 2.5, this can be written as:
> >
> > DEPENDS = "${@'virtual/${TARGET_PREFIX}gcc' if 'nptl' in '${GLIBC_ADDONS}' else 'virtual/${TARGET_PREFIX}gcc-initial'}
> Thanks for explaining these lines.
> So, in my case (no NPTL), glibc is a provider of libc-for-gcc,
> and I decided to put it as preferred provider into my local.conf
> (actually it should go into my distro.conf, but that comes later...):
> PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}libc-for-gcc:glibc"
>
> But what surprise, it still tried to build glibc-intermediate.
> Looking into the log file, I found the following lines:
> > DEBUG: providers for virtual/powerpc-angstrom-linux-libc-for-gcc are: ['glibc', 'glibc-intermediate', 'eglibc-intermediate']
> > DEBUG: update_data()
> > DEBUG: selecting /oe/own/packages/glibc/glibc_2.3.6.bb as PREFERRED_VERSION 2.3.6 of package glibc (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> > DEBUG: update_data()
> > DEBUG: selecting /oe/rep/org.openembedded.dev/packages/glibc/glibc-intermediate_2.5.bb as PREFERRED_VERSION 2.5 of package glibc-intermediate (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> > DEBUG: update_data()
> > NOTE: selecting glibc-intermediate to satisfy virtual/powerpc-angstrom-linux-libc-for-gcc due to PREFERRED_PROVIDERS
> This last line is simply wrong:  I've searched the whole repository
> and my overlay for respective lines, and the only file with such a
> preferred provider would be openprotium.conf, and that file isn't read.
>
> My first guess was that the version number 2.5 for glibc-intermediate
> is higher than the 2.3.6 for glibc and it was preferred for that.
> So I put back another line into my local.conf:
> PREFERRED_VERSION_glibc-intermediate = "2.3.6"
>
> But this didn't help.  The relevant lines from the log are now:
> > DEBUG: Added dependency virtual/powerpc-angstrom-linux-libc-for-gcc for /oe/own/packages/gcc/gcc-cross_4.1.1.bb
> > DEBUG: providers for virtual/powerpc-angstrom-linux-libc-for-gcc are: ['glibc', 'glibc-intermediate', 'eglibc-intermediate']
> > DEBUG: update_data()
> > DEBUG: selecting /oe/own/packages/glibc/glibc_2.3.6.bb as PREFERRED_VERSION 2.3.6 of package glibc (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> > DEBUG: update_data()
> > DEBUG: selecting /oe/own/packages/glibc/glibc-intermediate_2.3.6.bb as PREFERRED_VERSION 2.3.6 of package glibc-intermediate (for item virtual/powerpc-angstrom-linux-libc-for-gcc)
> > DEBUG: update_data()
> > NOTE: selecting glibc-intermediate to satisfy virtual/powerpc-angstrom-linux-libc-for-gcc due to PREFERRED_PROVIDERS
> > DEBUG: adding /oe/own/packages/glibc/glibc-intermediate_2.3.6.bb to satisfy virtual/powerpc-angstrom-linux-libc-for-gcc
>
> Does anybody have any idea why bitbake still believes that it
> should use glibc-intermediate for libc-for-gcc?
>
>   Detlef
>
> --
> Detlef Vollmann   vollmann engineering gmbh
> Linux and C++ for Embedded Systems    http://www.vollmann.ch/
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>

Re: Conditional Expressions in Python

[Khem Raj]

yes it looks more readable too.

On 9/9/07, Dr. Michael Lauer <mickey@vanille-media.de> wrote:
> Guys,
>
> I vote for rewriting our conditional expressions hack in .bb files:
>
> DEPENDS = "${@['virtual/${TARGET_PREFIX}gcc-initial','virtual/${TARGET_PREFIX}gcc']['nptl' in '${GLIBC_ADDONS}']}"
>
> Since Python 2.5, this can be written as:
>
> DEPENDS = "${@'virtual/${TARGET_PREFIX}gcc' if 'nptl' in '${GLIBC_ADDONS}' else 'virtual/${TARGET_PREFIX}gcc-initial'}
>
> which parses faster and is much more clear and concise. IIRC the
> resulting bytecode is also faster since there is no need to construct,
> access, and destroy temporary lists.
>
> Regards,
>
> :M:
> --
> Michael 'Mickey' Lauer | IT-Freelancer | http://www.vanille-media.de
>
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>

Re: PREFERRED_PROVIDER (was: Conditional Expressions in Python)

[Detlef Vollmann]

Khem Raj wrote:
> 
> which distro are you using. angstrom-2007.1.conf could be your problem
> 
> ./conf/distro/angstrom-2007.1.conf:335:PREFERRED_PROVIDER_virtual/powerpc-angstrom-linux-libc-for-gcc
>        = "glibc-intermediate"
Oops, sorry.
You're completely correct, it's my fault:
I searched for 'PREFERRED_PROVIDERS.*libc-for-gcc' and didn't think
of the PREFERRED_PROVIDER_.*.
Sorry again.

But now the next question:
What's the priority of the PREFERRED_PROVIDER* settings?
PREFERRED_PROVIDERS is one long list with potentially lots of
entries, possibly conflicting.
What gets priority?  The first entry for a specific target or the
last entry?

And it seems that a specific PREFERRED_PROVIDER_.* setting gets
priority over the PREFERRED_PROVIDERS list.  Is that always correct?

And as the distro.conf is read after the local.conf, there's no
way to override a setting from distro.conf in local.conf, correct?

For me that's not really a problem, as I have my own distro.conf
that includes (requires) angstrom.conf, but I think local.conf
is always more specific for the user (hance its name) than distro.conf,
so it would be nice if distro settings could be overridden in
local.conf.

  Detlef

-- 
Detlef Vollmann   vollmann engineering gmbh
Linux and C++ for Embedded Systems    http://www.vollmann.ch/