Machine specific sysroot issue

Machine specific sysroot issue

[Paul Eggleton]

Hi all,

I've brought over the Opie recipes from OE in preparation for creating a 
separate layer, and after making a few changes everything compiles OK; however 
I'm finding that the change to machine-specific sysroots is causing some 
problems.

libopie2, a non-machine specific package, among other things installs a file 
called include.pro into ${OPIEDIR}, which is configured to be 
${STAGING_DIR_HOST}. In OE-classic ${STAGING_DIR_HOST} is not machine-specific, 
so if you build libopie2 for machine A then build some other package that 
needs include.pro for machine B, then everything is fine. In the OE-core case 
with machine-specific sysroots it's looking for the file in the sysroot for the 
current machine, which if you've been building for several different machines 
may not be where it has actually been installed.

What's the correct way to fix this? Is libopie2 now installing this file in the 
wrong place?

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Machine specific sysroot issue

[Khem Raj]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/14/2011 12:50 PM, Paul Eggleton wrote:
> Hi all,
> 
> I've brought over the Opie recipes from OE in preparation for
> creating a separate layer, and after making a few changes everything
> compiles OK; however I'm finding that the change to machine-specific
> sysroots is causing some problems.
> 
> libopie2, a non-machine specific package, among other things installs
> a file called include.pro into ${OPIEDIR}, which is configured to be
>  ${STAGING_DIR_HOST}. In OE-classic ${STAGING_DIR_HOST} is not
> machine-specific, so if you build libopie2 for machine A then build
> some other package that needs include.pro for machine B, then
> everything is fine. In the OE-core case with machine-specific
> sysroots it's looking for the file in the sysroot for the current
> machine, which if you've been building for several different machines
>  may not be where it has actually been installed.
> 
> What's the correct way to fix this? Is libopie2 now installing this
> file in the wrong place?

I think with shared state it should be able to reuse what was built
before. It does not share the target sysroot like oe.dev
as long as the .pro file is part of a package it should be able to stage
it into machine sysroot and use it without causing a recompile

> 
> Cheers, Paul
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk5IRrYACgkQuwUzVZGdMxSvEQCfblKkiL3uZLnwyjI+Nhymj9zf
zRUAn1BjKeIcTkHXvTFzAr2M5G0UK8qd
=6zRI
-----END PGP SIGNATURE-----

Re: Machine specific sysroot issue

[Richard Purdie]

On Sun, 2011-08-14 at 20:50 +0100, Paul Eggleton wrote:
> I've brought over the Opie recipes from OE in preparation for creating a 
> separate layer, and after making a few changes everything compiles OK; however 
> I'm finding that the change to machine-specific sysroots is causing some 
> problems.
> 
> libopie2, a non-machine specific package, among other things installs a file 
> called include.pro into ${OPIEDIR}, which is configured to be 
> ${STAGING_DIR_HOST}. In OE-classic ${STAGING_DIR_HOST} is not machine-specific, 
> so if you build libopie2 for machine A then build some other package that 
> needs include.pro for machine B, then everything is fine. In the OE-core case 
> with machine-specific sysroots it's looking for the file in the sysroot for the 
> current machine, which if you've been building for several different machines 
> may not be where it has actually been installed.
> 
> What's the correct way to fix this? Is libopie2 now installing this file in the 
> wrong place?

It sounds like its being installed to the right location but that opie
isn't looking for it in the right place later.

How does opie know where to look for the file?

Cheers,

Richard

Re: Machine specific sysroot issue

[Paul Eggleton]

On Monday 15 August 2011 15:15:01 Richard Purdie wrote:
> It sounds like its being installed to the right location but that opie
> isn't looking for it in the right place later.
> 
> How does opie know where to look for the file?

Same way it gets installed, it looks for it in ${OPIEDIR}. It's just that the 
second time around if the machine is not the same, ${OPIEDIR} is not the same 
either.

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Machine specific sysroot issue

[Koen Kooi]

Op 15 aug. 2011, om 16:23 heeft Paul Eggleton het volgende geschreven:

> On Monday 15 August 2011 15:15:01 Richard Purdie wrote:
>> It sounds like its being installed to the right location but that opie
>> isn't looking for it in the right place later.
>> 
>> How does opie know where to look for the file?
> 
> Same way it gets installed, it looks for it in ${OPIEDIR}. It's just that the 
> second time around if the machine is not the same, ${OPIEDIR} is not the same 
> either.

That sounds like an expanded version is getting stored somewhere where it shouldn't

Re: Machine specific sysroot issue

[Paul Eggleton]

On Monday 15 August 2011 15:30:53 Koen Kooi wrote:
> > Same way it gets installed, it looks for it in ${OPIEDIR}. It's just that
> > the second time around if the machine is not the same, ${OPIEDIR} is not
> > the same either.
> 
> That sounds like an expanded version is getting stored somewhere where it
> shouldn't

How could that be the case? If it stayed the same I could understand it...

It's pretty straightforward; OPIEDIR is set to STAGING_DIR_HOST, but 
STAGING_DIR_HOST is now ${STAGING_DIR}/${MACHINE},  thus if you change the 
machine, STAGING_DIR_HOST also changes. In OE-dev (and Poky prior to machine-
specific sysroots) STAGING_DIR_HOST was ${STAGING_DIR}/${BASEPKG_HOST_SYS}, 
thus there was no problem.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Machine specific sysroot issue

[Koen Kooi]

Op 15 aug. 2011, om 16:42 heeft Paul Eggleton het volgende geschreven:

> On Monday 15 August 2011 15:30:53 Koen Kooi wrote:
>>> Same way it gets installed, it looks for it in ${OPIEDIR}. It's just that
>>> the second time around if the machine is not the same, ${OPIEDIR} is not
>>> the same either.
>> 
>> That sounds like an expanded version is getting stored somewhere where it
>> shouldn't
> 
> How could that be the case? If it stayed the same I could understand it...
> 
> It's pretty straightforward; OPIEDIR is set to STAGING_DIR_HOST, but 
> STAGING_DIR_HOST is now ${STAGING_DIR}/${MACHINE},  thus if you change the 
> machine, STAGING_DIR_HOST also changes. In OE-dev (and Poky prior to machine-
> specific sysroots) STAGING_DIR_HOST was ${STAGING_DIR}/${BASEPKG_HOST_SYS}, 
> thus there was no problem.

if a recipe (or upstream code) does something like

echo $OPIEDIR > foo.config

And after you stage foo.config it will point to the old sysroot when changing machine. In other words: something in opie isn't relocatable.

Re: Machine specific sysroot issue

[Phil Blundell]

On Sun, 2011-08-14 at 20:50 +0100, Paul Eggleton wrote:
> libopie2, a non-machine specific package, among other things installs a file 
> called include.pro into ${OPIEDIR}, which is configured to be 
> ${STAGING_DIR_HOST}. In OE-classic ${STAGING_DIR_HOST} is not machine-specific, 
> so if you build libopie2 for machine A then build some other package that 
> needs include.pro for machine B, then everything is fine. In the OE-core case 
> with machine-specific sysroots it's looking for the file in the sysroot for the 
> current machine, which if you've been building for several different machines 
> may not be where it has actually been installed.

I can't immediately think why this should be behaving differently to any
other library (eglibc for example).  It's quite normal for those
packages to be architecture-specific and not machine-specific.

What I think is meant to happen when you change MACHINE is that
populate_sysroot will rerun (because ${MACHINE} is factored into the
stamp for that task) but it will use all the previously cached build
artifacts from sstate and hence you won't end up having to rebuild
everything.  It sounds as if that isn't happening in this case for some
reason, though I can't immediately think what libopie2 might be doing to
defeat it.

Is it just include.pro that goes wrong, or does the whole of libopie2 go
missing from the sysroot when you change MACHINE?

p.

Re: Machine specific sysroot issue

[Paul Eggleton]

On Monday 15 August 2011 16:03:44 Phil Blundell wrote:
> I can't immediately think why this should be behaving differently to any
> other library (eglibc for example).  It's quite normal for those
> packages to be architecture-specific and not machine-specific.
> 
> What I think is meant to happen when you change MACHINE is that
> populate_sysroot will rerun (because ${MACHINE} is factored into the
> stamp for that task) but it will use all the previously cached build
> artifacts from sstate and hence you won't end up having to rebuild
> everything.  It sounds as if that isn't happening in this case for some
> reason, though I can't immediately think what libopie2 might be doing to
> defeat it.
> 
> Is it just include.pro that goes wrong, or does the whole of libopie2 go
> missing from the sysroot when you change MACHINE?

Actually it's just include.pro - everything else gets copied to the sysroot 
for the new machine just fine.

I've sort of figured this out. include.pro is getting installed to the sysroot  
and not ${D} in do_install (and it's the only file handled this way). I'm 
guessing that because of this it never gets picked up by sstate.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Machine specific sysroot issue

[Khem Raj]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> I've sort of figured this out. include.pro is getting installed to
> the sysroot and not ${D} in do_install (and it's the only file
> handled this way). I'm guessing that because of this it never gets
> picked up by sstate.

yes seems the cause to me

> 
> Cheers, Paul
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk5JQHgACgkQuwUzVZGdMxTEQACggmD5qFdYqsfZ0GhbMVE9LMYL
e2AAnAz5FMDMA5d9F+msjxDulXuQpAY5
=92OM
-----END PGP SIGNATURE-----

Re: Machine specific sysroot issue

[Phil Blundell]

On Mon, 2011-08-15 at 16:49 +0100, Paul Eggleton wrote:
> Actually it's just include.pro - everything else gets copied to the sysroot 
> for the new machine just fine.
> 
> I've sort of figured this out. include.pro is getting installed to the sysroot  
> and not ${D} in do_install (and it's the only file handled this way). 

Oh right.  Yeah, libopie2 certainly shouldn't be doing that.  Any sort
of direct poking at the sysroot is basically a bad idea nowadays, and if
include.pro is an important file then presumably you want it in the -dev
package anyway.

p.

Re: Machine specific sysroot issue

[Richard Purdie]

On Mon, 2011-08-15 at 16:49 +0100, Paul Eggleton wrote:
> On Monday 15 August 2011 16:03:44 Phil Blundell wrote:
> > I can't immediately think why this should be behaving differently to any
> > other library (eglibc for example).  It's quite normal for those
> > packages to be architecture-specific and not machine-specific.
> > 
> > What I think is meant to happen when you change MACHINE is that
> > populate_sysroot will rerun (because ${MACHINE} is factored into the
> > stamp for that task) but it will use all the previously cached build
> > artifacts from sstate and hence you won't end up having to rebuild
> > everything.  It sounds as if that isn't happening in this case for some
> > reason, though I can't immediately think what libopie2 might be doing to
> > defeat it.
> > 
> > Is it just include.pro that goes wrong, or does the whole of libopie2 go
> > missing from the sysroot when you change MACHINE?
> 
> Actually it's just include.pro - everything else gets copied to the sysroot 
> for the new machine just fine.
> 
> I've sort of figured this out. include.pro is getting installed to the sysroot  
> and not ${D} in do_install (and it's the only file handled this way). I'm 
> guessing that because of this it never gets picked up by sstate.

This is the problem. You want to install the file into ${D} and then all
should be well...

Cheers,

Richard