STAGING_BINDIR

STAGING_BINDIR

[Richard Purdie]

Hi,

I have hit a problem. At first I thought it was going to be a simple one
but it isn't and raises a question I've wondered about for a long time.

The problem is I had both an x86 build and an arm build in the same tree
with multimachine. In theory, this should work and it very nearly does
except I see odd compile failures in gnutls. The compile failures turn
out to be from x86 assembler. The x86 assembler is creeping in due to
corrupted include parameters from gcc. The corrupted parameters are
coming from running the "libgcrypt-config" binary. 

To make things more interesting, this only happens if you build x86,
then arm. If you build arm first, it all works...

Basically we have a race. Both the armv5te libgcrypt build and the x86
libgcrypt build install the "libgcrypt-config" binary into
STAGING_BINDIR.

from bitbake.conf we see:

STAGING_DIR = "${TMPDIR}/staging"
STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin"
STAGING_LIBDIR = "${STAGING_DIR}/${HOST_SYS}/lib"
STAGING_INCDIR = "${STAGING_DIR}/${HOST_SYS}/include"
STAGING_DATADIR = "${STAGING_DIR}/${HOST_SYS}/share"
STAGING_LOADER_DIR = "${STAGING_DIR}/${HOST_SYS}/loader"
STAGING_FIRMWARE_DIR = "${STAGING_DIR}/${HOST_SYS}/firmware"

which has always struck me as odd - BINDIR uses ${BUILD_SYS} rather than
${HOST_SYS}. I did discuss this a long time ago when I was a lot newer
to OE. multimachine wasn't as widely supported than and it was argued
that since binaries installed there were to be run on ${BUILD_SYS}
rather than ${HOST_SYS}, this was really ok. If we change this to
${HOST_SYS}, my build issues would disappear as the arm and x86 versions
will end up in different places. I think the time has come to do
something about this but its not that simple :)

The -native packages install to ${BUILD_SYS}/bin and the arch specific
ones to ${HOST_SYS}/bin. So we add both to PATH, problem solved?

What happens when we start using packaged staging and ${HOST_SYS}/bin
contains arm binaries? Our staging generally doesn't at the moment but
is likely to in future and we do not want arm binaries in PATH.

My proposal is therefore to use:

STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin/${HOST_SYS}" and have
native.bbclass override this with ${BUILD_SYS}/bin

We'd then add both ${BUILD_SYS}/bin and ${BUILD_SYS}/bin/${HOST_SYS} to
the PATH.

Does anyone see a better way to handle this?

I suspect there are going to be problems with binaries missing from
${STAGING_BINDIR} when we switch to packages staging. I just wonder why
we haven't see those problems with packaged staging yet? Is anyone
actively using/testing it?

Regards,

Richard

Re: STAGING_BINDIR

[Koen Kooi]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Richard Purdie schreef:

> What happens when we start using packaged staging and ${HOST_SYS}/bin
> contains arm binaries? Our staging generally doesn't at the moment but
> is likely to in future and we do not want arm binaries in PATH.

Ncurses seems to cause exactly that when used with packaged-staging :(

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFZv+hMkyGM64RGpERAo6IAJ9hGAK5SKLY0ojq6NKjfueoZfmsFQCgluYP
8xrnYljfuiJCTnFroZzPXpc=
=kOC0
-----END PGP SIGNATURE-----

Re: STAGING_BINDIR

[Richard Purdie]

On Fri, 2006-11-24 at 14:13 +0000, Richard Purdie wrote:
> STAGING_DIR = "${TMPDIR}/staging"
> STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin"
> STAGING_LIBDIR = "${STAGING_DIR}/${HOST_SYS}/lib"
> STAGING_INCDIR = "${STAGING_DIR}/${HOST_SYS}/include"
> STAGING_DATADIR = "${STAGING_DIR}/${HOST_SYS}/share"
> STAGING_LOADER_DIR = "${STAGING_DIR}/${HOST_SYS}/loader"
> STAGING_FIRMWARE_DIR = "${STAGING_DIR}/${HOST_SYS}/firmware"
> [...]
> My proposal is therefore to use:
> 
> STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin/${HOST_SYS}" and have
> native.bbclass override this with ${BUILD_SYS}/bin
> 
> We'd then add both ${BUILD_SYS}/bin and ${BUILD_SYS}/bin/${HOST_SYS} to
> the PATH.
> 
> Does anyone see a better way to handle this?

I've done some experimentation and had a brief discussion with mickeyl
on irc. I'm going to update this proposal with the result of this and
some real world testing.

We need three staging binary directories:

1. STAGING_BINDIR = "${STAGING_DIR}/${HOST_SYS}/bin"

HOST_SYS binaries (say arm) go here. This is *not* in PATH. We never
want to run arm binaries on x86 for example.

2. STAGING_BINDIR_CROSS = "${STAGING_DIR}/${BUILD_SYS}/bin/${HOST_SYS}"

Binaries that run on BUILD_SYS (x86) but produce results for HOST_SYS
(e.g. gcrypt-config or *-config) go here. The path might change into
somewhere in CROSS_DIR at some point once I work out more about the
structure there but that is easy if everything references the right
variable.

This is in PATH before 3. below.

3. STAGING_BINDIR_NATIVE = "${STAGING_DIR}/${BUILD_SYS}/bin"

Anything generated by the -native builds (to run on x86) goes here. This
appears in PATH, after STAGING_BINDIR_CROSS.

In native.bbclass we set STAGING_BINDIR = STAGING_BINDIR_NATIVE =
STAGING_BINDIR_CROSS. This means we have the same path twice in PATH but
apart from that several nice things happen. Native programs access
native *-config files in the NATIVE bindir for example. Also, this means
-native packages need no changes to work with the new variables! :).


How do we go about this change? I propose to start to split the current
references to STAGING_BINDIR into these three types. Initially we update
binbake.conf to say:

STAGING_BINDIR = "${STAGING_DIR}/${HOST_SYS}/bin"
STAGING_BINDIR_NATIVE = "${STAGING_DIR}/${HOST_SYS}/bin"
STAGING_BINDIR_CROSS = "${STAGING_DIR}/${HOST_SYS}/bin"

which means nothing breaks whilst we rename variables. I have a patch
which starts to do this.

As some point we then throw a switch and try out the new values I
mentioned above which will work with multimachine. This should give us a
structured transition.

This has important future implications for packaged staging. It means
installed packages can put arm binaries in STAGING_BINDIR and we won't
corrupt PATH with unexecutable binaries. It does mean we will need to
mark packages with cross binaries to install so packaged staging can
either be somehow supplemented with a cross package to install (or the
staging package is different from the installable package). If we have
references to STAGING_BINDIR_CROSS in the affected packages, this makes
that process easier.

My testing did show some interesting cases we're going to have to catch.
Taking libidl as an example, its do_stage does a "make install bindir=
${STAGING_BINDIR}". This is fine but libIDL-config-2 is a cross script
and should be in STAGING_BINDIR_CROSS and is installed into the wrong
place. My local copy has do_stage updated to read:

cp ${STAGING_BINDIR}/libIDL-config-2 ${STAGING_BINDIR}/libIDL-config-2.orig
cat ${STAGING_BINDIR}/libIDL-config-2.orig | sed -e 's:${includedir}:${STAGING_INCDIR}:' > ${STAGING_BINDIR}/libIDL-config-2

if [ "${STAGING_BINDIR}" != "${STAGING_BINDIR_CROSS}" ]; then
   mv ${STAGING_BINDIR}/libIDL-config-2 ${STAGING_BINDIR_CROSS}/libIDL-config-2
fi

(the file has to handle the native and non-native cases, hence the if)

As always, I'm open to alternative suggestions, help and assistance.
Does anyone have a problem with the changes to at least use the three
different variable names? That shouldn't break anything whilst we set
them all equal in bitbake.conf, yet lets us clean up the metadata.

Cheers,

Richard

Re: STAGING_BINDIR

[Richard Purdie]

I now have two rough patches:

http://www.rpsys.net/openzaurus/temp/stage-bindir1.patch

which just renames variables. Should be safe to apply with a change to
add the new names to bitbake.conf. I will probably apply this soon
(Monday evening?) unless anyone objects. I don't fancy having this lying
around for long.

http://www.rpsys.net/openzaurus/temp/stage-bindir2.patch

These are the more invasive changes or need special handling.

Of all the references to STAGING_BINDIR I filtered, these were the two
I didn't like the look of or understand:

These look totally wrong:
./packages/qte/qtopia-core_4.2.0.bb:    install -m 0755 ${STAGE_TEMP}/${bindir}/rcc ${STAGING_BINDIR}
./packages/qte/qtopia-core_4.2.0.bb:    install -m 0755 ${STAGE_TEMP}/${bindir}/moc ${STAGING_BINDIR}
./packages/qte/qtopia-core_4.2.0.bb:    install -m 0755 ${STAGE_TEMP}/${bindir}/uic ${STAGING_BINDIR}
./packages/qte/qtopia-core_4.2.0.bb:    install -m 0755 ${STAGE_TEMP}/${bindir}/qmake ${STAGING_BINDIR}

And I have no idea what this is doing but it looks wrong:
./packages/uqm/uqm_0.3.bb:      ./build-opts.sh ${STAGING_DIR} ${STAGING_BINDIR} ${STAGING_LIBDIR}
./packages/uqm/uqm_0.5.0.bb:    ./build-opts.sh ${STAGING_DIR} ${STAGING_BINDIR} ${STAGING_LIBDIR}

If anyone fancies trying a build from scratch with both patches applied
for their favourite packages/images, particularly e-image, feedback
would be appreciated.

Cheers,

Richard

STAGING_BINDIR

[Richard Purdie]

Hi,

I have hit a problem. At first I thought it was going to be a simple one
but it isn't and raises a question I've wondered about for a long time.

The problem is I had both an x86 build and an arm build in the same tree
with multimachine. In theory, this should work and it very nearly does
except I see odd compile failures in gnutls. The compile failures turn
out to be from x86 assembler. The x86 assembler is creeping in due to
corrupted include parameters from gcc. The corrupted parameters are
coming from running the "libgcrypt-config" binary. 

To make things more interesting, this only happens if you build x86,
then arm. If you build arm first, it all works...

Basically we have a race. Both the armv5te libgcrypt build and the x86
libgcrypt build install the "libgcrypt-config" binary into
STAGING_BINDIR.

from bitbake.conf we see:

STAGING_DIR = "${TMPDIR}/staging"
STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin"
STAGING_LIBDIR = "${STAGING_DIR}/${HOST_SYS}/lib"
STAGING_INCDIR = "${STAGING_DIR}/${HOST_SYS}/include"
STAGING_DATADIR = "${STAGING_DIR}/${HOST_SYS}/share"
STAGING_LOADER_DIR = "${STAGING_DIR}/${HOST_SYS}/loader"
STAGING_FIRMWARE_DIR = "${STAGING_DIR}/${HOST_SYS}/firmware"

which has always struck me as odd - BINDIR uses ${BUILD_SYS} rather than
${HOST_SYS}. I did discuss this a long time ago when I was a lot newer
to OE. multimachine wasn't as widely supported than and it was argued
that since binaries installed there were to be run on ${BUILD_SYS}
rather than ${HOST_SYS}, this was really ok. If we change this to
${HOST_SYS}, my build issues would disappear as the arm and x86 versions
will end up in different places. I think the time has come to do
something about this but its not that simple :)

The -native packages install to ${BUILD_SYS}/bin and the arch specific
ones to ${HOST_SYS}/bin. So we add both to PATH, problem solved?

What happens when we start using packaged staging and ${HOST_SYS}/bin
contains arm binaries? Our staging generally doesn't at the moment but
is likely to in future and we do not want arm binaries in PATH.

My proposal is therefore to use:

STAGING_BINDIR = "${STAGING_DIR}/${BUILD_SYS}/bin/${HOST_SYS}" and have
native.bbclass override this with ${BUILD_SYS}/bin

We'd then add both ${BUILD_SYS}/bin and ${BUILD_SYS}/bin/${HOST_SYS} to
the PATH.

Does anyone see a better way to handle this?

I suspect there are going to be problems with binaries missing from
${STAGING_BINDIR} when we switch to packages staging. I just wonder why
we haven't see those problems with packaged staging yet? Is anyone
actively using/testing it?

Regards,

Richard