* [Buildroot] [autobuild.buildroot.net] Build results for 2014-03-01
@ 2014-03-02 7:30 Thomas Petazzoni
2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni
0 siblings, 1 reply; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-02 7:30 UTC (permalink / raw)
To: buildroot
Build statistics for 2014-03-01
===============================
success : 42
failures : 48
timeouts : 0
TOTAL : 90
Classification of failures by reason
====================================
host-python-setuptools-2.1.2 | 16
php-gnupg-1.3.3 | 5
lvm2-2.02.103 | 4
parted-3.1 | 4
python-pyasn-0.17 | 3
bash-4.3 | 3
host-scons-2.3.0 | 2
crda-1.1.3 | 2
mplayer-1.1.1 | 1
libnftnl-1.0.0 | 1
pciutils-3.2.1 | 1
cairo-1.12.10 | 1
host-python3-3.4.0rc1 | 1
zeromq-3.2.4 | 1
fbgrab-1.2 | 1
libmicrohttpd-0.9.34 | 1
libsigsegv-2.10 | 1
Detail of failures
===================
i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/44a962623be757143a3d6d37fc6cd14964094f79/
i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/9c16032f9630e09e3709a69ced6716de725c153d/
x86_64 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/6902eee5e39d9fd79b1e78ccd54382299f1b25ea/
bfin | cairo-1.12.10 | NOK | http://autobuild.buildroot.net/results/c421cde66d8d2d9bdf9ecbe531fb15b86f85d909/
powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/87bfdc235ee441153c36489a0a1ec97266899d5a/
powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/d442f571f14758a4bdf7d694cecd5a34116be40a/
bfin | fbgrab-1.2 | NOK | http://autobuild.buildroot.net/results/b87b9397f4bf22579e7fca9763c9f099bb63bd8f/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/f22ffbf62d4bc8a1f910269e06636181c324644f/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5e6f3c45dd480ef2bed8515d6a48579642e28274/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/aa6334d89d47d91fdb696d9248036ec27b4b752a/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/95d0e3656d539258bffa9ca3637d98bc48876a40/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/177f027edd981ed2ec081d93a15993e51f408620/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/87909291c77a45b1fbe99f641ea8f7d4b8b50e13/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/4952e3a5bd6bac437b8b382dfa18f52eb17505da/
arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/91df531bb3bb57f9b562ed358d6e4a7a970d5e8a/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5deb94ab960aa2f4428f7e9d2d5315ba79116ce1/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/11725dbff02b07096552a8c0f85d90d969a31fc9/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/1b4713983e9148408081ee432717cb9f6afa5ac9/
arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b8a4c5dce70d6215945293cd6fd28282c3896a78/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/de0d965f2e1d6e70b234c2708d558273291f3a5f/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/05c721a141e698d0df49b2867a218d31e7acb311/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b01f64c75e71fb4e5282f690ee76a2a49842e889/
i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
arm | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/
powerpc | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/
i686 | libmicrohttpd-0.9.34 | NOK | http://autobuild.buildroot.net/results/edb1b5ee1ffaa6a495b4d7c2976f618c8331b5d2/
powerpc | libnftnl-1.0.0 | NOK | http://autobuild.buildroot.net/results/c13ef3757b90838549d6f396155a20890d163ff9/
nios2 | libsigsegv-2.10 | NOK | http://autobuild.buildroot.net/results/82dac744ce01c9ebb8c769aea02bf29bc7494aa6/
arm | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/dd5ceda4e3778a763d26b8a81a08391f13f02307/
powerpc | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/d6f4d388c62ec2137560aabbff46c4a30c584e18/
mips | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/bc9b84964667fd37155df97c35252d6b73be724f/
microblaze | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/affded5ab5590e6f7313c59f511b81c1bb645c92/
arm | mplayer-1.1.1 | NOK | http://autobuild.buildroot.net/results/ee409788bbf44434a53c14ede7928367ba8e1c85/
arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/87ee883d745fafd88fa446244cfe69659ff06f3c/
arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/1e90929bfaf617381d490dc0b99f333968624ce7/
arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/c4cb972758a58c4b62e600c14936af4fd559295b/
i686 | parted-3.1 | NOK | http://autobuild.buildroot.net/results/4b5f5dc38a08f622992ea22fd58c1d82b80890c8/
bfin | pciutils-3.2.1 | NOK | http://autobuild.buildroot.net/results/f5c6d4453d546001423244476adf2c6f40411e2c/
i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/e3b2d68b22224fcda95174bd64803a58aae1606a/
powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/fe26034ee29c7334d09cba8a80ae5bfd2787a552/
i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/df9f7c7c30d9cdba65b4f5c6d46c8aa91de4179a/
powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/efbce29a2fa1d8561c10613d2145b9b382ddc83c/
powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/757fb0b12af12de4f128d9c503d45b20776c2313/
arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/3c968459f6cc020a3c6bc77f49245d69ed92df20/
arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/e009a9a5f8a311b5fcb41212d517a2d52ff2db9d/
arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/c11893d19f5614236f1d30a5fca781b94d2f9f80/
bfin | zeromq-3.2.4 | NOK | http://autobuild.buildroot.net/results/59182657d03ca48dc46c96d4ff14667324f4e4f6/
--
http://autobuild.buildroot.net
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-02 7:30 [Buildroot] [autobuild.buildroot.net] Build results for 2014-03-01 Thomas Petazzoni
@ 2014-03-02 8:57 ` Thomas Petazzoni
2014-03-02 9:50 ` Samuel Martin
2014-03-04 23:18 ` Arnout Vandecappelle
0 siblings, 2 replies; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-02 8:57 UTC (permalink / raw)
To: buildroot
Hello all,
On Sun, 2 Mar 2014 08:30:07 +0100 (CET), Thomas Petazzoni wrote:
> i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/44a962623be757143a3d6d37fc6cd14964094f79/
> i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/9c16032f9630e09e3709a69ced6716de725c153d/
> x86_64 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/6902eee5e39d9fd79b1e78ccd54382299f1b25ea/F
Fixed by
http://git.buildroot.net/buildroot/commit/?id=7a95111e48f5fe387f592463effa233834c6fda1
> bfin | cairo-1.12.10 | NOK | http://autobuild.buildroot.net/results/c421cde66d8d2d9bdf9ecbe531fb15b86f85d909/
To be fixed.
> powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/87bfdc235ee441153c36489a0a1ec97266899d5a/
> powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/d442f571f14758a4bdf7d694cecd5a34116be40a/
Issue when host-python3 is built before crda. The easiest solution is
probably to make the crda Python script Python 3 compatible and submit
the patch upstream.
> bfin | fbgrab-1.2 | NOK | http://autobuild.buildroot.net/results/b87b9397f4bf22579e7fca9763c9f099bb63bd8f/
Missing -l flags?
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/f22ffbf62d4bc8a1f910269e06636181c324644f/
> powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5e6f3c45dd480ef2bed8515d6a48579642e28274/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/aa6334d89d47d91fdb696d9248036ec27b4b752a/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/95d0e3656d539258bffa9ca3637d98bc48876a40/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/177f027edd981ed2ec081d93a15993e51f408620/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/87909291c77a45b1fbe99f641ea8f7d4b8b50e13/
> powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/4952e3a5bd6bac437b8b382dfa18f52eb17505da/
> arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/91df531bb3bb57f9b562ed358d6e4a7a970d5e8a/
> powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5deb94ab960aa2f4428f7e9d2d5315ba79116ce1/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/11725dbff02b07096552a8c0f85d90d969a31fc9/
> powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/1b4713983e9148408081ee432717cb9f6afa5ac9/
> arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b8a4c5dce70d6215945293cd6fd28282c3896a78/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/de0d965f2e1d6e70b234c2708d558273291f3a5f/
> powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/05c721a141e698d0df49b2867a218d31e7acb311/
> i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b01f64c75e71fb4e5282f690ee76a2a49842e889/
Fixed by
http://git.buildroot.net/buildroot/commit/?id=bec21dcfa7042d1f7065cf467c4f9c777f0fd072
> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
Parallel installation issue with Python 3. Either we should make Python
3 entirely use MAKE1, or fix the problem. I have analyzed the problem,
and it is caused by the parallel installation of libpython into the
HOST_DIR and the execution of ./python in the Python source directory.
> arm | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/
> powerpc | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/
Scons is not compatible with Python 3. But host-scons is not a target
package, it's a host package, so we cannot add a Python 3 related
dependency to it. So the two solutions that I see are:
* Change our host-python and host-python3 packages so that they are
not exclusive from each other, and then find a way for host-scons to
express the fact that it really needs host-python, and not the
host-python version matching the selected target python version.
Note that achieving this might be difficult.
* Make the three target packages that actually need host-scons
"depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next
version of SCons is supposed to support Python 3, so we could
consider it as a temporary solution.
Suggestions?
> i686 | libmicrohttpd-0.9.34 | NOK | http://autobuild.buildroot.net/results/edb1b5ee1ffaa6a495b4d7c2976f618c8331b5d2/
Fixed by
http://git.buildroot.net/buildroot/commit/?id=25c438bce1c02775f2c23d8878ab71c3022b0c95
> powerpc | libnftnl-1.0.0 | NOK | http://autobuild.buildroot.net/results/c13ef3757b90838549d6f396155a20890d163ff9/
Probably needs some kernel headers dependency. To be checked in detail.
Any taker?
> nios2 | libsigsegv-2.10 | NOK | http://autobuild.buildroot.net/results/82dac744ce01c9ebb8c769aea02bf29bc7494aa6/
Needs a small patch to support NIOS 2. The only information needed is
to know whether the stack grows upward or downward on the architecture.
> arm | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/dd5ceda4e3778a763d26b8a81a08391f13f02307/
> powerpc | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/d6f4d388c62ec2137560aabbff46c4a30c584e18/
> mips | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/bc9b84964667fd37155df97c35252d6b73be724f/
> microblaze | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/affded5ab5590e6f7313c59f511b81c1bb645c92/
Fixed by
http://git.buildroot.net/buildroot/commit/?id=fb36a82a745e93afc92bb5aff29daea8daecb6d3.
> arm | mplayer-1.1.1 | NOK | http://autobuild.buildroot.net/results/ee409788bbf44434a53c14ede7928367ba8e1c85/
ARM floating point issue.
> arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/87ee883d745fafd88fa446244cfe69659ff06f3c/
> arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/1e90929bfaf617381d490dc0b99f333968624ce7/
> arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/c4cb972758a58c4b62e600c14936af4fd559295b/
> i686 | parted-3.1 | NOK | http://autobuild.buildroot.net/results/4b5f5dc38a08f622992ea22fd58c1d82b80890c8/
Fixed by
http://git.buildroot.net/buildroot/commit/?id=8d89fd2b7c09af47f2d1b81cfa4de45a421d8a83.
> bfin | pciutils-3.2.1 | NOK | http://autobuild.buildroot.net/results/f5c6d4453d546001423244476adf2c6f40411e2c/
Blackfin specific issue related to the usage of _ as prefix for
assembly symbols.
> i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/e3b2d68b22224fcda95174bd64803a58aae1606a/
> powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/fe26034ee29c7334d09cba8a80ae5bfd2787a552/
> i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/df9f7c7c30d9cdba65b4f5c6d46c8aa91de4179a/
> powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/efbce29a2fa1d8561c10613d2145b9b382ddc83c/
> powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/757fb0b12af12de4f128d9c503d45b20776c2313/
Fixed by
http://git.buildroot.net/buildroot/commit/?id=1188fcf1667ac316ea9df024b9346415a05b90db.
> arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/3c968459f6cc020a3c6bc77f49245d69ed92df20/
> arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/e009a9a5f8a311b5fcb41212d517a2d52ff2db9d/
> arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/c11893d19f5614236f1d30a5fca781b94d2f9f80/
Fixed by
http://git.buildroot.net/buildroot/commit/?id=e025e49100565617715e0de986ba670532098b14.
> bfin | zeromq-3.2.4 | NOK | http://autobuild.buildroot.net/results/59182657d03ca48dc46c96d4ff14667324f4e4f6/
Compiler does not provide the right compiler intrinsics. We should
check if more recent versions of gcc do not provide the necessary
intrinsics.
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni
@ 2014-03-02 9:50 ` Samuel Martin
2014-03-02 10:08 ` Thomas Petazzoni
2014-03-04 23:18 ` Arnout Vandecappelle
1 sibling, 1 reply; 10+ messages in thread
From: Samuel Martin @ 2014-03-02 9:50 UTC (permalink / raw)
To: buildroot
Thomas, all,
On Sun, Mar 2, 2014 at 9:57 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
[...]
>> arm | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/
>> powerpc | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/
>
> Scons is not compatible with Python 3. But host-scons is not a target
> package, it's a host package, so we cannot add a Python 3 related
> dependency to it. So the two solutions that I see are:
>
> * Change our host-python and host-python3 packages so that they are
> not exclusive from each other, and then find a way for host-scons to
> express the fact that it really needs host-python, and not the
> host-python version matching the selected target python version.
> Note that achieving this might be difficult.
host-python and host-python3 are exclusive at the kconfig level, not
the make target one, right or am I missing something?
AFAICS the only overlapping thing between the 2 host package is the
$(HOST_DIR)/usr/bin/python symlink.
So, we could imagine building host-python and host-python3 in the same build.
In such a case, the tricky point becomes the
$(HOST_DIR)/usr/bin/python symlink management:
- host-python should not install the symlink if python3 is enabled;
- host-python3 should deactivate the symlink installation when python
is enabled.
Then, the SCONS variable should explicity refer to
"$(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons"
So, we could be able to build host-python (for host-scons) even if
python3 is enabled.
host-python is the only package providing python2 stuff we would allow
to build when python3 is enabled.
For other packages, it should not change anything since the
$(HOST_DIR)/usr/bin/python symlink still reflects the configuration
(ie. its points to python3 if python3 is enabled, to python if
python(2) is enabled).
For example, crda needs host-python-m2crypto. Since there is no change
in the buildroot configuration (ie. the BR2_PACKAGE_PYTHON and
BR2_PACKAGE_PYTHON3 will still have the right value) and
$(HOST_DIR)/usr/bin/python still reflect this configuration,
host-python-m2crypto should still be built against the right python
version.
Of course, if we choose this way, it requires tests.
Comments?
>
> * Make the three target packages that actually need host-scons
> "depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next
> version of SCons is supposed to support Python 3, so we could
> consider it as a temporary solution.
I've just checked out scons repository and it cannot be bootstraped
with python3 :'(
>
> Suggestions?
See above ;)
Regards,
--
Samuel
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-02 9:50 ` Samuel Martin
@ 2014-03-02 10:08 ` Thomas Petazzoni
0 siblings, 0 replies; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-02 10:08 UTC (permalink / raw)
To: buildroot
Dear Samuel Martin,
On Sun, 2 Mar 2014 10:50:41 +0100, Samuel Martin wrote:
> host-python and host-python3 are exclusive at the kconfig level
Hum? There are no Kconfig options for host-python and host-python3.
It's the target Pythons that are mutually exclusive at the Kconfig
level. And since each target Python depends on the same version of
host-pythonX, this is what brings host-python3 when target python3 is
selected.
> the make target one, right or am I missing something?
> AFAICS the only overlapping thing between the 2 host package is the
> $(HOST_DIR)/usr/bin/python symlink.
>
> So, we could imagine building host-python and host-python3 in the same build.
> In such a case, the tricky point becomes the
> $(HOST_DIR)/usr/bin/python symlink management:
> - host-python should not install the symlink if python3 is enabled;
> - host-python3 should deactivate the symlink installation when python
> is enabled.
> Then, the SCONS variable should explicity refer to
> "$(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons"
> So, we could be able to build host-python (for host-scons) even if
> python3 is enabled.
You're missing something: the host-scons package is using the
host-python-package infrastructure. And this infrastructure uses the
host Python version of the selected target Python. This is where the
core of the problem is. We would need a way to tell the
host-python-package infrastructure to make a package depend on
host-python instead of host-python3, even if the selected target Python
is Python 3.
> > * Make the three target packages that actually need host-scons
> > "depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next
> > version of SCons is supposed to support Python 3, so we could
> > consider it as a temporary solution.
>
> I've just checked out scons repository and it cannot be bootstraped
> with python3 :'(
See the Scons homepage:
"""
SCons release 2.3.0 now available from the download page at
SourceForge. This release adds new features, and fixes and improves a
number of issues. This will be the last release to support Python
versions earlier than 2.7, as we begin to move toward supporting Python
3.
"""
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni
2014-03-02 9:50 ` Samuel Martin
@ 2014-03-04 23:18 ` Arnout Vandecappelle
2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle
2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni
1 sibling, 2 replies; 10+ messages in thread
From: Arnout Vandecappelle @ 2014-03-04 23:18 UTC (permalink / raw)
To: buildroot
On 02/03/14 09:57, Thomas Petazzoni wrote:
>> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
> Parallel installation issue with Python 3. Either we should make Python
> 3 entirely use MAKE1, or fix the problem. I have analyzed the problem,
> and it is caused by the parallel installation of libpython into the
> HOST_DIR and the execution of ./python in the Python source directory.
Is that because we export an LD_LIBRARY_PATH that points to the freshly
installed libpython? Because I would expect that ./python uses the build
directory's libpython, not the installed one...
Oh, but it actually does get called with the correct LD_LIBRARY_PATH:
LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib:
./python ...
The issue is that the executable contains an RPATH pointing to the
install location of libpython.so (in $(HOST_DIR)). When the file doesn't
exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the
build directory. When the file does exist, it can link against it -
except when it hasn't been fully written yet.
The solution would be if the executable would use RUNPATH instead of
RPATH - which according to e.g. [1] is what everybody should do (and I
tend to agree). IOW, the option --enable-new-dtags should be passed to
the linker ("new" is not really true BTW, the option has existed for
since more than 10 years). When we would add a toolchain wrapper for ld,
we could add it there.
But for the time being, I'll post a patch for python3 (and python, which
suffers from the same problem) that adds --enable-new-dtags there.
Regards,
Arnout
[1] http://blog.tremily.us/posts/rpath/
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython
2014-03-04 23:18 ` Arnout Vandecappelle
@ 2014-03-04 23:29 ` Arnout Vandecappelle
2014-03-05 18:46 ` Thomas Petazzoni
2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni
1 sibling, 1 reply; 10+ messages in thread
From: Arnout Vandecappelle @ 2014-03-04 23:29 UTC (permalink / raw)
To: buildroot
During installation, host-python and host-python3 run the freshly built
python executable. This is done with a proper LD_LIBRARY_PATH to make
sure it picks up the libpython in the build directory. However, the
python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib.
Therefore, if libpython exists there, it will be used instead.
If the install step is run in parallel, it is possible that libpython
is already partially copied to $(HOST_DIR)/usr/lib when python is run.
This gives an error like:
python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short
The fix is simple: use RUNPATH instead of RPATH, which allows
LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the
build directory is always used. RUNPATH is enabled by passing
--enable-new-dtags to the linker.
Fixes e.g.
http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
package/python/python.mk | 6 ++++++
package/python3/python3.mk | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/package/python/python.mk b/package/python/python.mk
index 94765cb..216448e 100644
--- a/package/python/python.mk
+++ b/package/python/python.mk
@@ -34,6 +34,12 @@ HOST_PYTHON_CONF_OPT += \
--disable-ssl \
--disable-pyo-build
+# Make sure that LD_LIBRARY_PATH overrides -rpath.
+# This is needed because libpython may be installed at the same time that
+# python is called.
+HOST_PYTHON_CONF_ENV += \
+ LDFLAGS="$(HOST_LDFLAGS) -Wl,--enable-new-dtags"
+
# Building host python in parallel sometimes triggers a "Bus error"
# during the execution of "./python setup.py build" in the
# installation step. It is probably due to the installation of a
diff --git a/package/python3/python3.mk b/package/python3/python3.mk
index 880b66e..3d95a6d 100644
--- a/package/python3/python3.mk
+++ b/package/python3/python3.mk
@@ -30,6 +30,12 @@ HOST_PYTHON3_CONF_OPT += \
--disable-idle3 \
--disable-pyo-build
+# Make sure that LD_LIBRARY_PATH overrides -rpath.
+# This is needed because libpython may be installed at the same time that
+# python is called.
+HOST_PYTHON3_CONF_ENV += \
+ LDFLAGS="$(HOST_LDFLAGS) -Wl,--enable-new-dtags"
+
PYTHON3_DEPENDENCIES = host-python3 libffi
HOST_PYTHON3_DEPENDENCIES = host-expat host-zlib
--
1.9.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-04 23:18 ` Arnout Vandecappelle
2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle
@ 2014-03-05 9:04 ` Thomas Petazzoni
2014-03-05 10:10 ` Arnout Vandecappelle
1 sibling, 1 reply; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-05 9:04 UTC (permalink / raw)
To: buildroot
Dear Arnout Vandecappelle,
On Wed, 05 Mar 2014 00:18:37 +0100, Arnout Vandecappelle wrote:
> On 02/03/14 09:57, Thomas Petazzoni wrote:
> >> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
> > Parallel installation issue with Python 3. Either we should make Python
> > 3 entirely use MAKE1, or fix the problem. I have analyzed the problem,
> > and it is caused by the parallel installation of libpython into the
> > HOST_DIR and the execution of ./python in the Python source directory.
>
> Is that because we export an LD_LIBRARY_PATH that points to the freshly
> installed libpython? Because I would expect that ./python uses the build
> directory's libpython, not the installed one...
>
> Oh, but it actually does get called with the correct LD_LIBRARY_PATH:
>
> LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib:
> ./python ...
>
> The issue is that the executable contains an RPATH pointing to the
> install location of libpython.so (in $(HOST_DIR)).
Correct. I came to the same conclusion when analyzing the problem the
other day (but I reported only my findings on IRC, and didn't send a
summary to the list, my bad).
> When the file doesn't
> exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the
> build directory. When the file does exist, it can link against it -
> except when it hasn't been fully written yet.
>
> The solution would be if the executable would use RUNPATH instead of
> RPATH - which according to e.g. [1] is what everybody should do (and I
> tend to agree). IOW, the option --enable-new-dtags should be passed to
> the linker ("new" is not really true BTW, the option has existed for
> since more than 10 years). When we would add a toolchain wrapper for ld,
> we could add it there.
>
> But for the time being, I'll post a patch for python3 (and python, which
> suffers from the same problem) that adds --enable-new-dtags there.
Indeed, DT_RUNPATH would fix the problem, but to me it feels more like
a workaround than a real fix, no? Wouldn't the real fix be to ensure
that the installation is atomic, i.e either the library is present in
$(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs
when Python is executed when the library file already exists in
$(HOST_DIR)/usr/lib, but its copy hasn't finished yet.
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni
@ 2014-03-05 10:10 ` Arnout Vandecappelle
2014-03-05 12:40 ` Thomas Petazzoni
0 siblings, 1 reply; 10+ messages in thread
From: Arnout Vandecappelle @ 2014-03-05 10:10 UTC (permalink / raw)
To: buildroot
On 05/03/14 10:04, Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
>
> On Wed, 05 Mar 2014 00:18:37 +0100, Arnout Vandecappelle wrote:
>> On 02/03/14 09:57, Thomas Petazzoni wrote:
>>>> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
>>> Parallel installation issue with Python 3. Either we should make Python
>>> 3 entirely use MAKE1, or fix the problem. I have analyzed the problem,
>>> and it is caused by the parallel installation of libpython into the
>>> HOST_DIR and the execution of ./python in the Python source directory.
>>
>> Is that because we export an LD_LIBRARY_PATH that points to the freshly
>> installed libpython? Because I would expect that ./python uses the build
>> directory's libpython, not the installed one...
>>
>> Oh, but it actually does get called with the correct LD_LIBRARY_PATH:
>>
>> LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib:
>> ./python ...
>>
>> The issue is that the executable contains an RPATH pointing to the
>> install location of libpython.so (in $(HOST_DIR)).
>
> Correct. I came to the same conclusion when analyzing the problem the
> other day (but I reported only my findings on IRC, and didn't send a
> summary to the list, my bad).
>
>> When the file doesn't
>> exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the
>> build directory. When the file does exist, it can link against it -
>> except when it hasn't been fully written yet.
>>
>> The solution would be if the executable would use RUNPATH instead of
>> RPATH - which according to e.g. [1] is what everybody should do (and I
>> tend to agree). IOW, the option --enable-new-dtags should be passed to
>> the linker ("new" is not really true BTW, the option has existed for
>> since more than 10 years). When we would add a toolchain wrapper for ld,
>> we could add it there.
>>
>> But for the time being, I'll post a patch for python3 (and python, which
>> suffers from the same problem) that adds --enable-new-dtags there.
>
> Indeed, DT_RUNPATH would fix the problem, but to me it feels more like
> a workaround than a real fix, no? Wouldn't the real fix be to ensure
> that the installation is atomic, i.e either the library is present in
> $(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs
> when Python is executed when the library file already exists in
> $(HOST_DIR)/usr/lib, but its copy hasn't finished yet.
In this case, that happens to be the problem. But in general, it is
possible that there is already an old version of the library in
$(HOST_DIR)/usr/lib, and then that one will be used instead of the newly
built one. Nothing catastrophic, but still wrong.
The thing is, RPATH was just a mistake to begin with: you _always_ want
LD_LIBRARY_PATH to take precedence over RPATH, otherwise LD_LIBRARY_PATH
doesn't make much sense. That's why I think that adding
--enable-new-dtags unconditionally for all builds would be a great idea.
I just noticed now that the idea of an ld wrapper that adds that option
would not work in this case anyway, because this is the host ld... Note
that some distros (I think Gentoo and SUSE at least) patch binutils to
make the --enable-new-dtags the default, so there you wouldn't even see
this issue.
Regards,
Arnout
--
Arnout Vandecappelle arnout at mind be
Senior Embedded Software Architect +32-16-286500
Essensium/Mind http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures
2014-03-05 10:10 ` Arnout Vandecappelle
@ 2014-03-05 12:40 ` Thomas Petazzoni
0 siblings, 0 replies; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-05 12:40 UTC (permalink / raw)
To: buildroot
Dear Arnout Vandecappelle,
On Wed, 05 Mar 2014 11:10:44 +0100, Arnout Vandecappelle wrote:
> > Indeed, DT_RUNPATH would fix the problem, but to me it feels more like
> > a workaround than a real fix, no? Wouldn't the real fix be to ensure
> > that the installation is atomic, i.e either the library is present in
> > $(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs
> > when Python is executed when the library file already exists in
> > $(HOST_DIR)/usr/lib, but its copy hasn't finished yet.
>
> In this case, that happens to be the problem. But in general, it is
> possible that there is already an old version of the library in
> $(HOST_DIR)/usr/lib, and then that one will be used instead of the newly
> built one. Nothing catastrophic, but still wrong.
Right.
> The thing is, RPATH was just a mistake to begin with: you _always_ want
> LD_LIBRARY_PATH to take precedence over RPATH, otherwise LD_LIBRARY_PATH
> doesn't make much sense. That's why I think that adding
> --enable-new-dtags unconditionally for all builds would be a great idea.
True. So we should add --enable-new-dtags to HOST_LDFLAGS ?
Unfortunately, I'm pretty sure there are many packages that don't honor
HOST_LDFLAGS.
> I just noticed now that the idea of an ld wrapper that adds that option
> would not work in this case anyway, because this is the host ld... Note
> that some distros (I think Gentoo and SUSE at least) patch binutils to
> make the --enable-new-dtags the default, so there you wouldn't even see
> this issue.
Ok, yes, indeed we don't have a wrapper for host ld, and it is
specifically for host binaries that we need to set a RUNPATH.
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython
2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle
@ 2014-03-05 18:46 ` Thomas Petazzoni
0 siblings, 0 replies; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-05 18:46 UTC (permalink / raw)
To: buildroot
Dear Arnout Vandecappelle (Essensium/Mind),
On Wed, 5 Mar 2014 00:29:49 +0100, Arnout Vandecappelle
(Essensium/Mind) wrote:
> During installation, host-python and host-python3 run the freshly built
> python executable. This is done with a proper LD_LIBRARY_PATH to make
> sure it picks up the libpython in the build directory. However, the
> python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib.
> Therefore, if libpython exists there, it will be used instead.
>
> If the install step is run in parallel, it is possible that libpython
> is already partially copied to $(HOST_DIR)/usr/lib when python is run.
> This gives an error like:
>
> python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short
>
> The fix is simple: use RUNPATH instead of RPATH, which allows
> LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the
> build directory is always used. RUNPATH is enabled by passing
> --enable-new-dtags to the linker.
>
> Fixes e.g.
> http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4
>
> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Applied, thanks.
Thomas
--
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-03-05 18:46 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-02 7:30 [Buildroot] [autobuild.buildroot.net] Build results for 2014-03-01 Thomas Petazzoni
2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni
2014-03-02 9:50 ` Samuel Martin
2014-03-02 10:08 ` Thomas Petazzoni
2014-03-04 23:18 ` Arnout Vandecappelle
2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle
2014-03-05 18:46 ` Thomas Petazzoni
2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni
2014-03-05 10:10 ` Arnout Vandecappelle
2014-03-05 12:40 ` Thomas Petazzoni
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox