* [Buildroot] [autobuild.buildroot.net] Build results for 2014-03-01
@ 2014-03-02 7:30 Thomas Petazzoni
2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni
0 siblings, 1 reply; 10+ messages in thread
From: Thomas Petazzoni @ 2014-03-02 7:30 UTC (permalink / raw)
To: buildroot
Build statistics for 2014-03-01
===============================
success : 42
failures : 48
timeouts : 0
TOTAL : 90
Classification of failures by reason
====================================
host-python-setuptools-2.1.2 | 16
php-gnupg-1.3.3 | 5
lvm2-2.02.103 | 4
parted-3.1 | 4
python-pyasn-0.17 | 3
bash-4.3 | 3
host-scons-2.3.0 | 2
crda-1.1.3 | 2
mplayer-1.1.1 | 1
libnftnl-1.0.0 | 1
pciutils-3.2.1 | 1
cairo-1.12.10 | 1
host-python3-3.4.0rc1 | 1
zeromq-3.2.4 | 1
fbgrab-1.2 | 1
libmicrohttpd-0.9.34 | 1
libsigsegv-2.10 | 1
Detail of failures
===================
i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/44a962623be757143a3d6d37fc6cd14964094f79/
i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/9c16032f9630e09e3709a69ced6716de725c153d/
x86_64 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/6902eee5e39d9fd79b1e78ccd54382299f1b25ea/
bfin | cairo-1.12.10 | NOK | http://autobuild.buildroot.net/results/c421cde66d8d2d9bdf9ecbe531fb15b86f85d909/
powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/87bfdc235ee441153c36489a0a1ec97266899d5a/
powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/d442f571f14758a4bdf7d694cecd5a34116be40a/
bfin | fbgrab-1.2 | NOK | http://autobuild.buildroot.net/results/b87b9397f4bf22579e7fca9763c9f099bb63bd8f/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/f22ffbf62d4bc8a1f910269e06636181c324644f/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5e6f3c45dd480ef2bed8515d6a48579642e28274/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/aa6334d89d47d91fdb696d9248036ec27b4b752a/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/95d0e3656d539258bffa9ca3637d98bc48876a40/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/177f027edd981ed2ec081d93a15993e51f408620/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/87909291c77a45b1fbe99f641ea8f7d4b8b50e13/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/4952e3a5bd6bac437b8b382dfa18f52eb17505da/
arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/91df531bb3bb57f9b562ed358d6e4a7a970d5e8a/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5deb94ab960aa2f4428f7e9d2d5315ba79116ce1/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/11725dbff02b07096552a8c0f85d90d969a31fc9/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/1b4713983e9148408081ee432717cb9f6afa5ac9/
arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b8a4c5dce70d6215945293cd6fd28282c3896a78/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/de0d965f2e1d6e70b234c2708d558273291f3a5f/
powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/05c721a141e698d0df49b2867a218d31e7acb311/
i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b01f64c75e71fb4e5282f690ee76a2a49842e889/
i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/
arm | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/
powerpc | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/
i686 | libmicrohttpd-0.9.34 | NOK | http://autobuild.buildroot.net/results/edb1b5ee1ffaa6a495b4d7c2976f618c8331b5d2/
powerpc | libnftnl-1.0.0 | NOK | http://autobuild.buildroot.net/results/c13ef3757b90838549d6f396155a20890d163ff9/
nios2 | libsigsegv-2.10 | NOK | http://autobuild.buildroot.net/results/82dac744ce01c9ebb8c769aea02bf29bc7494aa6/
arm | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/dd5ceda4e3778a763d26b8a81a08391f13f02307/
powerpc | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/d6f4d388c62ec2137560aabbff46c4a30c584e18/
mips | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/bc9b84964667fd37155df97c35252d6b73be724f/
microblaze | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/affded5ab5590e6f7313c59f511b81c1bb645c92/
arm | mplayer-1.1.1 | NOK | http://autobuild.buildroot.net/results/ee409788bbf44434a53c14ede7928367ba8e1c85/
arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/87ee883d745fafd88fa446244cfe69659ff06f3c/
arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/1e90929bfaf617381d490dc0b99f333968624ce7/
arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/c4cb972758a58c4b62e600c14936af4fd559295b/
i686 | parted-3.1 | NOK | http://autobuild.buildroot.net/results/4b5f5dc38a08f622992ea22fd58c1d82b80890c8/
bfin | pciutils-3.2.1 | NOK | http://autobuild.buildroot.net/results/f5c6d4453d546001423244476adf2c6f40411e2c/
i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/e3b2d68b22224fcda95174bd64803a58aae1606a/
powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/fe26034ee29c7334d09cba8a80ae5bfd2787a552/
i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/df9f7c7c30d9cdba65b4f5c6d46c8aa91de4179a/
powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/efbce29a2fa1d8561c10613d2145b9b382ddc83c/
powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/757fb0b12af12de4f128d9c503d45b20776c2313/
arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/3c968459f6cc020a3c6bc77f49245d69ed92df20/
arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/e009a9a5f8a311b5fcb41212d517a2d52ff2db9d/
arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/c11893d19f5614236f1d30a5fca781b94d2f9f80/
bfin | zeromq-3.2.4 | NOK | http://autobuild.buildroot.net/results/59182657d03ca48dc46c96d4ff14667324f4e4f6/
--
http://autobuild.buildroot.net
^ permalink raw reply [flat|nested] 10+ messages in thread* [Buildroot] Analysis of build failures 2014-03-02 7:30 [Buildroot] [autobuild.buildroot.net] Build results for 2014-03-01 Thomas Petazzoni @ 2014-03-02 8:57 ` Thomas Petazzoni 2014-03-02 9:50 ` Samuel Martin 2014-03-04 23:18 ` Arnout Vandecappelle 0 siblings, 2 replies; 10+ messages in thread From: Thomas Petazzoni @ 2014-03-02 8:57 UTC (permalink / raw) To: buildroot Hello all, On Sun, 2 Mar 2014 08:30:07 +0100 (CET), Thomas Petazzoni wrote: > i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/44a962623be757143a3d6d37fc6cd14964094f79/ > i686 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/9c16032f9630e09e3709a69ced6716de725c153d/ > x86_64 | bash-4.3 | NOK | http://autobuild.buildroot.net/results/6902eee5e39d9fd79b1e78ccd54382299f1b25ea/F Fixed by http://git.buildroot.net/buildroot/commit/?id=7a95111e48f5fe387f592463effa233834c6fda1 > bfin | cairo-1.12.10 | NOK | http://autobuild.buildroot.net/results/c421cde66d8d2d9bdf9ecbe531fb15b86f85d909/ To be fixed. > powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/87bfdc235ee441153c36489a0a1ec97266899d5a/ > powerpc | crda-1.1.3 | NOK | http://autobuild.buildroot.net/results/d442f571f14758a4bdf7d694cecd5a34116be40a/ Issue when host-python3 is built before crda. The easiest solution is probably to make the crda Python script Python 3 compatible and submit the patch upstream. > bfin | fbgrab-1.2 | NOK | http://autobuild.buildroot.net/results/b87b9397f4bf22579e7fca9763c9f099bb63bd8f/ Missing -l flags? > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/f22ffbf62d4bc8a1f910269e06636181c324644f/ > powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5e6f3c45dd480ef2bed8515d6a48579642e28274/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/aa6334d89d47d91fdb696d9248036ec27b4b752a/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/95d0e3656d539258bffa9ca3637d98bc48876a40/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/177f027edd981ed2ec081d93a15993e51f408620/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/87909291c77a45b1fbe99f641ea8f7d4b8b50e13/ > powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/4952e3a5bd6bac437b8b382dfa18f52eb17505da/ > arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b622fc14776f5a6c26ff0d1fb882f4ae6cccc5d8/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/91df531bb3bb57f9b562ed358d6e4a7a970d5e8a/ > powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/5deb94ab960aa2f4428f7e9d2d5315ba79116ce1/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/11725dbff02b07096552a8c0f85d90d969a31fc9/ > powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/1b4713983e9148408081ee432717cb9f6afa5ac9/ > arm | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b8a4c5dce70d6215945293cd6fd28282c3896a78/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/de0d965f2e1d6e70b234c2708d558273291f3a5f/ > powerpc | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/05c721a141e698d0df49b2867a218d31e7acb311/ > i686 | host-python-setuptools-2.1.2 | NOK | http://autobuild.buildroot.net/results/b01f64c75e71fb4e5282f690ee76a2a49842e889/ Fixed by http://git.buildroot.net/buildroot/commit/?id=bec21dcfa7042d1f7065cf467c4f9c777f0fd072 > i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/ Parallel installation issue with Python 3. Either we should make Python 3 entirely use MAKE1, or fix the problem. I have analyzed the problem, and it is caused by the parallel installation of libpython into the HOST_DIR and the execution of ./python in the Python source directory. > arm | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/ > powerpc | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/ Scons is not compatible with Python 3. But host-scons is not a target package, it's a host package, so we cannot add a Python 3 related dependency to it. So the two solutions that I see are: * Change our host-python and host-python3 packages so that they are not exclusive from each other, and then find a way for host-scons to express the fact that it really needs host-python, and not the host-python version matching the selected target python version. Note that achieving this might be difficult. * Make the three target packages that actually need host-scons "depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next version of SCons is supposed to support Python 3, so we could consider it as a temporary solution. Suggestions? > i686 | libmicrohttpd-0.9.34 | NOK | http://autobuild.buildroot.net/results/edb1b5ee1ffaa6a495b4d7c2976f618c8331b5d2/ Fixed by http://git.buildroot.net/buildroot/commit/?id=25c438bce1c02775f2c23d8878ab71c3022b0c95 > powerpc | libnftnl-1.0.0 | NOK | http://autobuild.buildroot.net/results/c13ef3757b90838549d6f396155a20890d163ff9/ Probably needs some kernel headers dependency. To be checked in detail. Any taker? > nios2 | libsigsegv-2.10 | NOK | http://autobuild.buildroot.net/results/82dac744ce01c9ebb8c769aea02bf29bc7494aa6/ Needs a small patch to support NIOS 2. The only information needed is to know whether the stack grows upward or downward on the architecture. > arm | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/dd5ceda4e3778a763d26b8a81a08391f13f02307/ > powerpc | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/d6f4d388c62ec2137560aabbff46c4a30c584e18/ > mips | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/bc9b84964667fd37155df97c35252d6b73be724f/ > microblaze | lvm2-2.02.103 | NOK | http://autobuild.buildroot.net/results/affded5ab5590e6f7313c59f511b81c1bb645c92/ Fixed by http://git.buildroot.net/buildroot/commit/?id=fb36a82a745e93afc92bb5aff29daea8daecb6d3. > arm | mplayer-1.1.1 | NOK | http://autobuild.buildroot.net/results/ee409788bbf44434a53c14ede7928367ba8e1c85/ ARM floating point issue. > arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/87ee883d745fafd88fa446244cfe69659ff06f3c/ > arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/1e90929bfaf617381d490dc0b99f333968624ce7/ > arm | parted-3.1 | NOK | http://autobuild.buildroot.net/results/c4cb972758a58c4b62e600c14936af4fd559295b/ > i686 | parted-3.1 | NOK | http://autobuild.buildroot.net/results/4b5f5dc38a08f622992ea22fd58c1d82b80890c8/ Fixed by http://git.buildroot.net/buildroot/commit/?id=8d89fd2b7c09af47f2d1b81cfa4de45a421d8a83. > bfin | pciutils-3.2.1 | NOK | http://autobuild.buildroot.net/results/f5c6d4453d546001423244476adf2c6f40411e2c/ Blackfin specific issue related to the usage of _ as prefix for assembly symbols. > i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/e3b2d68b22224fcda95174bd64803a58aae1606a/ > powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/fe26034ee29c7334d09cba8a80ae5bfd2787a552/ > i686 | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/df9f7c7c30d9cdba65b4f5c6d46c8aa91de4179a/ > powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/efbce29a2fa1d8561c10613d2145b9b382ddc83c/ > powerpc | php-gnupg-1.3.3 | NOK | http://autobuild.buildroot.net/results/757fb0b12af12de4f128d9c503d45b20776c2313/ Fixed by http://git.buildroot.net/buildroot/commit/?id=1188fcf1667ac316ea9df024b9346415a05b90db. > arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/3c968459f6cc020a3c6bc77f49245d69ed92df20/ > arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/e009a9a5f8a311b5fcb41212d517a2d52ff2db9d/ > arm | python-pyasn-0.17 | NOK | http://autobuild.buildroot.net/results/c11893d19f5614236f1d30a5fca781b94d2f9f80/ Fixed by http://git.buildroot.net/buildroot/commit/?id=e025e49100565617715e0de986ba670532098b14. > bfin | zeromq-3.2.4 | NOK | http://autobuild.buildroot.net/results/59182657d03ca48dc46c96d4ff14667324f4e4f6/ Compiler does not provide the right compiler intrinsics. We should check if more recent versions of gcc do not provide the necessary intrinsics. Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures 2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni @ 2014-03-02 9:50 ` Samuel Martin 2014-03-02 10:08 ` Thomas Petazzoni 2014-03-04 23:18 ` Arnout Vandecappelle 1 sibling, 1 reply; 10+ messages in thread From: Samuel Martin @ 2014-03-02 9:50 UTC (permalink / raw) To: buildroot Thomas, all, On Sun, Mar 2, 2014 at 9:57 AM, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote: [...] >> arm | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/1ee0d18f2a9e45b94e4db04fd5c54e6f7994b1be/ >> powerpc | host-scons-2.3.0 | NOK | http://autobuild.buildroot.net/results/d1421f2063ecfea9c4110414f481f779defa8ad5/ > > Scons is not compatible with Python 3. But host-scons is not a target > package, it's a host package, so we cannot add a Python 3 related > dependency to it. So the two solutions that I see are: > > * Change our host-python and host-python3 packages so that they are > not exclusive from each other, and then find a way for host-scons to > express the fact that it really needs host-python, and not the > host-python version matching the selected target python version. > Note that achieving this might be difficult. host-python and host-python3 are exclusive at the kconfig level, not the make target one, right or am I missing something? AFAICS the only overlapping thing between the 2 host package is the $(HOST_DIR)/usr/bin/python symlink. So, we could imagine building host-python and host-python3 in the same build. In such a case, the tricky point becomes the $(HOST_DIR)/usr/bin/python symlink management: - host-python should not install the symlink if python3 is enabled; - host-python3 should deactivate the symlink installation when python is enabled. Then, the SCONS variable should explicity refer to "$(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons" So, we could be able to build host-python (for host-scons) even if python3 is enabled. host-python is the only package providing python2 stuff we would allow to build when python3 is enabled. For other packages, it should not change anything since the $(HOST_DIR)/usr/bin/python symlink still reflects the configuration (ie. its points to python3 if python3 is enabled, to python if python(2) is enabled). For example, crda needs host-python-m2crypto. Since there is no change in the buildroot configuration (ie. the BR2_PACKAGE_PYTHON and BR2_PACKAGE_PYTHON3 will still have the right value) and $(HOST_DIR)/usr/bin/python still reflect this configuration, host-python-m2crypto should still be built against the right python version. Of course, if we choose this way, it requires tests. Comments? > > * Make the three target packages that actually need host-scons > "depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next > version of SCons is supposed to support Python 3, so we could > consider it as a temporary solution. I've just checked out scons repository and it cannot be bootstraped with python3 :'( > > Suggestions? See above ;) Regards, -- Samuel ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures 2014-03-02 9:50 ` Samuel Martin @ 2014-03-02 10:08 ` Thomas Petazzoni 0 siblings, 0 replies; 10+ messages in thread From: Thomas Petazzoni @ 2014-03-02 10:08 UTC (permalink / raw) To: buildroot Dear Samuel Martin, On Sun, 2 Mar 2014 10:50:41 +0100, Samuel Martin wrote: > host-python and host-python3 are exclusive at the kconfig level Hum? There are no Kconfig options for host-python and host-python3. It's the target Pythons that are mutually exclusive at the Kconfig level. And since each target Python depends on the same version of host-pythonX, this is what brings host-python3 when target python3 is selected. > the make target one, right or am I missing something? > AFAICS the only overlapping thing between the 2 host package is the > $(HOST_DIR)/usr/bin/python symlink. > > So, we could imagine building host-python and host-python3 in the same build. > In such a case, the tricky point becomes the > $(HOST_DIR)/usr/bin/python symlink management: > - host-python should not install the symlink if python3 is enabled; > - host-python3 should deactivate the symlink installation when python > is enabled. > Then, the SCONS variable should explicity refer to > "$(HOST_DIR)/usr/bin/python2 $(HOST_DIR)/usr/bin/scons" > So, we could be able to build host-python (for host-scons) even if > python3 is enabled. You're missing something: the host-scons package is using the host-python-package infrastructure. And this infrastructure uses the host Python version of the selected target Python. This is where the core of the problem is. We would need a way to tell the host-python-package infrastructure to make a package depend on host-python instead of host-python3, even if the selected target Python is Python 3. > > * Make the three target packages that actually need host-scons > > "depends on !BR2_PACKAGE_PYTHON3". This is a bit ugly, but the next > > version of SCons is supposed to support Python 3, so we could > > consider it as a temporary solution. > > I've just checked out scons repository and it cannot be bootstraped > with python3 :'( See the Scons homepage: """ SCons release 2.3.0 now available from the download page at SourceForge. This release adds new features, and fixes and improves a number of issues. This will be the last release to support Python versions earlier than 2.7, as we begin to move toward supporting Python 3. """ Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures 2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni 2014-03-02 9:50 ` Samuel Martin @ 2014-03-04 23:18 ` Arnout Vandecappelle 2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle 2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni 1 sibling, 2 replies; 10+ messages in thread From: Arnout Vandecappelle @ 2014-03-04 23:18 UTC (permalink / raw) To: buildroot On 02/03/14 09:57, Thomas Petazzoni wrote: >> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/ > Parallel installation issue with Python 3. Either we should make Python > 3 entirely use MAKE1, or fix the problem. I have analyzed the problem, > and it is caused by the parallel installation of libpython into the > HOST_DIR and the execution of ./python in the Python source directory. Is that because we export an LD_LIBRARY_PATH that points to the freshly installed libpython? Because I would expect that ./python uses the build directory's libpython, not the installed one... Oh, but it actually does get called with the correct LD_LIBRARY_PATH: LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib: ./python ... The issue is that the executable contains an RPATH pointing to the install location of libpython.so (in $(HOST_DIR)). When the file doesn't exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the build directory. When the file does exist, it can link against it - except when it hasn't been fully written yet. The solution would be if the executable would use RUNPATH instead of RPATH - which according to e.g. [1] is what everybody should do (and I tend to agree). IOW, the option --enable-new-dtags should be passed to the linker ("new" is not really true BTW, the option has existed for since more than 10 years). When we would add a toolchain wrapper for ld, we could add it there. But for the time being, I'll post a patch for python3 (and python, which suffers from the same problem) that adds --enable-new-dtags there. Regards, Arnout [1] http://blog.tremily.us/posts/rpath/ -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython 2014-03-04 23:18 ` Arnout Vandecappelle @ 2014-03-04 23:29 ` Arnout Vandecappelle 2014-03-05 18:46 ` Thomas Petazzoni 2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni 1 sibling, 1 reply; 10+ messages in thread From: Arnout Vandecappelle @ 2014-03-04 23:29 UTC (permalink / raw) To: buildroot During installation, host-python and host-python3 run the freshly built python executable. This is done with a proper LD_LIBRARY_PATH to make sure it picks up the libpython in the build directory. However, the python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib. Therefore, if libpython exists there, it will be used instead. If the install step is run in parallel, it is possible that libpython is already partially copied to $(HOST_DIR)/usr/lib when python is run. This gives an error like: python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short The fix is simple: use RUNPATH instead of RPATH, which allows LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the build directory is always used. RUNPATH is enabled by passing --enable-new-dtags to the linker. Fixes e.g. http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4 Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> --- package/python/python.mk | 6 ++++++ package/python3/python3.mk | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/package/python/python.mk b/package/python/python.mk index 94765cb..216448e 100644 --- a/package/python/python.mk +++ b/package/python/python.mk @@ -34,6 +34,12 @@ HOST_PYTHON_CONF_OPT += \ --disable-ssl \ --disable-pyo-build +# Make sure that LD_LIBRARY_PATH overrides -rpath. +# This is needed because libpython may be installed at the same time that +# python is called. +HOST_PYTHON_CONF_ENV += \ + LDFLAGS="$(HOST_LDFLAGS) -Wl,--enable-new-dtags" + # Building host python in parallel sometimes triggers a "Bus error" # during the execution of "./python setup.py build" in the # installation step. It is probably due to the installation of a diff --git a/package/python3/python3.mk b/package/python3/python3.mk index 880b66e..3d95a6d 100644 --- a/package/python3/python3.mk +++ b/package/python3/python3.mk @@ -30,6 +30,12 @@ HOST_PYTHON3_CONF_OPT += \ --disable-idle3 \ --disable-pyo-build +# Make sure that LD_LIBRARY_PATH overrides -rpath. +# This is needed because libpython may be installed at the same time that +# python is called. +HOST_PYTHON3_CONF_ENV += \ + LDFLAGS="$(HOST_LDFLAGS) -Wl,--enable-new-dtags" + PYTHON3_DEPENDENCIES = host-python3 libffi HOST_PYTHON3_DEPENDENCIES = host-expat host-zlib -- 1.9.0 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython 2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle @ 2014-03-05 18:46 ` Thomas Petazzoni 0 siblings, 0 replies; 10+ messages in thread From: Thomas Petazzoni @ 2014-03-05 18:46 UTC (permalink / raw) To: buildroot Dear Arnout Vandecappelle (Essensium/Mind), On Wed, 5 Mar 2014 00:29:49 +0100, Arnout Vandecappelle (Essensium/Mind) wrote: > During installation, host-python and host-python3 run the freshly built > python executable. This is done with a proper LD_LIBRARY_PATH to make > sure it picks up the libpython in the build directory. However, the > python binary has an RPATH pointing to the $(HOST_DIR)/usr/lib. > Therefore, if libpython exists there, it will be used instead. > > If the install step is run in parallel, it is possible that libpython > is already partially copied to $(HOST_DIR)/usr/lib when python is run. > This gives an error like: > > python: error while loading shared libraries: $(HOST_DIR)/usr/lib/libpython3.4m.so.1.0: file too short > > The fix is simple: use RUNPATH instead of RPATH, which allows > LD_LIBRARY_PATH to override RUNPATH. That way, the libpython in the > build directory is always used. RUNPATH is enabled by passing > --enable-new-dtags to the linker. > > Fixes e.g. > http://autobuild.buildroot.net/results/2a6/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4 > > Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be> Applied, thanks. Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures 2014-03-04 23:18 ` Arnout Vandecappelle 2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle @ 2014-03-05 9:04 ` Thomas Petazzoni 2014-03-05 10:10 ` Arnout Vandecappelle 1 sibling, 1 reply; 10+ messages in thread From: Thomas Petazzoni @ 2014-03-05 9:04 UTC (permalink / raw) To: buildroot Dear Arnout Vandecappelle, On Wed, 05 Mar 2014 00:18:37 +0100, Arnout Vandecappelle wrote: > On 02/03/14 09:57, Thomas Petazzoni wrote: > >> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/ > > Parallel installation issue with Python 3. Either we should make Python > > 3 entirely use MAKE1, or fix the problem. I have analyzed the problem, > > and it is caused by the parallel installation of libpython into the > > HOST_DIR and the execution of ./python in the Python source directory. > > Is that because we export an LD_LIBRARY_PATH that points to the freshly > installed libpython? Because I would expect that ./python uses the build > directory's libpython, not the installed one... > > Oh, but it actually does get called with the correct LD_LIBRARY_PATH: > > LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib: > ./python ... > > The issue is that the executable contains an RPATH pointing to the > install location of libpython.so (in $(HOST_DIR)). Correct. I came to the same conclusion when analyzing the problem the other day (but I reported only my findings on IRC, and didn't send a summary to the list, my bad). > When the file doesn't > exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the > build directory. When the file does exist, it can link against it - > except when it hasn't been fully written yet. > > The solution would be if the executable would use RUNPATH instead of > RPATH - which according to e.g. [1] is what everybody should do (and I > tend to agree). IOW, the option --enable-new-dtags should be passed to > the linker ("new" is not really true BTW, the option has existed for > since more than 10 years). When we would add a toolchain wrapper for ld, > we could add it there. > > But for the time being, I'll post a patch for python3 (and python, which > suffers from the same problem) that adds --enable-new-dtags there. Indeed, DT_RUNPATH would fix the problem, but to me it feels more like a workaround than a real fix, no? Wouldn't the real fix be to ensure that the installation is atomic, i.e either the library is present in $(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs when Python is executed when the library file already exists in $(HOST_DIR)/usr/lib, but its copy hasn't finished yet. Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures 2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni @ 2014-03-05 10:10 ` Arnout Vandecappelle 2014-03-05 12:40 ` Thomas Petazzoni 0 siblings, 1 reply; 10+ messages in thread From: Arnout Vandecappelle @ 2014-03-05 10:10 UTC (permalink / raw) To: buildroot On 05/03/14 10:04, Thomas Petazzoni wrote: > Dear Arnout Vandecappelle, > > On Wed, 05 Mar 2014 00:18:37 +0100, Arnout Vandecappelle wrote: >> On 02/03/14 09:57, Thomas Petazzoni wrote: >>>> i686 | host-python3-3.4.0rc1 | NOK | http://autobuild.buildroot.net/results/2a62de3247ba5ad273f03d01e690a3eeb11aa7b4/ >>> Parallel installation issue with Python 3. Either we should make Python >>> 3 entirely use MAKE1, or fix the problem. I have analyzed the problem, >>> and it is caused by the parallel installation of libpython into the >>> HOST_DIR and the execution of ./python in the Python source directory. >> >> Is that because we export an LD_LIBRARY_PATH that points to the freshly >> installed libpython? Because I would expect that ./python uses the build >> directory's libpython, not the installed one... >> >> Oh, but it actually does get called with the correct LD_LIBRARY_PATH: >> >> LD_LIBRARY_PATH=/home/peko/scratch/build/host-python3-3.4.0rc1:/home/peko/scratch/host/usr/lib: >> ./python ... >> >> The issue is that the executable contains an RPATH pointing to the >> install location of libpython.so (in $(HOST_DIR)). > > Correct. I came to the same conclusion when analyzing the problem the > other day (but I reported only my findings on IRC, and didn't send a > summary to the list, my bad). > >> When the file doesn't >> exist yet there, LD_LIBRARY_PATH kicks in and it finds the so file in the >> build directory. When the file does exist, it can link against it - >> except when it hasn't been fully written yet. >> >> The solution would be if the executable would use RUNPATH instead of >> RPATH - which according to e.g. [1] is what everybody should do (and I >> tend to agree). IOW, the option --enable-new-dtags should be passed to >> the linker ("new" is not really true BTW, the option has existed for >> since more than 10 years). When we would add a toolchain wrapper for ld, >> we could add it there. >> >> But for the time being, I'll post a patch for python3 (and python, which >> suffers from the same problem) that adds --enable-new-dtags there. > > Indeed, DT_RUNPATH would fix the problem, but to me it feels more like > a workaround than a real fix, no? Wouldn't the real fix be to ensure > that the installation is atomic, i.e either the library is present in > $(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs > when Python is executed when the library file already exists in > $(HOST_DIR)/usr/lib, but its copy hasn't finished yet. In this case, that happens to be the problem. But in general, it is possible that there is already an old version of the library in $(HOST_DIR)/usr/lib, and then that one will be used instead of the newly built one. Nothing catastrophic, but still wrong. The thing is, RPATH was just a mistake to begin with: you _always_ want LD_LIBRARY_PATH to take precedence over RPATH, otherwise LD_LIBRARY_PATH doesn't make much sense. That's why I think that adding --enable-new-dtags unconditionally for all builds would be a great idea. I just noticed now that the idea of an ld wrapper that adds that option would not work in this case anyway, because this is the host ld... Note that some distros (I think Gentoo and SUSE at least) patch binutils to make the --enable-new-dtags the default, so there you wouldn't even see this issue. Regards, Arnout -- Arnout Vandecappelle arnout at mind be Senior Embedded Software Architect +32-16-286500 Essensium/Mind http://www.mind.be G.Geenslaan 9, 3001 Leuven, Belgium BE 872 984 063 RPR Leuven LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle GPG fingerprint: 7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F ^ permalink raw reply [flat|nested] 10+ messages in thread
* [Buildroot] Analysis of build failures 2014-03-05 10:10 ` Arnout Vandecappelle @ 2014-03-05 12:40 ` Thomas Petazzoni 0 siblings, 0 replies; 10+ messages in thread From: Thomas Petazzoni @ 2014-03-05 12:40 UTC (permalink / raw) To: buildroot Dear Arnout Vandecappelle, On Wed, 05 Mar 2014 11:10:44 +0100, Arnout Vandecappelle wrote: > > Indeed, DT_RUNPATH would fix the problem, but to me it feels more like > > a workaround than a real fix, no? Wouldn't the real fix be to ensure > > that the installation is atomic, i.e either the library is present in > > $(HOST_DIR)/usr/lib, or it is not? The problem here apparently occurs > > when Python is executed when the library file already exists in > > $(HOST_DIR)/usr/lib, but its copy hasn't finished yet. > > In this case, that happens to be the problem. But in general, it is > possible that there is already an old version of the library in > $(HOST_DIR)/usr/lib, and then that one will be used instead of the newly > built one. Nothing catastrophic, but still wrong. Right. > The thing is, RPATH was just a mistake to begin with: you _always_ want > LD_LIBRARY_PATH to take precedence over RPATH, otherwise LD_LIBRARY_PATH > doesn't make much sense. That's why I think that adding > --enable-new-dtags unconditionally for all builds would be a great idea. True. So we should add --enable-new-dtags to HOST_LDFLAGS ? Unfortunately, I'm pretty sure there are many packages that don't honor HOST_LDFLAGS. > I just noticed now that the idea of an ld wrapper that adds that option > would not work in this case anyway, because this is the host ld... Note > that some distros (I think Gentoo and SUSE at least) patch binutils to > make the --enable-new-dtags the default, so there you wouldn't even see > this issue. Ok, yes, indeed we don't have a wrapper for host ld, and it is specifically for host binaries that we need to set a RUNPATH. Thomas -- Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2014-03-05 18:46 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-02 7:30 [Buildroot] [autobuild.buildroot.net] Build results for 2014-03-01 Thomas Petazzoni
2014-03-02 8:57 ` [Buildroot] Analysis of build failures Thomas Petazzoni
2014-03-02 9:50 ` Samuel Martin
2014-03-02 10:08 ` Thomas Petazzoni
2014-03-04 23:18 ` Arnout Vandecappelle
2014-03-04 23:29 ` [Buildroot] [PATCH] host-python{, 3}: fix parallel install of libpython Arnout Vandecappelle
2014-03-05 18:46 ` Thomas Petazzoni
2014-03-05 9:04 ` [Buildroot] Analysis of build failures Thomas Petazzoni
2014-03-05 10:10 ` Arnout Vandecappelle
2014-03-05 12:40 ` Thomas Petazzoni
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox