* [Buildroot] [PATCH v2, 1/1] boost: fix static detection of lock-free atomic ints
@ 2018-09-04 22:28 Fabrice Fontaine
2018-09-05 7:19 ` Thomas Petazzoni
0 siblings, 1 reply; 3+ messages in thread
From: Fabrice Fontaine @ 2018-09-04 22:28 UTC (permalink / raw)
To: buildroot
When build statically, boost is unable to detect lock-free atomics ints
because it tries to link dynamically, see
output/build/boost-1.67.0/bin.v2/config.log (with -d5 option):
Using shell: /bin/sh -c
argv[0] = '/bin/sh'
argv[1] = '-c'
argv[2] = '
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
'
gcc.compile.c++ bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
0.033561 sec system; 0.126314 sec user; 288.682473 sec clock
gcc.compile.c++ bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -I"." -c -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" "libs/system/src/error_code.cpp"
0.084060 sec system; 0.644133 sec user; 8.858824 sec clock
SEM: <s>gcc-link-semaphore now used by <pbin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi>libboost_system.so.1.67.0
Using shell: /bin/sh -c
argv[0] = '/bin/sh'
argv[1] = '-c'
argv[2] = '
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static
'
gcc.link.dll bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0
"/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static
ld (ld-elf2flt): -shared used without passing a shared library ID
collect2: error: ld a retourn? le statut de sortie 1
0.003123 sec system; 0.004732 sec user; 15.646509 sec clock
...failed gcc.link.dll bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0...
Due to this wrong detection, boost_atomic is wrongly selected by
boost_thread which cause a build failure on missing select
Add a hack to fix this behavior in libs/thread/build/Jamfile.v2
as I'm not sufficiently knowledgeable in bjam to know why the
configuration is not correctly passed
Fixes:
- http://autobuild.buildroot.org/results/f46d38991385cbc2a4fa14eb31074e770cd79803
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
---
Changes v1 -> v2 (after review of Arnout Vandecappelle):
- Fix URL of build failure
package/boost/boost.mk | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/package/boost/boost.mk b/package/boost/boost.mk
index b2605a70b7..fe986edb00 100644
--- a/package/boost/boost.mk
+++ b/package/boost/boost.mk
@@ -106,6 +106,16 @@ endif
# variants. Override that if we want static only or shared only.
ifeq ($(BR2_STATIC_LIBS),y)
BOOST_OPTS += link=static runtime-link=static
+
+# When detecting features such as lockfree atomic ints, boost does not take into
+# account the configuration provided by the user. Indeed, boost does not specify
+# any value and as a result the default value (shared) is used
+define BOOST_FIX_STATIC_FEATURES_DETECTION
+$(SED) 's/has_atomic_flag_lockfree_test.cpp/has_atomic_flag_lockfree_test.cpp : <link>static <runtime-link>static/g;' \
+ $(@D)/libs/thread/build/Jamfile.v2
+endef
+
+BOOST_PRE_CONFIGURE_HOOKS += BOOST_FIX_STATIC_FEATURES_DETECTION
else ifeq ($(BR2_SHARED_LIBS),y)
BOOST_OPTS += link=shared runtime-link=shared
endif
--
2.17.1
^ permalink raw reply related [flat|nested] 3+ messages in thread* [Buildroot] [PATCH v2, 1/1] boost: fix static detection of lock-free atomic ints
2018-09-04 22:28 [Buildroot] [PATCH v2, 1/1] boost: fix static detection of lock-free atomic ints Fabrice Fontaine
@ 2018-09-05 7:19 ` Thomas Petazzoni
2018-09-05 10:50 ` Fabrice Fontaine
0 siblings, 1 reply; 3+ messages in thread
From: Thomas Petazzoni @ 2018-09-05 7:19 UTC (permalink / raw)
To: buildroot
Hello,
Thanks for investigating this issue, I know what it takes to dive into
the Boost build system, however, I am a bit confused by what's
happening.
On Wed, 5 Sep 2018 00:28:20 +0200, Fabrice Fontaine wrote:
> When build statically, boost is unable to detect lock-free atomics ints
> because it tries to link dynamically, see
> output/build/boost-1.67.0/bin.v2/config.log (with -d5 option):
>
> Using shell: /bin/sh -c
> argv[0] = '/bin/sh'
> argv[1] = '-c'
> argv[2] = '
> "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
> '
> gcc.compile.c++ bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o
>
> "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o" "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
So here, it is compiling (no linking) the
has_atomic_flag_lockfree_test.cpp test file into its corresponding
object file. It does not use -shared at this point.
> 0.033561 sec system; 0.126314 sec user; 288.682473 sec clock
> gcc.compile.c++ bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o
>
> "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline -Wall -pedantic -g -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -I"." -c -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" "libs/system/src/error_code.cpp"
Here, it also compiles error_code.cpp into error_code.o, which I'm not
sure how it is related to the atomic stuff.
> 0.084060 sec system; 0.644133 sec user; 8.858824 sec clock
> SEM: <s>gcc-link-semaphore now used by <pbin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi>libboost_system.so.1.67.0
> Using shell: /bin/sh -c
> argv[0] = '/bin/sh'
> argv[1] = '-c'
> argv[2] = '
> "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0" -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o" -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g -Wl,-elf2flt -static
And here it tries to produce a shared library,
libboost_system.so.1.67.0, which just the error_code.o object file in
it ?
Questions:
- Why is Boost trying to _produce_ a shared library ?
- How is producing a shared library from the error_code.o object file
related to testing the availability of lock-free atomics ?
> +# When detecting features such as lockfree atomic ints, boost does not take into
> +# account the configuration provided by the user. Indeed, boost does not specify
> +# any value and as a result the default value (shared) is used
> +define BOOST_FIX_STATIC_FEATURES_DETECTION
> +$(SED) 's/has_atomic_flag_lockfree_test.cpp/has_atomic_flag_lockfree_test.cpp : <link>static <runtime-link>static/g;' \
> + $(@D)/libs/thread/build/Jamfile.v2
> +endef
Have you reported the issue upstream to see what they have to say ?
Thanks,
Thomas
--
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 3+ messages in thread
* [Buildroot] [PATCH v2, 1/1] boost: fix static detection of lock-free atomic ints
2018-09-05 7:19 ` Thomas Petazzoni
@ 2018-09-05 10:50 ` Fabrice Fontaine
0 siblings, 0 replies; 3+ messages in thread
From: Fabrice Fontaine @ 2018-09-05 10:50 UTC (permalink / raw)
To: buildroot
Dear Thomas,
Le mer. 5 sept. 2018 ? 09:19, Thomas Petazzoni <thomas.petazzoni@bootlin.com>
a ?crit :
> Hello,
>
> Thanks for investigating this issue, I know what it takes to dive into
> the Boost build system, however, I am a bit confused by what's
> happening.
>
> On Wed, 5 Sep 2018 00:28:20 +0200, Fabrice Fontaine wrote:
> > When build statically, boost is unable to detect lock-free atomics ints
> > because it tries to link dynamically, see
> > output/build/boost-1.67.0/bin.v2/config.log (with -d5 option):
> >
> > Using shell: /bin/sh -c
> > argv[0] = '/bin/sh'
> > argv[1] = '-c'
> > argv[2] = '
> > "/home/fabrice/buildroot/output/host/bin/arm-linux-g++"
> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os
> -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline
> -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter
> -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1
> -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o
> "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o"
> "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
> > '
> > gcc.compile.c++
> bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o
> >
> > "/home/fabrice/buildroot/output/host/bin/arm-linux-g++"
> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os
> -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline
> -Wall -pedantic -g -Wextra -Wno-long-long -Wno-unused-parameter
> -Wunused-function -pedantic -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1
> -DBOOST_THREAD_BUILD_DLL=1 -I"." -c -o
> "bin.v2/libs/thread/build/gcc-6.4.0/debug/threading-multi/has_atomic_flag_lockfree_test.o"
> "libs/thread/src/../build/has_atomic_flag_lockfree_test.cpp"
>
> So here, it is compiling (no linking) the
> has_atomic_flag_lockfree_test.cpp test file into its corresponding
> object file. It does not use -shared at this point.
>
> > 0.033561 sec system; 0.126314 sec user; 288.682473 sec clock
> > gcc.compile.c++
> bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o
> >
> > "/home/fabrice/buildroot/output/host/bin/arm-linux-g++"
> -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os
> -Wl,-elf2flt -static -Wl,-elf2flt -static -fPIC -pthread -O0 -fno-inline
> -Wall -pedantic -g -DBOOST_ALL_NO_LIB=1 -DBOOST_SYSTEM_DYN_LINK=1 -I"."
> -c -o
> "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o"
> "libs/system/src/error_code.cpp"
>
> Here, it also compiles error_code.cpp into error_code.o, which I'm not
> sure how it is related to the atomic stuff.
> > 0.084060 sec system; 0.644133 sec user; 8.858824 sec clock
> > SEM: <s>gcc-link-semaphore now used by
> <pbin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi>libboost_system.so.1.67.0
> > Using shell: /bin/sh -c
> > argv[0] = '/bin/sh'
> > argv[1] = '-c'
> > argv[2] = '
> > "/home/fabrice/buildroot/output/host/bin/arm-linux-g++" -o
> "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/libboost_system.so.1.67.0"
> -Wl,-h -Wl,libboost_system.so.1.67.0 -shared -Wl,--start-group
> "bin.v2/libs/system/build/gcc-6.4.0/debug/threading-multi/error_code.o"
> -Wl,-Bstatic -Wl,-Bdynamic -lrt -Wl,--end-group -fPIC -pthread -g
> -Wl,-elf2flt -static
>
> And here it tries to produce a shared library,
> libboost_system.so.1.67.0, which just the error_code.o object file in
> it ?
>
> Questions:
>
> - Why is Boost trying to _produce_ a shared library ?
>
From my understanding and thanks to your helpfull comments, I think I
understand better the issue.
The exe directive is after the "project boost/thread" statement in
libs/thread/build/Jamfile.v2, so bjam assumes that boost_system is needed
and tries to build it even if I don't know why it uses the default shared
linking mode (perhaps because the exe statement is executed before
boost/thread retrieve the user configuration).
So I tried to move the exe directive in the beginning of the file as
boost_system is not needed to check atomics and it seems to work, I will
send a v3 and get the feedback from the upstream project.
> - How is producing a shared library from the error_code.o object file
> related to testing the availability of lock-free atomics ?
>
> > +# When detecting features such as lockfree atomic ints, boost does not
> take into
> > +# account the configuration provided by the user. Indeed, boost does
> not specify
> > +# any value and as a result the default value (shared) is used
> > +define BOOST_FIX_STATIC_FEATURES_DETECTION
> > +$(SED)
> 's/has_atomic_flag_lockfree_test.cpp/has_atomic_flag_lockfree_test.cpp :
> <link>static <runtime-link>static/g;' \
> > + $(@D)/libs/thread/build/Jamfile.v2
> > +endef
>
> Have you reported the issue upstream to see what they have to say ?
> Thanks,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
Best Regards,
Fabrice
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180905/214695ba/attachment.html>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-09-05 10:50 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-09-04 22:28 [Buildroot] [PATCH v2, 1/1] boost: fix static detection of lock-free atomic ints Fabrice Fontaine
2018-09-05 7:19 ` Thomas Petazzoni
2018-09-05 10:50 ` Fabrice Fontaine
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox