Yocto development with C++11 threads and gcc

Yocto development with C++11 threads and gcc

[Peter A. Bigot]

The program below built on the target with the MACHINE=beaglebone 
gcc-4.9.1 compiler from Yocto/OpenEmbedded poky master produces this error:

beaglebone[52]$ g++ -std=c++1y -pthread test.cc && ./a.out
starting
joining
pure virtual method called
terminate called without an active exception
Aborted (core dumped)

When the program is recompiled with the defines for 
__GCC_HAVE_SYNC_COMPARE_AND_SWAP_X enabled as suggested at 
https://groups.google.com/d/msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ 
it works:

beaglebone[53]$ g++ -std=c++1y -pthread test.cc && ./a.out
starting
joining
doit
done

Preliminary analysis confirms that the built-ins for those defines are 
not being added by the compiler because it thinks the target doesn't 
support those operations.  Nonetheless, it doesn't use the substitutes 
that are obviously available.

Can anybody recall anything about the way GCC is built under OE that 
would explain this?

Peter

/* g++ -std=c++1y -pthread test.cc && ./a.out */
#if 0
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4
#define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
#endif

#include <iostream>
#include <thread>

void
doit ()
{
   std::cerr << "doit\n";
}

int main (int argc,
           char * argv [])
{
   std::cerr << "starting\n";
   std::thread thr{doit};
   std::cerr << "joining\n";
   thr.join();
   std::cerr << "done\n";
   return 0;
}

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/11/2014 02:02 PM, Peter A. Bigot wrote:
> The program below built on the target with the MACHINE=beaglebone 
> gcc-4.9.1 compiler from Yocto/OpenEmbedded poky master produces this 
> error:
>
> beaglebone[52]$ g++ -std=c++1y -pthread test.cc && ./a.out
> starting
> joining
> pure virtual method called
> terminate called without an active exception
> Aborted (core dumped)
>
> When the program is recompiled with the defines for 
> __GCC_HAVE_SYNC_COMPARE_AND_SWAP_X enabled as suggested at 
> https://groups.google.com/d/msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ it 
> works:
>
> beaglebone[53]$ g++ -std=c++1y -pthread test.cc && ./a.out
> starting
> joining
> doit
> done
>
> Preliminary analysis confirms that the built-ins for those defines are 
> not being added by the compiler because it thinks the target doesn't 
> support those operations.  Nonetheless, it doesn't use the substitutes 
> that are obviously available.
>
> Can anybody recall anything about the way GCC is built under OE that 
> would explain this?

Not an OE problem.  See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62100

I'm testing a patch locally and awaiting GCC maintainer comment before 
proposing it for OE.

Peter

>
> Peter
>
> /* g++ -std=c++1y -pthread test.cc && ./a.out */
> #if 0
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
> #endif
>
> #include <iostream>
> #include <thread>
>
> void
> doit ()
> {
>   std::cerr << "doit\n";
> }
>
> int main (int argc,
>           char * argv [])
> {
>   std::cerr << "starting\n";
>   std::thread thr{doit};
>   std::cerr << "joining\n";
>   thr.join();
>   std::cerr << "done\n";
>   return 0;
> }
>

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/11/2014 07:12 PM, Peter A. Bigot wrote:
> On 08/11/2014 02:02 PM, Peter A. Bigot wrote:
>> The program below built on the target with the MACHINE=beaglebone 
>> gcc-4.9.1 compiler from Yocto/OpenEmbedded poky master produces this 
>> error:
>>
>> beaglebone[52]$ g++ -std=c++1y -pthread test.cc && ./a.out
>> starting
>> joining
>> pure virtual method called
>> terminate called without an active exception
>> Aborted (core dumped)
>>
>> When the program is recompiled with the defines for 
>> __GCC_HAVE_SYNC_COMPARE_AND_SWAP_X enabled as suggested at 
>> https://groups.google.com/d/msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ 
>> it works:
>>
>> beaglebone[53]$ g++ -std=c++1y -pthread test.cc && ./a.out
>> starting
>> joining
>> doit
>> done
>>
>> Preliminary analysis confirms that the built-ins for those defines 
>> are not being added by the compiler because it thinks the target 
>> doesn't support those operations.  Nonetheless, it doesn't use the 
>> substitutes that are obviously available.
>>
>> Can anybody recall anything about the way GCC is built under OE that 
>> would explain this?
>
> Not an OE problem.  See: 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62100
>
> I'm testing a patch locally and awaiting GCC maintainer comment before 
> proposing it for OE.

The GNU folks objected to the workaround and suggested that the behavior 
is evidence that libstdc++ was not built with the same ABI flags as the 
compiler uses by default.

I built the release version of gcc 4.9.1 on the beaglebone and confirmed 
that it works fine.

This is indeed an OE-Core bug.

I've just sent a series of cleanup patches to gcc that may make it 
easier to detect differences.  I've spent some time changing the 
"obvious" things and have been unable to figure out where the 
compilation is going wrong.  I'm hoping somebody else can help out; as 
C++11 becomes more prevalent the workaround recommended on stackoverflow 
and other sites will continue to propagate, and it's not the right solution.

Peter

Re: Yocto development with C++11 threads and gcc

[Khem Raj]

On Mon, Aug 11, 2014 at 12:02 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> The program below built on the target with the MACHINE=beaglebone gcc-4.9.1
> compiler from Yocto/OpenEmbedded poky master produces this error:
>
> beaglebone[52]$ g++ -std=c++1y -pthread test.cc && ./a.out
> starting
> joining
> pure virtual method called
> terminate called without an active exception
> Aborted (core dumped)
>
> When the program is recompiled with the defines for
> __GCC_HAVE_SYNC_COMPARE_AND_SWAP_X enabled as suggested at
> https://groups.google.com/d/msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ it
> works:
>
> beaglebone[53]$ g++ -std=c++1y -pthread test.cc && ./a.out
> starting
> joining
> doit
> done
>
> Preliminary analysis confirms that the built-ins for those defines are not
> being added by the compiler because it thinks the target doesn't support
> those operations.  Nonetheless, it doesn't use the substitutes that are
> obviously available.
>
> Can anybody recall anything about the way GCC is built under OE that would
> explain this?

Can you add -mcpu=cortex-a8 to your cmdline and see if it solves the problem ?

>
> Peter
>
> /* g++ -std=c++1y -pthread test.cc && ./a.out */
> #if 0
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4
> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
> #endif
>
> #include <iostream>
> #include <thread>
>
> void
> doit ()
> {
>   std::cerr << "doit\n";
> }
>
> int main (int argc,
>           char * argv [])
> {
>   std::cerr << "starting\n";
>   std::thread thr{doit};
>   std::cerr << "joining\n";
>   thr.join();
>   std::cerr << "done\n";
>   return 0;
> }
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/13/2014 04:18 PM, Khem Raj wrote:
> On Mon, Aug 11, 2014 at 12:02 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> The program below built on the target with the MACHINE=beaglebone gcc-4.9.1
>> compiler from Yocto/OpenEmbedded poky master produces this error:
>>
>> beaglebone[52]$ g++ -std=c++1y -pthread test.cc && ./a.out
>> starting
>> joining
>> pure virtual method called
>> terminate called without an active exception
>> Aborted (core dumped)
>>
>> When the program is recompiled with the defines for
>> __GCC_HAVE_SYNC_COMPARE_AND_SWAP_X enabled as suggested at
>> https://groups.google.com/d/msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ it
>> works:
>>
>> beaglebone[53]$ g++ -std=c++1y -pthread test.cc && ./a.out
>> starting
>> joining
>> doit
>> done
>>
>> Preliminary analysis confirms that the built-ins for those defines are not
>> being added by the compiler because it thinks the target doesn't support
>> those operations.  Nonetheless, it doesn't use the substitutes that are
>> obviously available.
>>
>> Can anybody recall anything about the way GCC is built under OE that would
>> explain this?
> Can you add -mcpu=cortex-a8 to your cmdline and see if it solves the problem ?

Yes, it does.  That's a good clue.

Peter

>> Peter
>>
>> /* g++ -std=c++1y -pthread test.cc && ./a.out */
>> #if 0
>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2
>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4
>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
>> #endif
>>
>> #include <iostream>
>> #include <thread>
>>
>> void
>> doit ()
>> {
>>    std::cerr << "doit\n";
>> }
>>
>> int main (int argc,
>>            char * argv [])
>> {
>>    std::cerr << "starting\n";
>>    std::thread thr{doit};
>>    std::cerr << "joining\n";
>>    thr.join();
>>    std::cerr << "done\n";
>>    return 0;
>> }
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/13/2014 04:23 PM, Peter A. Bigot wrote:
> On 08/13/2014 04:18 PM, Khem Raj wrote:
>> On Mon, Aug 11, 2014 at 12:02 PM, Peter A. Bigot <pab@pabigot.com> 
>> wrote:
>>> The program below built on the target with the MACHINE=beaglebone 
>>> gcc-4.9.1
>>> compiler from Yocto/OpenEmbedded poky master produces this error:
>>>
>>> beaglebone[52]$ g++ -std=c++1y -pthread test.cc && ./a.out
>>> starting
>>> joining
>>> pure virtual method called
>>> terminate called without an active exception
>>> Aborted (core dumped)
>>>
>>> When the program is recompiled with the defines for
>>> __GCC_HAVE_SYNC_COMPARE_AND_SWAP_X enabled as suggested at
>>> https://groups.google.com/d/msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ 
>>> it
>>> works:
>>>
>>> beaglebone[53]$ g++ -std=c++1y -pthread test.cc && ./a.out
>>> starting
>>> joining
>>> doit
>>> done
>>>
>>> Preliminary analysis confirms that the built-ins for those defines 
>>> are not
>>> being added by the compiler because it thinks the target doesn't 
>>> support
>>> those operations.  Nonetheless, it doesn't use the substitutes that are
>>> obviously available.
>>>
>>> Can anybody recall anything about the way GCC is built under OE that 
>>> would
>>> explain this?
>> Can you add -mcpu=cortex-a8 to your cmdline and see if it solves the 
>> problem ?
>
> Yes, it does.  That's a good clue.

More specifically, the build logs say gcc-runtime was built with:

arm-poky-linux-gnueabi-gcc  -march=armv7-a -mthumb-interwork 
-mfloat-abi=hard -mfpu=neon -mtune=cortex-a8

and using those flags instead of -mcpu=cortex-a8 also solves the problem.

So either gcc-runtime needs to stop using those flags, or gcc itself 
needs to default to them.  Or some other solution.

In any case, Khem can you run with this?  It'd be fixed a lot better 
that way....

Peter

>
> Peter
>
>>> Peter
>>>
>>> /* g++ -std=c++1y -pthread test.cc && ./a.out */
>>> #if 0
>>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
>>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_2
>>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4
>>> #define __GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
>>> #endif
>>>
>>> #include <iostream>
>>> #include <thread>
>>>
>>> void
>>> doit ()
>>> {
>>>    std::cerr << "doit\n";
>>> }
>>>
>>> int main (int argc,
>>>            char * argv [])
>>> {
>>>    std::cerr << "starting\n";
>>>    std::thread thr{doit};
>>>    std::cerr << "joining\n";
>>>    thr.join();
>>>    std::cerr << "done\n";
>>>    return 0;
>>> }
>>>
>>> -- 
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

Re: Yocto development with C++11 threads and gcc

[Khem Raj]

On Wed, Aug 13, 2014 at 2:36 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> In any case, Khem can you run with this?  It'd be fixed a lot better that
> way....

We do not configure target gcc with right matching cpu defaults,
atomic instruction strex/ldrex are only added after armv6 but defaults
for gcc if not specified is armv5t and hence it does not use the right
set as expected by libstdc++ which has been cross compiled. so while
you are at it and can reproduce it. Try to add

EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a", "
--with-cpu=armv7-a", "", d)}'

to gcc-target.inc and see if resulting gcc is any better

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/13/2014 05:05 PM, Khem Raj wrote:
> On Wed, Aug 13, 2014 at 2:36 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> In any case, Khem can you run with this?  It'd be fixed a lot better that
>> way....
> We do not configure target gcc with right matching cpu defaults,
> atomic instruction strex/ldrex are only added after armv6 but defaults
> for gcc if not specified is armv5t and hence it does not use the right
> set as expected by libstdc++ which has been cross compiled. so while
> you are at it and can reproduce it. Try to add
>
> EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a", "
> --with-cpu=armv7-a", "", d)}'
>
> to gcc-target.inc and see if resulting gcc is any better

I had to make it:

EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a", 
"--with-cpu=generic-armv7-a", "", d)}'

to get gcc to build but at runtime I then get:

beaglebone[16]$ g++ -std=c++11 -pthread test.cc && ./a.out
Assembler messages:
Error: unknown cpu `generic-armv7-a'
Error: unrecognized option -mcpu=generic-armv7-a

which indicates the flag's being passed to the assembler which doesn't 
recognize it even though g++ is happy with it.  I suppose we could hack 
binutils to substitute whatever spelling it wants to see.

(Also tried --with-cpu=arm7, but that generates assembler errors related 
to unsupported RM mode "bx lr").

The approach bothers me, though.  Instead of explicitly changing 
gcc-target to match gcc-runtime, shouldn't it be a general rule that 
gcc-runtime not apply OE-specific target flags that aren't going to be 
used by direct invocations of the compiler outside of the OE build 
environment?  That seems a little more robust, as the default target 
flags may be changed upstream or by bbappends within OE, and having to 
make them match in gcc-runtime as well would be a headache.

And would we need similar overrides for other architectures? There's 
something similar already in gcc-configure-common.inc for mips64.

Peter

Re: Yocto development with C++11 threads and gcc

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 2221 bytes --]

On Wednesday, August 13, 2014, Peter A. Bigot <pab@pabigot.com> wrote:

> On 08/13/2014 05:05 PM, Khem Raj wrote:
>
>> On Wed, Aug 13, 2014 at 2:36 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>
>>> In any case, Khem can you run with this?  It'd be fixed a lot better that
>>> way....
>>>
>> We do not configure target gcc with right matching cpu defaults,
>> atomic instruction strex/ldrex are only added after armv6 but defaults
>> for gcc if not specified is armv5t and hence it does not use the right
>> set as expected by libstdc++ which has been cross compiled. so while
>> you are at it and can reproduce it. Try to add
>>
>> EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a", "
>> --with-cpu=armv7-a", "", d)}'
>>
>> to gcc-target.inc and see if resulting gcc is any better
>>
>
> I had to make it:
>
> EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a",
> "--with-cpu=generic-armv7-a", "", d)}'
>
>
Sorry a typo there you   need  --with-arch

to get gcc to build but at runtime I then get:

>
> beaglebone[16]$ g++ -std=c++11 -pthread test.cc && ./a.out
> Assembler messages:
> Error: unknown cpu `generic-armv7-a'
> Error: unrecognized option -mcpu=generic-armv7-a
>
> which indicates the flag's being passed to the assembler which doesn't
> recognize it even though g++ is happy with it.  I suppose we could hack
> binutils to substitute whatever spelling it wants to see.
>
> (Also tried --with-cpu=arm7, but that generates assembler errors related
> to unsupported RM mode "bx lr").
>
> The approach bothers me, though.  Instead of explicitly changing
> gcc-target to match gcc-runtime, shouldn't it be a general rule that
> gcc-runtime not apply OE-specific target flags that aren't going to be used
> by direct invocations of the compiler outside of the OE build environment?
>  That seems a little more robust, as the default target flags may be
> changed upstream or by bbappends within OE, and having to make them match
> in gcc-runtime as well would be a headache.
>
> And would we need similar overrides for other architectures? There's
> something similar already in gcc-configure-common.inc for mips64.
>
> Peter
>

[-- Attachment #2: Type: text/html, Size: 2976 bytes --]

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

[-- Attachment #1: Type: text/plain, Size: 3325 bytes --]

On 08/13/2014 07:49 PM, Khem Raj wrote:
>
>
> On Wednesday, August 13, 2014, Peter A. Bigot <pab@pabigot.com 
> <mailto:pab@pabigot.com>> wrote:
>
>     On 08/13/2014 05:05 PM, Khem Raj wrote:
>
>         On Wed, Aug 13, 2014 at 2:36 PM, Peter A. Bigot
>         <pab@pabigot.com> wrote:
>
>             In any case, Khem can you run with this?  It'd be fixed a
>             lot better that
>             way....
>
>         We do not configure target gcc with right matching cpu defaults,
>         atomic instruction strex/ldrex are only added after armv6 but
>         defaults
>         for gcc if not specified is armv5t and hence it does not use
>         the right
>         set as expected by libstdc++ which has been cross compiled. so
>         while
>         you are at it and can reproduce it. Try to add
>
>         EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a", "
>         --with-cpu=armv7-a", "", d)}'
>
>         to gcc-target.inc and see if resulting gcc is any better
>
>
>     I had to make it:
>
>     EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a",
>     "--with-cpu=generic-armv7-a", "", d)}'
>
>
> Sorry a typo there you   need  --with-arch

OK, that works. So do we need to do the same thing for every 
TUNE_FEATURES element that ends up changing the value of -march= in 
TUNE_CCARGS which ends up getting passed into gcc-runtime?

If so would it be better to add a TUNE_ARCH setting to all the 
tune-foo.inc files and use that in both TUNE_CCARGS and the --with-arch= 
flag passed to gcc?  Just to avoid having this stuff hidden inside 
gcc-target.inc where it's pretty obscure.

>
> to get gcc to build but at runtime I then get:
>
>
>     beaglebone[16]$ g++ -std=c++11 -pthread test.cc && ./a.out
>     Assembler messages:
>     Error: unknown cpu `generic-armv7-a'
>     Error: unrecognized option -mcpu=generic-armv7-a
>
>     which indicates the flag's being passed to the assembler which
>     doesn't recognize it even though g++ is happy with it.  I suppose
>     we could hack binutils to substitute whatever spelling it wants to
>     see.
>
>     (Also tried --with-cpu=arm7, but that generates assembler errors
>     related to unsupported RM mode "bx lr").
>
>     The approach bothers me, though.  Instead of explicitly changing
>     gcc-target to match gcc-runtime, shouldn't it be a general rule
>     that gcc-runtime not apply OE-specific target flags that aren't
>     going to be used by direct invocations of the compiler outside of
>     the OE build environment?  That seems a little more robust, as the
>     default target flags may be changed upstream or by bbappends
>     within OE, and having to make them match in gcc-runtime as well
>     would be a headache.
>

Just to record one reason why this isn't trivial: although it's possible 
to strip ${TARGET_CC_ARCH} from ${CXX}, doing so removes 
-mfloat-abi=hard which makes gcc-runtime try to build a library that 
supports soft float, and the compiler didn't generate the necessary 
gnu/stubs-soft.h header for that.

>
>     And would we need similar overrides for other architectures?
>     There's something similar already in gcc-configure-common.inc for
>     mips64.
>
>     Peter
>


[-- Attachment #2: Type: text/html, Size: 5214 bytes --]

Re: Yocto development with C++11 threads and gcc

[Khem Raj]

On Wed, Aug 13, 2014 at 6:55 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a",
>> "--with-cpu=generic-armv7-a", "", d)}'
>>
>
> Sorry a typo there you   need  --with-arch
>
>
> OK, that works. So do we need to do the same thing for every TUNE_FEATURES
> element that ends up changing the value of -march= in TUNE_CCARGS which ends
> up getting passed into gcc-runtime?

Not really but arm v6+ is an exception here due to reasons explained
earlier. gcc as a target package is a bit
different than rest of them since it has mind of its own when it comes
to configuring it. It encodes certain
defaults into its driver etc. based on the configure parameters.
Ideally when we pass the flags via CFLAGS
it does it for most of packages but not for gcc.

>
> If so would it be better to add a TUNE_ARCH setting to all the tune-foo.inc
> files and use that in both TUNE_CCARGS and the --with-arch= flag passed to
> gcc?  Just to avoid having this stuff hidden inside gcc-target.inc where
> it's pretty obscure.

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/14/2014 12:32 AM, Khem Raj wrote:
> On Wed, Aug 13, 2014 at 6:55 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>> EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a",
>>> "--with-cpu=generic-armv7-a", "", d)}'
>>>
>> Sorry a typo there you   need  --with-arch
>>
>>
>> OK, that works. So do we need to do the same thing for every TUNE_FEATURES
>> element that ends up changing the value of -march= in TUNE_CCARGS which ends
>> up getting passed into gcc-runtime?
> Not really but arm v6+ is an exception here due to reasons explained
> earlier. gcc as a target package is a bit
> different than rest of them since it has mind of its own when it comes
> to configuring it. It encodes certain
> defaults into its driver etc. based on the configure parameters.
> Ideally when we pass the flags via CFLAGS
> it does it for most of packages but not for gcc.

tl;dr: the proposed fix to gcc-target works for my example but I'm 
concerned it's a bandaid that doesn't address a more long-term risk.

I understand gcc-target is different.  I frame the issue as: bitbake 
(via toolchain-scripts.bbclass) ensures OE builds its applications with 
TARGET_CC_ARCH flags such as -march=FOO, but building gcc itself 
requires EXTRA_OECONF to pass different flags such as --with-arch=FOO 
which set the default behavior for the target compiler.

There are actually a bunch of these gcc configuration options:

  --with-schedule=cpu
  --with-arch=cpu
  --with-arch-32=cpu
  --with-arch-64=cpu
  --with-tune=cpu
  --with-tune-32=cpu
  --with-tune-64=cpu
  --with-abi=abi
  --with-fpu=type
  --with-float=type

all of which affect the default for the corresponding -m flags, and any 
of which may affect each other within gcc's configure script.

It sounds like the intent is that the default behavior of the target gcc 
should match what bitbake is passing via TARGET_CC_ARCH.  Of course, if 
that were guaranteed there wouldn't be any need for OE to apply 
TARGET_CC_ARCH at all, but I wouldn't suggest changing that.

My concern is that gcc-runtime invokes configure in subdirectories of 
gcc's source tree, bypassing the top-level configure where some of the 
translation from (e.g.) --with-arch=FOO to -march=FOO is done.  It then 
builds the libraries with TARGET_CC_ARCH flags appended to CC, CXX, and CPP.

What this means is that the libraries built by gcc-runtime use 
TARGET_CC_ARCH settings that don't necessarily match the target 
compiler's defaults, and that ABI conflicts can result by linking in 
those libraries when the non-default settings were absent in non-OE 
application builds.  ABI can only be "guaranteed" if every one of the 
-mFOO=BAR passed in TARGET_CC_ARCH (*or defaulted by the compiler*) has 
a corresponding -with-FOO=BAR option passed to (*or inferred by*) gcc's 
configure.

That's a pretty strong assumption to make.

It may be that this can be worked around for the specific case I raised 
by explicitly adding --with-arch=armv7-a to gcc-target's EXTRA_OECONF. I 
do have to wonder whether the same should be done for any of armv7 
armv7-m armv7-r armv7e-m armv7ve armv8-a armv8-a+crc which are other 
-march options that are armv6+, and whether there are other ABI issues 
that might be hiding now or in the future because TARGET_CC_ARCH makes 
more assumptions than gcc-target does.

The solution I propose is to rework gcc-runtime's override of CC/CXX/CPP 
so the libraries are built the same way they would be if they had been 
built during gcc-target.

 From initial attempts this won't be easy to do.  I'd be happy to keep 
trying if this worries other people, but if I'm being too picky I'll 
just suck it up and move on.

Peter

Re: Yocto development with C++11 threads and gcc

[Richard Purdie]

On Thu, 2014-08-14 at 04:15 -0500, Peter A. Bigot wrote:
> What this means is that the libraries built by gcc-runtime use 
> TARGET_CC_ARCH settings that don't necessarily match the target 
> compiler's defaults, and that ABI conflicts can result by linking in 
> those libraries when the non-default settings were absent in non-OE 
> application builds.  ABI can only be "guaranteed" if every one of the 
> -mFOO=BAR passed in TARGET_CC_ARCH (*or defaulted by the compiler*) has 
> a corresponding -with-FOO=BAR option passed to (*or inferred by*) gcc's 
> configure.
> 
> That's a pretty strong assumption to make.
> 
> It may be that this can be worked around for the specific case I raised 
> by explicitly adding --with-arch=armv7-a to gcc-target's EXTRA_OECONF. I 
> do have to wonder whether the same should be done for any of armv7 
> armv7-m armv7-r armv7e-m armv7ve armv8-a armv8-a+crc which are other 
> -march options that are armv6+, and whether there are other ABI issues 
> that might be hiding now or in the future because TARGET_CC_ARCH makes 
> more assumptions than gcc-target does.
> 
> The solution I propose is to rework gcc-runtime's override of CC/CXX/CPP 
> so the libraries are built the same way they would be if they had been 
> built during gcc-target.
> 
>  From initial attempts this won't be easy to do.  I'd be happy to keep 
> trying if this worries other people, but if I'm being too picky I'll 
> just suck it up and move on.

Its a valid concern, I just don't think anyone else has run into the
kinds of issues you're seeing :/. We don't want to diverge too far from
upstream gcc, equally, we need a working compiler...

Cheers,

Richard

Re: Yocto development with C++11 threads and gcc

[Peter A. Bigot]

On 08/14/2014 04:40 PM, Richard Purdie wrote:
> On Thu, 2014-08-14 at 04:15 -0500, Peter A. Bigot wrote:
>> What this means is that the libraries built by gcc-runtime use
>> TARGET_CC_ARCH settings that don't necessarily match the target
>> compiler's defaults, and that ABI conflicts can result by linking in
>> those libraries when the non-default settings were absent in non-OE
>> application builds.  ABI can only be "guaranteed" if every one of the
>> -mFOO=BAR passed in TARGET_CC_ARCH (*or defaulted by the compiler*) has
>> a corresponding -with-FOO=BAR option passed to (*or inferred by*) gcc's
>> configure.
>>
>> That's a pretty strong assumption to make.
>>
>> It may be that this can be worked around for the specific case I raised
>> by explicitly adding --with-arch=armv7-a to gcc-target's EXTRA_OECONF. I
>> do have to wonder whether the same should be done for any of armv7
>> armv7-m armv7-r armv7e-m armv7ve armv8-a armv8-a+crc which are other
>> -march options that are armv6+, and whether there are other ABI issues
>> that might be hiding now or in the future because TARGET_CC_ARCH makes
>> more assumptions than gcc-target does.
>>
>> The solution I propose is to rework gcc-runtime's override of CC/CXX/CPP
>> so the libraries are built the same way they would be if they had been
>> built during gcc-target.
>>
>>   From initial attempts this won't be easy to do.  I'd be happy to keep
>> trying if this worries other people, but if I'm being too picky I'll
>> just suck it up and move on.
> Its a valid concern, I just don't think anyone else has run into the
> kinds of issues you're seeing :/.

I think it's more that somebody found a workaround and posted it on 
stackoverflow so nobody reported back to OE.  A potential client I 
talked to yesterday mentioned having run into this exact problem a month 
or so ago, and the earliest mention of it I've found was from a year ago 
[1].  As use of threading in C++11 becomes more common I'd expect it to 
have had increased visibility.

I'm content with the solution in my v2 gcc patch series, but I expect 
someday the underlying cause will re-appear as new processors have new 
features with ABI impacts due to gcc target optimization. Hopefully when 
that happens the discussion here will be of some help in identifying the 
problem.

Peter

[1] 
https://groups.google.com/forum/#!msg/automatak-dnp3/Jisp_zGhd5I/ck_Cj6nO8joJ

Re: Yocto development with C++11 threads and gcc

[Khem Raj]

On Thu, Aug 14, 2014 at 3:00 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> I'm content with the solution in my v2 gcc patch series, but I expect
> someday the underlying cause will re-appear as new processors have new
> features with ABI impacts due to gcc target optimization. Hopefully when
> that happens the discussion here will be of some help in identifying the
> problem.

general usecase on arm thus far is to use cross compilers and I think
we are good there. target compilation is not a norm but this might
change soon but then we can propose to change defaults in gcc to use
armv6+, we build arm native gcc using canadian cross method, thats
kind of different than usual known gcc compilation methods so there
may be less favors for this use case.

Re: Yocto development with C++11 threads and gcc

[Khem Raj]

On Thu, Aug 14, 2014 at 2:15 AM, Peter A. Bigot <pab@pabigot.com> wrote:
> On 08/14/2014 12:32 AM, Khem Raj wrote:
>>
>> On Wed, Aug 13, 2014 at 6:55 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>>>
>>>> EXTRA_OECONF += '${@bb.utils.contains("TUNE_FEATURES", "armv7a",
>>>> "--with-cpu=generic-armv7-a", "", d)}'
>>>>
>>> Sorry a typo there you   need  --with-arch
>>>
>>>
>>> OK, that works. So do we need to do the same thing for every
>>> TUNE_FEATURES
>>> element that ends up changing the value of -march= in TUNE_CCARGS which
>>> ends
>>> up getting passed into gcc-runtime?
>>
>> Not really but arm v6+ is an exception here due to reasons explained
>> earlier. gcc as a target package is a bit
>> different than rest of them since it has mind of its own when it comes
>> to configuring it. It encodes certain
>> defaults into its driver etc. based on the configure parameters.
>> Ideally when we pass the flags via CFLAGS
>> it does it for most of packages but not for gcc.
>
>
> tl;dr: the proposed fix to gcc-target works for my example but I'm concerned
> it's a bandaid that doesn't address a more long-term risk.
>
> I understand gcc-target is different.  I frame the issue as: bitbake (via
> toolchain-scripts.bbclass) ensures OE builds its applications with
> TARGET_CC_ARCH flags such as -march=FOO, but building gcc itself requires
> EXTRA_OECONF to pass different flags such as --with-arch=FOO which set the
> default behavior for the target compiler.
>
> There are actually a bunch of these gcc configuration options:
>
>  --with-schedule=cpu
>  --with-arch=cpu
>  --with-arch-32=cpu
>  --with-arch-64=cpu
>  --with-tune=cpu
>  --with-tune-32=cpu
>  --with-tune-64=cpu
>  --with-abi=abi
>  --with-fpu=type
>  --with-float=type
>
> all of which affect the default for the corresponding -m flags, and any of
> which may affect each other within gcc's configure script.
>
> It sounds like the intent is that the default behavior of the target gcc
> should match what bitbake is passing via TARGET_CC_ARCH.  Of course, if that
> were guaranteed there wouldn't be any need for OE to apply TARGET_CC_ARCH at
> all, but I wouldn't suggest changing that.
>
> My concern is that gcc-runtime invokes configure in subdirectories of gcc's
> source tree, bypassing the top-level configure where some of the translation
> from (e.g.) --with-arch=FOO to -march=FOO is done.  It then builds the
> libraries with TARGET_CC_ARCH flags appended to CC, CXX, and CPP.
>
> What this means is that the libraries built by gcc-runtime use
> TARGET_CC_ARCH settings that don't necessarily match the target compiler's
> defaults, and that ABI conflicts can result by linking in those libraries
> when the non-default settings were absent in non-OE application builds.  ABI
> can only be "guaranteed" if every one of the -mFOO=BAR passed in
> TARGET_CC_ARCH (*or defaulted by the compiler*) has a corresponding
> -with-FOO=BAR option passed to (*or inferred by*) gcc's configure.
>
> That's a pretty strong assumption to make.
>
> It may be that this can be worked around for the specific case I raised by
> explicitly adding --with-arch=armv7-a to gcc-target's EXTRA_OECONF. I do
> have to wonder whether the same should be done for any of armv7 armv7-m
> armv7-r armv7e-m armv7ve armv8-a armv8-a+crc which are other -march options
> that are armv6+, and whether there are other ABI issues that might be hiding
> now or in the future because TARGET_CC_ARCH makes more assumptions than
> gcc-target does.
>
> The solution I propose is to rework gcc-runtime's override of CC/CXX/CPP so
> the libraries are built the same way they would be if they had been built
> during gcc-target.
>

we are actually pulling out the target runtime out of gut of gcc-cross
and then doing the match with target gcc
so as long as we make target gcc recipe confirm to this we are good. I
dont feel like we need to engineer a general solution here. Since what
we are addressing is an anamoly not norm. So lets keep it that way.
meaning patch it for arm where needed.
one day all  < armv5 architectures will die out and we will be a happy
lot :) oh wait then aarch64-32 might spring another ABI concern who
knows

> From initial attempts this won't be easy to do.  I'd be happy to keep trying
> if this worries other people, but if I'm being too picky I'll just suck it
> up and move on.
>
> Peter