[Buildroot] Two or three stages gcc build?

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Hello Khem,

(This e-mail is Cc'ed to the crossgcc@ and the buildroot@ mailing lists)

We started discussing on IRC about whether a three stage or a two stage
gcc build was needed, but seems like due to timezone issues, we're not
able to discuss simultaneously, so I'm trying to do things by e-mail. I
would like to understand a little bit better since when we can get back
to a two stages gcc build process instead of the three stages build
process that was needed since the introduction of NPTL support.

Currently, what we have in Buildroot is:

 1. Build gcc initial (--without-headers --with-newlib --disable-shared)

 2. Configure uClibc, install headers, crti.o, crtn.o, crt1.o, install
    a fake libc.so and libm.so (just empty C files being compiled as a
    share library)

 3. Build gcc intermediate (no special options passed)

 4. Build uClibc and install it.

 5. Build gcc final

Apparently, this three stage build process is no longer needed. In
OE-Core, you did the following commit:

commit b0faebd1f07e1616004bd19664395932e7c2c48f
Author: Khem Raj <raj.khem@gmail.com>
Date:   Wed Aug 15 23:12:51 2012 -0700

    gcc-cross: Make gcc-cross-initial as the only intermediate gcc stage
    
    Now glibc can be compiled with gcc-cross-initial therefore prepare
    the stage to drop gcc-cross-intermediate
    
    Also drop arm-nolibfloat.patch should not be needed anymore
    half of changes in this patch are meant for OABI which we dont
    use anymore
    
    (From OE-Core rev: 30617bde61a3b0a0944b49a0c9fb7159dacbb19f)
    
    Signed-off-by: Khem Raj <raj.khem@gmail.com>
    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

So according to this, the fact that we can go back to a two stages
build seems to be related to glibc. So what happens to uClibc in this
case?

In fe9a576e8d27460468dfe5eac90aad49ab26a8d3 (gcc-cross-intermediate,
gcc-crosssdk-intermediate: Remove) you completely remove the
intermediate recipes, so I assume they are really no longer needed,
even by the uClibc build, since I believe OE-core also supports uClibc
builds (thanks to your work).

Could you explain in more details which version of gcc or glibc/uClibc
made it possible to go back to a 2-stages build? Both Yann E. Morin
(for Crosstool-NG) and myself (for Buildroot) are interested in
understanding this in order to improve those tools. Also, what should
be the two stages now? Just:

 1. Build gcc initial (no C library at all)
 2. Build C library completely
 3. Build gcc final

 ?

Thanks for your insights,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Dear Khem Raj,

On Thu, 5 Sep 2013 10:31:46 -0700, Khem Raj wrote:

> I am late in reply but here it is.

No problem. Thanks for your reply.


> > In fe9a576e8d27460468dfe5eac90aad49ab26a8d3 (gcc-cross-intermediate,
> > gcc-crosssdk-intermediate: Remove) you completely remove the
> > intermediate recipes, so I assume they are really no longer needed,
> > even by the uClibc build, since I believe OE-core also supports uClibc
> > builds (thanks to your work).
> 
> thats correct. See the code around where limits.h is installed by gcc initial
> uclibc was less of an issue, glibc was more until last year when a lot of cleanup
> was done especially where build time dependency on libgcc_s and libgcc_eh was averted
> in cross gcc and buildroot you can do the same like i have done for OE I can help out

In the mean time, I've experimented a bit and actually implemented it.
The patch hasn't been merged yet in Buildroot, but it has been posted:
http://lists.busybox.net/pipermail/buildroot/2013-September/077344.html.

Don't hesitate to have a look and tell me what you think.

Thanks!

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Dear Khem Raj,

On Fri, 6 Sep 2013 09:59:34 -0700, Khem Raj wrote:

> > In the mean time, I've experimented a bit and actually implemented it.
> > The patch hasn't been merged yet in Buildroot, but it has been posted:
> > http://lists.busybox.net/pipermail/buildroot/2013-September/077344.html.
> > 
> > Don't hesitate to have a look and tell me what you think.
> 
> OK. The patch looks ok to me. one thing
> 
> Do you need to support multiple toolchains ? and how far do you want to go back ? I would much rather not
> have THREE_STAGE_BUILD options at all.

We do need to support building gcc 4.2.x, because that's the only
version for which thee AVR32 architecture is supported. I'm hoping that
in a year or two we can drop AVR32 support (the architecture has been
deprecated by Atmel), and therefore simplify a few things, but for now,
I believe we need to be able to support older gcc versions than 4.7/4.8.

Thanks for having a look at the patch!

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Dear Martin Guy,

On Sat, 7 Sep 2013 13:45:45 +0200, Martin Guy wrote:
> On 07/09/2013, Thomas Petazzoni <thomas.petazzoni@free-electrons.com> wrote:
> > We do need to support building gcc 4.2.x, because that's the only
> > version for which thee AVR32 architecture is supported.
> 
>  A few months ago I incorporated and improved the new AVR32 support
> for gcc 4.3 and 4.4. Since AVR's dirty patches also affect code
> generation on other non-AVR32 processors, I chose to do the
> development outside the main branch but the results, a fork of ct-ng,
> are visible at https://spaces.atmel.com/gf/project/ct-ng/

Yes, I have seen that (maybe you announced it already on the ct-ng
mailing list). I personally don't care enough about AVR32 to spend some
time integrating this in Buildroot, but I've Cc'ed Simon Dawson who
might be interested.

That said, even gcc 4.4 is relatively "old" by these days standards, so
I am not sure it would make a huge difference compared to keeping the
existing gcc 4.2.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Dear Khem Raj,

On Fri, 6 Sep 2013 09:59:34 -0700, Khem Raj wrote:

> > In the mean time, I've experimented a bit and actually implemented it.
> > The patch hasn't been merged yet in Buildroot, but it has been posted:
> > http://lists.busybox.net/pipermail/buildroot/2013-September/077344.html.
> > 
> > Don't hesitate to have a look and tell me what you think.
> 
> OK. The patch looks ok to me. one thing

After merging the patch, we got a report that it breaks SSP support,
see [1]. After investigation, depending on the architecture, the
__stack_chk_fail symbol maybe be part of the TLS (it's the case on x86,
but on ARM, for example).

However, since we're now only doing gcc-initial -> uclibc -> gcc-final,
and gcc-initial has no thread/TLS support, it breaks the build of
uClibc when SSP support is enabled (in such a case, uClibc is built
with -fstack-protector, which creates references to __stack_chk_fail,
but this symbol isn't available because we don't have TLS support).

Have you already seen this problem? What solution do you suggest?

Thanks!

Thomas

[1]
http://buildroot-busybox.2317881.n4.nabble.com/uClibc-fails-to-build-with-stack-smash-protection-td51478.html
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Khem,

Any input on the below questions? :-)

Thanks!

Thomas

On Thu, 19 Sep 2013 07:02:55 +0200, Thomas Petazzoni wrote:
> Dear Khem Raj,
> 
> On Fri, 6 Sep 2013 09:59:34 -0700, Khem Raj wrote:
> 
> > > In the mean time, I've experimented a bit and actually implemented it.
> > > The patch hasn't been merged yet in Buildroot, but it has been posted:
> > > http://lists.busybox.net/pipermail/buildroot/2013-September/077344.html.
> > > 
> > > Don't hesitate to have a look and tell me what you think.
> > 
> > OK. The patch looks ok to me. one thing
> 
> After merging the patch, we got a report that it breaks SSP support,
> see [1]. After investigation, depending on the architecture, the
> __stack_chk_fail symbol maybe be part of the TLS (it's the case on x86,
> but on ARM, for example).
> 
> However, since we're now only doing gcc-initial -> uclibc -> gcc-final,
> and gcc-initial has no thread/TLS support, it breaks the build of
> uClibc when SSP support is enabled (in such a case, uClibc is built
> with -fstack-protector, which creates references to __stack_chk_fail,
> but this symbol isn't available because we don't have TLS support).
> 
> Have you already seen this problem? What solution do you suggest?
> 
> Thanks!
> 
> Thomas
> 
> [1]
> http://buildroot-busybox.2317881.n4.nabble.com/uClibc-fails-to-build-with-stack-smash-protection-td51478.html



-- 
Thomas Petazzoni, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[Buildroot] Two or three stages gcc build?

[soochon radee]

Hi,

I am also seeing the two-phase build break eglibc trying to link
SSP-related symbols on 64-bit targets (powerpc/970, x86_64/core2) in
libc/resolv. This happens even if SSP is enabled in .config

I've been digging around gcc-initial's configure arguments and directories
passed around to eglibc's configure script to no avail. But, I've been able
to make it work by forcing eglibc to assume no SSP support with the
following patch. Hopefully this will point a way to a better solution.

diff --git a/package/glibc/glibc.mk b/package/glibc/glibc.mk
index 4dd2e63..1e8b325 100644
--- a/package/glibc/glibc.mk
+++ b/package/glibc/glibc.mk
@@ -63,6 +63,7 @@ define GLIBC_CONFIGURE_CMDS
                CXXFLAGS="-O2 $(GLIBC_EXTRA_CFLAGS)" \
                $(SHELL) $(@D)/$(GLIBC_SRC_SUBDIR)/configure \
                ac_cv_path_BASH_SHELL=/bin/bash \
+               libc_cv_ssp=no \
                libc_cv_forced_unwind=yes \
                --target=$(GNU_TARGET_NAME) \
                --host=$(GNU_TARGET_NAME) \

Thanks,
Soochon

> Dear Khem Raj,
>
> On Fri, 6 Sep 2013 09:59:34 -0700, Khem Raj wrote:
>
> > > In the mean time, I've experimented a bit and actually implemented it.
> > > The patch hasn't been merged yet in Buildroot, but it has been posted:
> > >
http://lists.busybox.net/pipermail/buildroot/2013-September/077344.html.
> > >
> > > Don't hesitate to have a look and tell me what you think.
> >
> > OK. The patch looks ok to me. one thing
>
> After merging the patch, we got a report that it breaks SSP support,
> see [1]. After investigation, depending on the architecture, the
> __stack_chk_fail symbol maybe be part of the TLS (it's the case on x86,
> but on ARM, for example).
>
> However, since we're now only doing gcc-initial -> uclibc -> gcc-final,
> and gcc-initial has no thread/TLS support, it breaks the build of
> uClibc when SSP support is enabled (in such a case, uClibc is built
> with -fstack-protector, which creates references to __stack_chk_fail,
> but this symbol isn't available because we don't have TLS support).
>
> Have you already seen this problem? What solution do you suggest?
>
> Thanks!
>
> Thomas
>
> [1]
>
http://buildroot-busybox.2317881.n4.nabble.com/uClibc-fails-to-build-with-stack-smash-protection-td51478.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20130919/4dd2c02c/attachment-0001.html>

[Buildroot] Two or three stages gcc build?

[Thomas Petazzoni]

Dear soochon radee,

On Thu, 19 Sep 2013 13:02:33 -0400, soochon radee wrote:

> I am also seeing the two-phase build break eglibc trying to link
> SSP-related symbols on 64-bit targets (powerpc/970, x86_64/core2) in
> libc/resolv. This happens even if SSP is enabled in .config
> 
> I've been digging around gcc-initial's configure arguments and directories
> passed around to eglibc's configure script to no avail. But, I've been able
> to make it work by forcing eglibc to assume no SSP support with the
> following patch. Hopefully this will point a way to a better solution.

Well, I've been able to fix the problem on ARM by explicitly telling
gcc-initial that the libc *will* have SSP support. However, it doesn't
work on x86 because SSP requires TLS support, which isn't provided by
gcc-initial. I am not yet sure how to fix this (enable threads in
gcc-initial, if it is even possible? switch to a three stages build
when SSP is enabled?).

I'll continue experimenting around with this.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com