[PATCH] arm64: kill off the libgcc dependency

[PATCH] arm64: kill off the libgcc dependency

[Kevin Hao]

The arm64 doesn't depend on the libgcc to build the vmlinux image.

Signed-off-by: Kevin Hao <haokexin@gmail.com>
---
 arch/arm64/Makefile | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 1c43cec971b5..a20c28348be4 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -15,8 +15,6 @@ CPPFLAGS_vmlinux.lds = -DTEXT_OFFSET=$(TEXT_OFFSET)
 OBJCOPYFLAGS	:=-O binary -R .note -R .note.gnu.build-id -R .comment -S
 GZFLAGS		:=-9
 
-LIBGCC 		:= $(shell $(CC) $(KBUILD_CFLAGS) -print-libgcc-file-name)
-
 KBUILD_DEFCONFIG := defconfig
 
 KBUILD_CFLAGS	+= -mgeneral-regs-only
@@ -50,7 +48,6 @@ core-$(CONFIG_KVM) += arch/arm64/kvm/
 core-$(CONFIG_XEN) += arch/arm64/xen/
 core-$(CONFIG_CRYPTO) += arch/arm64/crypto/
 libs-y		:= arch/arm64/lib/ $(libs-y)
-libs-y		+= $(LIBGCC)
 libs-$(CONFIG_EFI_STUB) += drivers/firmware/efi/libstub/
 
 # Default target when executing plain make
-- 
1.9.3

[PATCH] arm64: kill off the libgcc dependency

[Will Deacon]

On Wed, Jan 14, 2015 at 10:38:54AM +0000, Kevin Hao wrote:
> The arm64 doesn't depend on the libgcc to build the vmlinux image.

Yes it does; we make use of __builtin_* for our bitops, for example.
The compiler guys don't guarantee that these will be inlined.

Will

[PATCH] arm64: kill off the libgcc dependency

[Arnd Bergmann]

On Wednesday 14 January 2015 10:48:26 Will Deacon wrote:
> On Wed, Jan 14, 2015 at 10:38:54AM +0000, Kevin Hao wrote:
> > The arm64 doesn't depend on the libgcc to build the vmlinux image.
> 
> Yes it does; we make use of __builtin_* for our bitops, for example.
> The compiler guys don't guarantee that these will be inlined.

Is that an architecture-specific statement?

The debate has been going on for ages in Linux, most architecture
maintainers however decided against using libgcc and copy the functions
that are actually needed into the kernel sources.

	Arnd

[PATCH] arm64: kill off the libgcc dependency

[Will Deacon]

On Wed, Jan 14, 2015 at 11:35:22AM +0000, Arnd Bergmann wrote:
> On Wednesday 14 January 2015 10:48:26 Will Deacon wrote:
> > On Wed, Jan 14, 2015 at 10:38:54AM +0000, Kevin Hao wrote:
> > > The arm64 doesn't depend on the libgcc to build the vmlinux image.
> > 
> > Yes it does; we make use of __builtin_* for our bitops, for example.
> > The compiler guys don't guarantee that these will be inlined.
> 
> Is that an architecture-specific statement?

I think the inlining guarantees apply to all architectures, but it affects
the arm64 kernel because we include <asm-generic/bitops/builtin-*>.

> The debate has been going on for ages in Linux, most architecture
> maintainers however decided against using libgcc and copy the functions
> that are actually needed into the kernel sources.

What's the benefit of copying them?

Will

[PATCH] arm64: kill off the libgcc dependency

[Arnd Bergmann]

On Wednesday 14 January 2015 11:48:27 Will Deacon wrote:
> On Wed, Jan 14, 2015 at 11:35:22AM +0000, Arnd Bergmann wrote:
> > On Wednesday 14 January 2015 10:48:26 Will Deacon wrote:
> > > On Wed, Jan 14, 2015 at 10:38:54AM +0000, Kevin Hao wrote:
> > > > The arm64 doesn't depend on the libgcc to build the vmlinux image.
> > > 
> > > Yes it does; we make use of __builtin_* for our bitops, for example.
> > > The compiler guys don't guarantee that these will be inlined.
> > 
> > Is that an architecture-specific statement?
> 
> I think the inlining guarantees apply to all architectures, but it affects
> the arm64 kernel because we include <asm-generic/bitops/builtin-*>.

Is it realistic to expect those to ever be implemented out of line?
I would expect them to behave like assembler intrinsics, at least
with the normal optimization level.

> > The debate has been going on for ages in Linux, most architecture
> > maintainers however decided against using libgcc and copy the functions
> > that are actually needed into the kernel sources.
> 
> What's the benefit of copying them?

You have a better control over what is included. On 32-bit architectures,
the 64-bit division is notorious here, we really don't want to accidentally
call the libgcc implementation but instead have our own do_div() helper.

Similarly I'd think that if we get a compiler that uses an external
__ffs implementation, we should implement an inline assembly instead
of taking the libgcc version.

Another problem that has happened in the past is libgcc functions that
in turn call glibc functions. To avoid that, you'd really want to build
the kernel with -ffreestanding, but that opens another can of worms.

	Arnd

[PATCH] arm64: kill off the libgcc dependency

[Ard Biesheuvel]

On 14 January 2015 at 12:38, Arnd Bergmann <arnd@arndb.de> wrote:
> On Wednesday 14 January 2015 11:48:27 Will Deacon wrote:
>> On Wed, Jan 14, 2015 at 11:35:22AM +0000, Arnd Bergmann wrote:
>> > On Wednesday 14 January 2015 10:48:26 Will Deacon wrote:
>> > > On Wed, Jan 14, 2015 at 10:38:54AM +0000, Kevin Hao wrote:
>> > > > The arm64 doesn't depend on the libgcc to build the vmlinux image.
>> > >
>> > > Yes it does; we make use of __builtin_* for our bitops, for example.
>> > > The compiler guys don't guarantee that these will be inlined.
>> >
>> > Is that an architecture-specific statement?
>>
>> I think the inlining guarantees apply to all architectures, but it affects
>> the arm64 kernel because we include <asm-generic/bitops/builtin-*>.
>
> Is it realistic to expect those to ever be implemented out of line?
> I would expect them to behave like assembler intrinsics, at least
> with the normal optimization level.
>
>> > The debate has been going on for ages in Linux, most architecture
>> > maintainers however decided against using libgcc and copy the functions
>> > that are actually needed into the kernel sources.
>>
>> What's the benefit of copying them?
>
> You have a better control over what is included. On 32-bit architectures,
> the 64-bit division is notorious here, we really don't want to accidentally
> call the libgcc implementation but instead have our own do_div() helper.
>
> Similarly I'd think that if we get a compiler that uses an external
> __ffs implementation, we should implement an inline assembly instead
> of taking the libgcc version.
>
> Another problem that has happened in the past is libgcc functions that
> in turn call glibc functions. To avoid that, you'd really want to build
> the kernel with -ffreestanding, but that opens another can of worms.
>

IMO libgcc.a cannot be used at all in the kernel, as it is not built
with -mgeneral-regs-only so we have no guarantee that it will leave
the NEON registers alone.

-- 
Ard.

[PATCH] arm64: kill off the libgcc dependency

[Will Deacon]

On Wed, Jan 14, 2015 at 12:52:49PM +0000, Ard Biesheuvel wrote:
> On 14 January 2015 at 12:38, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Wednesday 14 January 2015 11:48:27 Will Deacon wrote:
> >> On Wed, Jan 14, 2015 at 11:35:22AM +0000, Arnd Bergmann wrote:
> >> > On Wednesday 14 January 2015 10:48:26 Will Deacon wrote:
> >> > > On Wed, Jan 14, 2015 at 10:38:54AM +0000, Kevin Hao wrote:
> >> > > > The arm64 doesn't depend on the libgcc to build the vmlinux image.
> >> > >
> >> > > Yes it does; we make use of __builtin_* for our bitops, for example.
> >> > > The compiler guys don't guarantee that these will be inlined.
> >> >
> >> > Is that an architecture-specific statement?
> >>
> >> I think the inlining guarantees apply to all architectures, but it affects
> >> the arm64 kernel because we include <asm-generic/bitops/builtin-*>.
> >
> > Is it realistic to expect those to ever be implemented out of line?
> > I would expect them to behave like assembler intrinsics, at least
> > with the normal optimization level.
> >
> >> > The debate has been going on for ages in Linux, most architecture
> >> > maintainers however decided against using libgcc and copy the functions
> >> > that are actually needed into the kernel sources.
> >>
> >> What's the benefit of copying them?
> >
> > You have a better control over what is included. On 32-bit architectures,
> > the 64-bit division is notorious here, we really don't want to accidentally
> > call the libgcc implementation but instead have our own do_div() helper.
> >
> > Similarly I'd think that if we get a compiler that uses an external
> > __ffs implementation, we should implement an inline assembly instead
> > of taking the libgcc version.
> >
> > Another problem that has happened in the past is libgcc functions that
> > in turn call glibc functions. To avoid that, you'd really want to build
> > the kernel with -ffreestanding, but that opens another can of worms.
> >
> 
> IMO libgcc.a cannot be used at all in the kernel, as it is not built
> with -mgeneral-regs-only so we have no guarantee that it will leave
> the NEON registers alone.

That's a very good point. In which case, we'd need to extend this patch
to implement the bitops that we currently rely on the __builtins for.

Will

[PATCH] arm64: kill off the libgcc dependency

[Russell King - ARM Linux]

On Wed, Jan 14, 2015 at 12:52:49PM +0000, Ard Biesheuvel wrote:
> On 14 January 2015 at 12:38, Arnd Bergmann <arnd@arndb.de> wrote:
> > You have a better control over what is included. On 32-bit architectures,
> > the 64-bit division is notorious here, we really don't want to accidentally
> > call the libgcc implementation but instead have our own do_div() helper.
> >
> > Similarly I'd think that if we get a compiler that uses an external
> > __ffs implementation, we should implement an inline assembly instead
> > of taking the libgcc version.
> >
> > Another problem that has happened in the past is libgcc functions that
> > in turn call glibc functions. To avoid that, you'd really want to build
> > the kernel with -ffreestanding, but that opens another can of worms.
> >
> 
> IMO libgcc.a cannot be used at all in the kernel, as it is not built
> with -mgeneral-regs-only so we have no guarantee that it will leave
> the NEON registers alone.

It's actually exactly issues like this which have prompted the kernel to
use its own libgcc on many architectures.

Not only is it what Arnd says above, but it's also to do with making sure
that we know what goes into the kernel is sane.

Although libgcc is part of the compiler, libgcc is built with the
expectation that it will be running in userland - it expects to link
to a libc.  That's why you can't build libgcc without having the glibc
headers around.

For example, on ARM, libgcc.a may contain calls to __div0, and it may
provide its own __div0 implementation, which invokes various Linux
system calls.

Various functions reference _GLOBAL_OFFSET_TABLE_ which we don't want
to have in the kernel.

Depending on how the compiler was built, libgcc.a may be built for
ARMv4, ARMv5, ARMv6 or ARMv7, and may not be appropriate for linking
with the kernel.  (Eg, you wouldn't want to try to link an ARMv7
libgcc.a in an ARMv5 kernel - and we /really/ don't want the mess
of having different toolchains for different ARM arch versions.)

It may also drag in the userspace exception unwind support.

Meanwhile, having the kernel build the compiler support functions that
it needs ensures that (a) we know what compiler support functions are
being used, (b) we know the implementation of those support functions
are sane for use in the kernel, (c) we can build them with appropriate
compiler flags for best performance, and (d) we remove an unnecessary
dependency on the build toolchain.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[PATCH] arm64: kill off the libgcc dependency

[Kevin Hao]

On Wed, Jan 14, 2015 at 01:37:42PM +0000, Will Deacon wrote:
> On Wed, Jan 14, 2015 at 12:52:49PM +0000, Ard Biesheuvel wrote:
> > IMO libgcc.a cannot be used at all in the kernel, as it is not built
> > with -mgeneral-regs-only so we have no guarantee that it will leave
> > the NEON registers alone.
> 
> That's a very good point. In which case, we'd need to extend this patch
> to implement the bitops that we currently rely on the __builtins for.

Sorry, I am not sure I get what you mean. These builtin functions for bitops
are not implemented in the libgcc, they are inlined in compiler. So why do
we need to reimplement these bitops when removing the link with libgcc?

Thanks,
Kevin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20150115/25bfe1a0/attachment-0001.sig>

[PATCH] arm64: kill off the libgcc dependency

[Kevin Hao]

On Wed, Jan 14, 2015 at 02:39:26PM +0000, Russell King - ARM Linux wrote:
> It's actually exactly issues like this which have prompted the kernel to
> use its own libgcc on many architectures.
> 
> Not only is it what Arnd says above, but it's also to do with making sure
> that we know what goes into the kernel is sane.
> 
> Although libgcc is part of the compiler, libgcc is built with the
> expectation that it will be running in userland - it expects to link
> to a libc.  That's why you can't build libgcc without having the glibc
> headers around.
> 
> For example, on ARM, libgcc.a may contain calls to __div0, and it may
> provide its own __div0 implementation, which invokes various Linux
> system calls.
> 
> Various functions reference _GLOBAL_OFFSET_TABLE_ which we don't want
> to have in the kernel.
> 
> Depending on how the compiler was built, libgcc.a may be built for
> ARMv4, ARMv5, ARMv6 or ARMv7, and may not be appropriate for linking
> with the kernel.  (Eg, you wouldn't want to try to link an ARMv7
> libgcc.a in an ARMv5 kernel - and we /really/ don't want the mess
> of having different toolchains for different ARM arch versions.)
> 
> It may also drag in the userspace exception unwind support.
> 
> Meanwhile, having the kernel build the compiler support functions that
> it needs ensures that (a) we know what compiler support functions are
> being used, (b) we know the implementation of those support functions
> are sane for use in the kernel, (c) we can build them with appropriate
> compiler flags for best performance, and (d) we remove an unnecessary
> dependency on the build toolchain.

Thanks Russell for the detailed explanation. I will put this into the
commit log.

Thanks,
Kevin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20150115/aafd24dc/attachment.sig>

[PATCH] arm64: kill off the libgcc dependency

[Will Deacon]

On Thu, Jan 15, 2015 at 07:07:30AM +0000, Kevin Hao wrote:
> On Wed, Jan 14, 2015 at 01:37:42PM +0000, Will Deacon wrote:
> > On Wed, Jan 14, 2015 at 12:52:49PM +0000, Ard Biesheuvel wrote:
> > > IMO libgcc.a cannot be used at all in the kernel, as it is not built
> > > with -mgeneral-regs-only so we have no guarantee that it will leave
> > > the NEON registers alone.
> > 
> > That's a very good point. In which case, we'd need to extend this patch
> > to implement the bitops that we currently rely on the __builtins for.
> 
> Sorry, I am not sure I get what you mean. These builtin functions for bitops
> are not implemented in the libgcc, they are inlined in compiler. So why do
> we need to reimplement these bitops when removing the link with libgcc?

I'm just relaying what the tools guys told us. Namely, that they don't
guarantee that they are inlined and could generate a branch to a libgcc
function. As Ard says, they could also use fpsimd registers, so whether
or not they are inlined is moot anyway.

Will

[PATCH] arm64: kill off the libgcc dependency

[Ard Biesheuvel]

On 15 January 2015 at 09:59, Will Deacon <will.deacon@arm.com> wrote:
> On Thu, Jan 15, 2015 at 07:07:30AM +0000, Kevin Hao wrote:
>> On Wed, Jan 14, 2015 at 01:37:42PM +0000, Will Deacon wrote:
>> > On Wed, Jan 14, 2015 at 12:52:49PM +0000, Ard Biesheuvel wrote:
>> > > IMO libgcc.a cannot be used at all in the kernel, as it is not built
>> > > with -mgeneral-regs-only so we have no guarantee that it will leave
>> > > the NEON registers alone.
>> >
>> > That's a very good point. In which case, we'd need to extend this patch
>> > to implement the bitops that we currently rely on the __builtins for.
>>
>> Sorry, I am not sure I get what you mean. These builtin functions for bitops
>> are not implemented in the libgcc, they are inlined in compiler. So why do
>> we need to reimplement these bitops when removing the link with libgcc?
>
> I'm just relaying what the tools guys told us. Namely, that they don't
> guarantee that they are inlined and could generate a branch to a libgcc
> function. As Ard says, they could also use fpsimd registers, so whether
> or not they are inlined is moot anyway.
>

Which builtins are you referring to, exactly?

~/linux-2.6$ git grep __builtin arch/arm64
arch/arm64/kernel/module.c:
__builtin_return_address(0));
arch/arm64/kernel/return_address.c:     frame.fp = (unsigned
long)__builtin_frame_address(0);
arch/arm64/kernel/stacktrace.c:         frame.fp = (unsigned
long)__builtin_frame_address(0);
arch/arm64/kernel/traps.c:              frame.fp = (unsigned
long)__builtin_frame_address(0);
arch/arm64/mm/ioremap.c:
__builtin_return_address(0));
arch/arm64/mm/ioremap.c:
__builtin_return_address(0));

none of which would ever require library support, as far as I can
judge, and the arm64 kernel builds fine with -fno-builtin added (i.e.,
builtins can only be invoked explicitly using their __builtin- alias)
and libgcc.a removed.

-- 
Ard.

[PATCH] arm64: kill off the libgcc dependency

[Will Deacon]

On Thu, Jan 15, 2015 at 11:21:25AM +0000, Ard Biesheuvel wrote:
> On 15 January 2015 at 09:59, Will Deacon <will.deacon@arm.com> wrote:
> > On Thu, Jan 15, 2015 at 07:07:30AM +0000, Kevin Hao wrote:
> >> On Wed, Jan 14, 2015 at 01:37:42PM +0000, Will Deacon wrote:
> >> > On Wed, Jan 14, 2015 at 12:52:49PM +0000, Ard Biesheuvel wrote:
> >> > > IMO libgcc.a cannot be used at all in the kernel, as it is not built
> >> > > with -mgeneral-regs-only so we have no guarantee that it will leave
> >> > > the NEON registers alone.
> >> >
> >> > That's a very good point. In which case, we'd need to extend this patch
> >> > to implement the bitops that we currently rely on the __builtins for.
> >>
> >> Sorry, I am not sure I get what you mean. These builtin functions for bitops
> >> are not implemented in the libgcc, they are inlined in compiler. So why do
> >> we need to reimplement these bitops when removing the link with libgcc?
> >
> > I'm just relaying what the tools guys told us. Namely, that they don't
> > guarantee that they are inlined and could generate a branch to a libgcc
> > function. As Ard says, they could also use fpsimd registers, so whether
> > or not they are inlined is moot anyway.
> >
> 
> Which builtins are you referring to, exactly?

Things like __builtin_ctzl used by __ffs in
asm-generic/bitops/builtin-__ffs.h (included by our asm/bitops.h).

> ~/linux-2.6$ git grep __builtin arch/arm64
> arch/arm64/kernel/module.c:
> __builtin_return_address(0));
> arch/arm64/kernel/return_address.c:     frame.fp = (unsigned
> long)__builtin_frame_address(0);
> arch/arm64/kernel/stacktrace.c:         frame.fp = (unsigned
> long)__builtin_frame_address(0);
> arch/arm64/kernel/traps.c:              frame.fp = (unsigned
> long)__builtin_frame_address(0);
> arch/arm64/mm/ioremap.c:
> __builtin_return_address(0));
> arch/arm64/mm/ioremap.c:
> __builtin_return_address(0));
> 
> none of which would ever require library support, as far as I can
> judge, and the arm64 kernel builds fine with -fno-builtin added (i.e.,
> builtins can only be invoked explicitly using their __builtin- alias)
> and libgcc.a removed.

Catalin and I just spoke again with our tools engineers. Whilst they
don't guarantee the inlining, they *wouldn't* inline an implementation
using fpsimd registers into an application compiled with -mgeneral-regs-only
(at least, this would be a compiler bug).

So I've come full circle on this patch and now think it's the right thing
to do :) If, for any reason, a builtin isn't inlined then we will know
about it at link-time and can add our own implementation as and when we
need to. With the current state of mainline, we wouldn't know about the
use of fpsimd registers until silent data corruption started to occur.

For the original patch:

  Acked-by: Will Deacon <will.deacon@arm.com>

Will