Bulid regression with VDSO enabled

Bulid regression with VDSO enabled

[Stefan Agner]

Hi,

When enabling VDSO (which is default in multi_v7_defconfig), I get this
linking error on v4.1-rc1 and todays master:

  OBJCOPY arch/arm/vdso/vdso.so
BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
linking with -N
arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
value
BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
linking with -N
arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
make[2]: *** [arch/arm/vdso/vdso.so] Error 1
make[1]: *** [arch/arm/vdso] Error 2
make[1]: *** Waiting for unfinished jobs....

I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
new requirements to the toolchain or a bug?

--
Stefan

Bulid regression with VDSO enabled

[Nathan Lynch]

On 04/30/2015 06:44 AM, Stefan Agner wrote:
>   OBJCOPY arch/arm/vdso/vdso.so
> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
> linking with -N
> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
> value
> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
> linking with -N
> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
> make[1]: *** [arch/arm/vdso] Error 2
> make[1]: *** Waiting for unfinished jobs....
> 
> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
> new requirements to the toolchain or a bug?

I've not encountered this before, and I'm not able to recreate it with
that toolchain.

If you're using the Linaro toolchain I would expect the name of objcopy
to be arm-linux-gnueabihf-objcopy, but your log shows
arm-angstrom-linux-gnueabi-objcopy.  What's going on there?

Bulid regression with VDSO enabled

[Stefan Agner]

On 2015-04-30 16:38, Nathan Lynch wrote:
> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>   OBJCOPY arch/arm/vdso/vdso.so
>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>> linking with -N
>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>> value
>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>> linking with -N
>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>> make[1]: *** [arch/arm/vdso] Error 2
>> make[1]: *** Waiting for unfinished jobs....
>>
>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>> new requirements to the toolchain or a bug?
> 
> I've not encountered this before, and I'm not able to recreate it with
> that toolchain.
> 
> If you're using the Linaro toolchain I would expect the name of objcopy
> to be arm-linux-gnueabihf-objcopy, but your log shows
> arm-angstrom-linux-gnueabi-objcopy.  What's going on there?

The toolchain is built by the OpenEmbedded build system. I used the
Angstrom distribution, which uses the Linaro Layer (daisy branch) which
built that toolchain:
https://git.linaro.org/openembedded/meta-linaro.git/shortlog/refs/heads/daisy

I tried the official release and I also could not reproduce the issue,
so it seems to be toolchain related...?

--
Stefan

Bulid regression with VDSO enabled

[Nathan Lynch]

On 04/30/2015 10:20 AM, Stefan Agner wrote:
> On 2015-04-30 16:38, Nathan Lynch wrote:
>> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>>   OBJCOPY arch/arm/vdso/vdso.so
>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>> linking with -N
>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>>> value
>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>> linking with -N
>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>>> make[1]: *** [arch/arm/vdso] Error 2
>>> make[1]: *** Waiting for unfinished jobs....
>>>
>>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>>> new requirements to the toolchain or a bug?
>>
>> I've not encountered this before, and I'm not able to recreate it with
>> that toolchain.
>>
>> If you're using the Linaro toolchain I would expect the name of objcopy
>> to be arm-linux-gnueabihf-objcopy, but your log shows
>> arm-angstrom-linux-gnueabi-objcopy.  What's going on there?
> 
> The toolchain is built by the OpenEmbedded build system. I used the
> Angstrom distribution, which uses the Linaro Layer (daisy branch) which
> built that toolchain:
> https://git.linaro.org/openembedded/meta-linaro.git/shortlog/refs/heads/daisy
> 
> I tried the official release and I also could not reproduce the issue,
> so it seems to be toolchain related...?

Okay thanks, that gives me more to go on.  I probably won't be able to
devote more time to this today, but hopefully tomorrow.

Bulid regression with VDSO enabled

[Stefan Agner]

On 2015-04-30 17:48, Nathan Lynch wrote:
> On 04/30/2015 10:20 AM, Stefan Agner wrote:
>> On 2015-04-30 16:38, Nathan Lynch wrote:
>>> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>>>   OBJCOPY arch/arm/vdso/vdso.so
>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>> linking with -N
>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>>>> value
>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>> linking with -N
>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>>>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>>>> make[1]: *** [arch/arm/vdso] Error 2
>>>> make[1]: *** Waiting for unfinished jobs....
>>>>
>>>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>>>> new requirements to the toolchain or a bug?
>>>
>>> I've not encountered this before, and I'm not able to recreate it with
>>> that toolchain.
>>>
>>> If you're using the Linaro toolchain I would expect the name of objcopy
>>> to be arm-linux-gnueabihf-objcopy, but your log shows
>>> arm-angstrom-linux-gnueabi-objcopy.  What's going on there?
>>
>> The toolchain is built by the OpenEmbedded build system. I used the
>> Angstrom distribution, which uses the Linaro Layer (daisy branch) which
>> built that toolchain:
>> https://git.linaro.org/openembedded/meta-linaro.git/shortlog/refs/heads/daisy
>>
>> I tried the official release and I also could not reproduce the issue,
>> so it seems to be toolchain related...?
> 
> Okay thanks, that gives me more to go on.  I probably won't be able to
> devote more time to this today, but hopefully tomorrow.

FWIW, I just looked into it a bit more myself, however don't understand
what is wrong with that toolchain so far. The generated linker command
looks good for both toolchains, it is really that one linker creates the
problem with the very same arguments while the other does not.

The toolchain generated by OpenEmbedded is probably quite different from
the official releases. There are a bunch of configurations which are
tweaked depending on the local OE configuration as well:
https://github.com/openembedded/oe-core/blob/master/meta/recipes-devtools/gcc/gcc-configure-common.inc

I also dumped the headers of the working/non-working output file, there
seem to be different sections generated, maybe a hint?

ags at linuxdev /tmp $
/tmp/gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux/bin/arm-linux-gnueabihf-objdump
-h /tmp/vdso.so.raw.working

/tmp/vdso.so.raw.working:     file format elf32-littlearm

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .hash         00000028  000000d4  000000d4  000000d4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .dynsym       00000050  000000fc  000000fc  000000fc  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .dynstr       00000044  0000014c  0000014c  0000014c  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .gnu.version  0000000a  00000190  00000190  00000190  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .gnu.version_d 00000038  0000019c  0000019c  0000019c  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .note         00000024  000001d4  000001d4  000001d4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  6 .dynamic      00000080  000001f8  000001f8  000001f8  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  7 .ARM.exidx    00000028  00000278  00000278  00000278  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .text         0000042c  000002a0  000002a0  000002a0  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  9 .got.plt      0000000c  000006cc  000006cc  000006cc  2**2
                  CONTENTS, ALLOC, LOAD, DATA
 10 .comment      0000003a  00000000  00000000  000006d8  2**0
                  CONTENTS, READONLY
 11 .ARM.attributes 0000002f  00000000  00000000  00000712  2**0
                  CONTENTS, READONLY
ags at linuxdev /tmp $
/tmp/gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux/bin/arm-linux-gnueabihf-objdump
-h /tmp/vdso.so.raw 

/tmp/vdso.so.raw:     file format elf32-littlearm

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .hash         00000028  000000b4  000000b4  000000b4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .dynsym       00000050  000000dc  000000dc  000000dc  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .dynstr       0000004b  0000012c  0000012c  0000012c  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .gnu.version  0000000a  00000178  00000178  00000178  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .gnu.version_d 00000038  00000184  00000184  00000184  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .note         00000040  000001bc  000001bc  000001bc  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  6 .dynamic      00000088  000001fc  000001fc  000001fc  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  7 .ARM.extab    00000000  00000284  00000284  00000284  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .ARM.exidx    00000028  00000284  00000284  00000284  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  9 .interp       00000013  000002ac  000002ac  000002ac  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .text         0000042c  000002c0  000002c0  000002c0  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 11 .comment      0000003b  00000000  00000000  000006ec  2**0
                  CONTENTS, READONLY
 12 .ARM.attributes 0000002f  00000000  00000000  00000727  2**0
                  CONTENTS, READONLY


--
Stefan

Bulid regression with VDSO enabled

[Nathan Lynch]

On 04/30/2015 10:58 AM, Stefan Agner wrote:
> On 2015-04-30 17:48, Nathan Lynch wrote:
>> On 04/30/2015 10:20 AM, Stefan Agner wrote:
>>> On 2015-04-30 16:38, Nathan Lynch wrote:
>>>> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>>>>   OBJCOPY arch/arm/vdso/vdso.so
>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>> linking with -N
>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>>>>> value
>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>> linking with -N
>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>>>>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>>>>> make[1]: *** [arch/arm/vdso] Error 2
>>>>> make[1]: *** Waiting for unfinished jobs....
>>>>>
>>>>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>>>>> new requirements to the toolchain or a bug?
>>>>
>>>> I've not encountered this before, and I'm not able to recreate it with
>>>> that toolchain.
>>>>
>>>> If you're using the Linaro toolchain I would expect the name of objcopy
>>>> to be arm-linux-gnueabihf-objcopy, but your log shows
>>>> arm-angstrom-linux-gnueabi-objcopy.  What's going on there?
>>>
>>> The toolchain is built by the OpenEmbedded build system. I used the
>>> Angstrom distribution, which uses the Linaro Layer (daisy branch) which
>>> built that toolchain:
>>> https://git.linaro.org/openembedded/meta-linaro.git/shortlog/refs/heads/daisy
>>>
>>> I tried the official release and I also could not reproduce the issue,
>>> so it seems to be toolchain related...?
>>
>> Okay thanks, that gives me more to go on.  I probably won't be able to
>> devote more time to this today, but hopefully tomorrow.
> 
> FWIW, I just looked into it a bit more myself, however don't understand
> what is wrong with that toolchain so far. The generated linker command
> looks good for both toolchains, it is really that one linker creates the
> problem with the very same arguments while the other does not.
> 
> The toolchain generated by OpenEmbedded is probably quite different from
> the official releases. There are a bunch of configurations which are
> tweaked depending on the local OE configuration as well:
> https://github.com/openembedded/oe-core/blob/master/meta/recipes-devtools/gcc/gcc-configure-common.inc

A relevant difference would seem to be that ld defaults to gold for this
toolchain:

$ arm-angstrom-linux-gnueabi-ld --version
GNU gold (GNU Binutils 2.24) 1.11
...

I've not been explicitly testing the vdso code with gold until now,
sorry.  Is gold a "supported" linker for the ARM kernel these days?  If
I disable CONFIG_VDSO I encounter this during final link:

  LD      init/built-in.o
arm-angstrom-linux-gnueabi-ld: --pic-veneer: unknown option
arm-angstrom-linux-gnueabi-ld: use the --help option for usage information


I've managed to recreate this locally now, and I'll see what can be
done.  Commit 378ed3ccd2a0 "x86, vdso: Make the vdso linker script
compatible with Gold" looks relevant.

support status of gold (was: Bulid regression with VDSO enabled)

[Russell King - ARM Linux]

On Tue, May 05, 2015 at 11:42:42AM -0500, Nathan Lynch wrote:
> (following up with a wider audience, with lots of investigation
> back-and-forth elided)
> 
> To reiterate - is gold currently considered a suitable linker for the
> ARM kernel?

I have no idea, sorry.  Presumably given the issues you're seeing, the
answer is that it's not supported at the moment.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

support status of gold (was: Bulid regression with VDSO enabled)

[Ard Biesheuvel]

On 5 May 2015 at 18:42, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
> (following up with a wider audience, with lots of investigation
> back-and-forth elided)
>
>>>>>> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>>>>>>   OBJCOPY arch/arm/vdso/vdso.so
>>>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>>>> linking with -N
>>>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>>>>>>> value
>>>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>>>> linking with -N
>>>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>>>>>>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>>>>>>> make[1]: *** [arch/arm/vdso] Error 2
>>>>>>> make[1]: *** Waiting for unfinished jobs....
>>>>>>>
>>>>>>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>>>>>>> new requirements to the toolchain or a bug?
>
> [...]
>
> On 05/01/2015 10:22 AM, Nathan Lynch wrote:
>>
>> A relevant difference would seem to be that ld defaults to gold for this
>> toolchain:
>>
>> $ arm-angstrom-linux-gnueabi-ld --version
>> GNU gold (GNU Binutils 2.24) 1.11
>> ...
>>
>> I've not been explicitly testing the vdso code with gold until now,
>> sorry.  Is gold a "supported" linker for the ARM kernel these days?
>
> To reiterate - is gold currently considered a suitable linker for the
> ARM kernel?
>
> I'm able to build a 4.0/multi_v7_defconfig kernel with gold and boot it
> successfully in qemu, but 4.1-rc1 introduced a couple of issues:
>
> - arch/arm/Makefile adds --pic-veneer to LDFLAGS, which ld.gold does not
> implement:
>     LD      init/built-in.o
>   arm-angstrom-linux-gnueabi-ld: --pic-veneer: unknown option
>

We could replace --pic-veneer with $(call ld-option,--pic-veneer) to
ensure that it is only passed if the linker supports it, but the
problem here is that the option was added for a reason, i.e., to
ensure that if the linker needs to insert veneers, they will be
position independent even if the binary we are building is not. This
specifically applies to the ARM kernel, since it is not compiled and
linked with -fpic but may still execute at another offset than it was
linked at (while running with the MMU off)

> - The VDSO linker script is interpreted differently by ld.gold,
> resulting in build errors (see original report above).  I've made some
> progress on resolving this, but I still don't have a working VDSO at
> runtime.
>
> Even when I successfully build the rest of the kernel with gold (4.0 or
> 4.1-rc), I see many warnings like this:
>
>   arm-angstrom-linux-gnueabi-ld: warning: unwinding may not work because
>   EXIDX input section 41 of arch/arm/mach-omap2/built-in.o is not in
>   EXIDX output section
>
> But I'm unsure of their significance.
>

Bulid regression with VDSO enabled

[Russell King - ARM Linux]

On Fri, May 08, 2015 at 01:13:50PM +0200, Stefan Agner wrote:
> On 2015-05-01 17:22, Nathan Lynch wrote:
> > A relevant difference would seem to be that ld defaults to gold for this
> > toolchain:
> > 
> > $ arm-angstrom-linux-gnueabi-ld --version
> > GNU gold (GNU Binutils 2.24) 1.11
> > ...
> > 
> > I've not been explicitly testing the vdso code with gold until now,
> > sorry.  Is gold a "supported" linker for the ARM kernel these days?  If
> > I disable CONFIG_VDSO I encounter this during final link:
> > 
> >   LD      init/built-in.o
> > arm-angstrom-linux-gnueabi-ld: --pic-veneer: unknown option
> > arm-angstrom-linux-gnueabi-ld: use the --help option for usage information
> > 
> > 
> > I've managed to recreate this locally now, and I'll see what can be
> > done.  Commit 378ed3ccd2a0 "x86, vdso: Make the vdso linker script
> > compatible with Gold" looks relevant.
> 
> This error does not look like the error I have:
> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
> linking with -N
> /build/ags/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]:
> Bad value
> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
> linking with -N
> /build/ags/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so:
> Bad value
> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
> make[1]: *** [arch/arm/vdso] Error 2
> make[1]: *** Waiting for unfinished jobs....
> 
> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
> build command. Interestingly thought, I had the same issue when using
> gold linker...
> 
> I just updated my toolchain to Linaro's 2014.11 based version, still
> using meta-linaro/OpenEmbedded build system, but still the same issue.
> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
> linking with -N
> /build/linuxdev/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]:
> Bad value
> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
> linking with -N
> /build/linuxdev/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so:
> Bad value
> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
> make[1]: *** [arch/arm/vdso] Error 2
> make[1]: *** Waiting for unfinished jobs....
> 
> The toolchain comes with GCC version 4.9.3... So the problem seems to be
> quite unrelated to the toolchain version, but more to the actual
> configuration of the toolchain.

This kind of thing annoys me.  Your statement here is totally incorrect
because you're counting the number of apples and reporting it as the
number of oranges.

The version of GCC has nothing to do with the version of the linker.
The linker is part of binutils, which is an entirely separate software
package from the compiler.

What I know is that this works with the stock GNU binutils version 2.22,
which is what I'm using here.  Can you report which version of binutils
you're using please (and I hope that if it has non-upstream changes,
that the version number reflects that fact.)

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

Bulid regression with VDSO enabled

[Stefan Agner]

On 2015-05-08 13:28, Russell King - ARM Linux wrote:
> On Fri, May 08, 2015 at 01:13:50PM +0200, Stefan Agner wrote:
>> On 2015-05-01 17:22, Nathan Lynch wrote:
>> > A relevant difference would seem to be that ld defaults to gold for this
>> > toolchain:
>> >
>> > $ arm-angstrom-linux-gnueabi-ld --version
>> > GNU gold (GNU Binutils 2.24) 1.11
>> > ...
>> >
>> > I've not been explicitly testing the vdso code with gold until now,
>> > sorry.  Is gold a "supported" linker for the ARM kernel these days?  If
>> > I disable CONFIG_VDSO I encounter this during final link:
>> >
>> >   LD      init/built-in.o
>> > arm-angstrom-linux-gnueabi-ld: --pic-veneer: unknown option
>> > arm-angstrom-linux-gnueabi-ld: use the --help option for usage information
>> >
>> >
>> > I've managed to recreate this locally now, and I'll see what can be
>> > done.  Commit 378ed3ccd2a0 "x86, vdso: Make the vdso linker script
>> > compatible with Gold" looks relevant.
>>
>> This error does not look like the error I have:
>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>> linking with -N
>> /build/ags/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]:
>> Bad value
>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>> linking with -N
>> /build/ags/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so:
>> Bad value
>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>> make[1]: *** [arch/arm/vdso] Error 2
>> make[1]: *** Waiting for unfinished jobs....
>>
>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>> build command. Interestingly thought, I had the same issue when using
>> gold linker...
>>
>> I just updated my toolchain to Linaro's 2014.11 based version, still
>> using meta-linaro/OpenEmbedded build system, but still the same issue.
>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>> linking with -N
>> /build/linuxdev/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]:
>> Bad value
>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>> linking with -N
>> /build/linuxdev/oe-core/build/out-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so:
>> Bad value
>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>> make[1]: *** [arch/arm/vdso] Error 2
>> make[1]: *** Waiting for unfinished jobs....
>>
>> The toolchain comes with GCC version 4.9.3... So the problem seems to be
>> quite unrelated to the toolchain version, but more to the actual
>> configuration of the toolchain.
> 
> This kind of thing annoys me.  Your statement here is totally incorrect
> because you're counting the number of apples and reporting it as the
> number of oranges.
> 
> The version of GCC has nothing to do with the version of the linker.
> The linker is part of binutils, which is an entirely separate software
> package from the compiler.

Yeah sure, sorry about that. The essence I wanted to convey was that it
happens with two different releases of the Linaro toolchain, which
happens to be denoted with the GCC version number + that year.month
thing. The two versions which come with Angstrom/meta-linaro are:

- Linaro toolchain 4.8-2014.04 (which comes with Angstrom 2014.06,
meta-linaro, daisy branch)
- Linaro toolchain 4.9-2014.11 (which comes with Angstrom 2014.12,
meta-linaro, dizzy branch)

> What I know is that this works with the stock GNU binutils version 2.22,
> which is what I'm using here.  Can you report which version of binutils
> you're using please (and I hope that if it has non-upstream changes,
> that the version number reflects that fact.)

The newer toolchain does, while the older does not:

...ld.bfd --version
GNU ld (GNU Binutils) Linaro 2014.11-2 2.24.0.20141017

...ld.bfd --version
GNU ld (GNU Binutils) 2.24

It looks like the Linaro release 2014.04 really did not alter binutils.
However, meta-linaro, the OpenEmbedded build layer, puts several patches
on-top of that, but failing to alter the version number.

https://git.linaro.org/openembedded/meta-linaro.git/tree/refs/heads/daisy:/meta-linaro-toolchain/recipes-devtools/binutils/binutils-linaro-2.24

Quite likely that the problem come with one of those patches. But the
linaro-dev mailing list on CC, maybe somebody knows more?

--
Stefan

Bulid regression with VDSO enabled

[Nathan Lynch]

On 05/08/2015 06:13 AM, Stefan Agner wrote:
> On 2015-05-01 17:22, Nathan Lynch wrote:
>> On 04/30/2015 10:58 AM, Stefan Agner wrote:
>>> On 2015-04-30 17:48, Nathan Lynch wrote:
>>>> On 04/30/2015 10:20 AM, Stefan Agner wrote:
>>>>> On 2015-04-30 16:38, Nathan Lynch wrote:
>>>>>> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>>>>>>   OBJCOPY arch/arm/vdso/vdso.so
>>>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>>>> linking with -N
>>>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>>>>>>> value
>>>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>>>> linking with -N
>>>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>>>>>>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>>>>>>> make[1]: *** [arch/arm/vdso] Error 2
>>>>>>> make[1]: *** Waiting for unfinished jobs....
>>>>>>>
>>>>>>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>>>>>>> new requirements to the toolchain or a bug?
>>>>>>
>>>>>> I've not encountered this before, and I'm not able to recreate it with
>>>>>> that toolchain.
>>>>>>
>>>>>> If you're using the Linaro toolchain I would expect the name of objcopy
>>>>>> to be arm-linux-gnueabihf-objcopy, but your log shows
>>>>>> arm-angstrom-linux-gnueabi-objcopy.  What's going on there?
>>>>>
>>>>> The toolchain is built by the OpenEmbedded build system. I used the
>>>>> Angstrom distribution, which uses the Linaro Layer (daisy branch) which
>>>>> built that toolchain:
>>>>> https://git.linaro.org/openembedded/meta-linaro.git/shortlog/refs/heads/daisy
>>>>>
>>>>> I tried the official release and I also could not reproduce the issue,
>>>>> so it seems to be toolchain related...?
>>>>
>>>> Okay thanks, that gives me more to go on.  I probably won't be able to
>>>> devote more time to this today, but hopefully tomorrow.
>>>
>>> FWIW, I just looked into it a bit more myself, however don't understand
>>> what is wrong with that toolchain so far. The generated linker command
>>> looks good for both toolchains, it is really that one linker creates the
>>> problem with the very same arguments while the other does not.
>>>
>>> The toolchain generated by OpenEmbedded is probably quite different from
>>> the official releases. There are a bunch of configurations which are
>>> tweaked depending on the local OE configuration as well:
>>> https://github.com/openembedded/oe-core/blob/master/meta/recipes-devtools/gcc/gcc-configure-common.inc
>>
>> A relevant difference would seem to be that ld defaults to gold for this
>> toolchain:
>>
>> $ arm-angstrom-linux-gnueabi-ld --version
>> GNU gold (GNU Binutils 2.24) 1.11
>> ...
>>
>> I've not been explicitly testing the vdso code with gold until now,
>> sorry.  Is gold a "supported" linker for the ARM kernel these days?  If
>> I disable CONFIG_VDSO I encounter this during final link:
>>
>>   LD      init/built-in.o
>> arm-angstrom-linux-gnueabi-ld: --pic-veneer: unknown option
>> arm-angstrom-linux-gnueabi-ld: use the --help option for usage information
>>
>>
>> I've managed to recreate this locally now, and I'll see what can be
>> done.  Commit 378ed3ccd2a0 "x86, vdso: Make the vdso linker script
>> compatible with Gold" looks relevant.
> 
> This error does not look like the error I have:

I can see I perhaps phrased that badly; when I said I can recreate "this"
I meant I could recreate the issue you reported.  I was intending to
point out that even if the vdso build with gold is fixed or worked
around, the kernel still won't link because gold doesn't implement
--pic-veneer.


> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
> build command. Interestingly thought, I had the same issue when using
> gold linker...

When I try this, ld.gold is used regardless.

You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:

$ readelf -n arch/arm/vdso/vdso.so.raw                            

Displaying notes found at file offset 0x000001bc with length 0x00000040:
  Owner                 Data size       Description
  GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
  GNU                  0x00000014       NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: f025409550acb7f955c61d95691291da078e6688

Bulid regression with VDSO enabled

[Stefan Agner]

On 2015-05-08 17:27, Nathan Lynch wrote:
> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>> On 2015-05-01 17:22, Nathan Lynch wrote:
>>> On 04/30/2015 10:58 AM, Stefan Agner wrote:
>>>> On 2015-04-30 17:48, Nathan Lynch wrote:
>>>>> On 04/30/2015 10:20 AM, Stefan Agner wrote:
>>>>>> On 2015-04-30 16:38, Nathan Lynch wrote:
>>>>>>> On 04/30/2015 06:44 AM, Stefan Agner wrote:
>>>>>>>>   OBJCOPY arch/arm/vdso/vdso.so
>>>>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>>>>> linking with -N
>>>>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so[.hash]: Bad
>>>>>>>> value
>>>>>>>> BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
>>>>>>>> linking with -N
>>>>>>>> arm-angstrom-linux-gnueabi-objcopy:arch/arm/vdso/vdso.so: Bad value
>>>>>>>> make[2]: *** [arch/arm/vdso/vdso.so] Error 1
>>>>>>>> make[1]: *** [arch/arm/vdso] Error 2
>>>>>>>> make[1]: *** Waiting for unfinished jobs....
>>>>>>>>
>>>>>>>> I'm using GCC 4.8.3 (Linaro GCC 4.8-2014.04) on Fedora 21. Any specific
>>>>>>>> new requirements to the toolchain or a bug?
>>>>>>>
>>>>>>> I've not encountered this before, and I'm not able to recreate it with
>>>>>>> that toolchain.
>>>>>>>
>>>>>>> If you're using the Linaro toolchain I would expect the name of objcopy
>>>>>>> to be arm-linux-gnueabihf-objcopy, but your log shows
>>>>>>> arm-angstrom-linux-gnueabi-objcopy.  What's going on there?
>>>>>>
>>>>>> The toolchain is built by the OpenEmbedded build system. I used the
>>>>>> Angstrom distribution, which uses the Linaro Layer (daisy branch) which
>>>>>> built that toolchain:
>>>>>> https://git.linaro.org/openembedded/meta-linaro.git/shortlog/refs/heads/daisy
>>>>>>
>>>>>> I tried the official release and I also could not reproduce the issue,
>>>>>> so it seems to be toolchain related...?
>>>>>
>>>>> Okay thanks, that gives me more to go on.  I probably won't be able to
>>>>> devote more time to this today, but hopefully tomorrow.
>>>>
>>>> FWIW, I just looked into it a bit more myself, however don't understand
>>>> what is wrong with that toolchain so far. The generated linker command
>>>> looks good for both toolchains, it is really that one linker creates the
>>>> problem with the very same arguments while the other does not.
>>>>
>>>> The toolchain generated by OpenEmbedded is probably quite different from
>>>> the official releases. There are a bunch of configurations which are
>>>> tweaked depending on the local OE configuration as well:
>>>> https://github.com/openembedded/oe-core/blob/master/meta/recipes-devtools/gcc/gcc-configure-common.inc
>>>
>>> A relevant difference would seem to be that ld defaults to gold for this
>>> toolchain:
>>>
>>> $ arm-angstrom-linux-gnueabi-ld --version
>>> GNU gold (GNU Binutils 2.24) 1.11
>>> ...
>>>
>>> I've not been explicitly testing the vdso code with gold until now,
>>> sorry.  Is gold a "supported" linker for the ARM kernel these days?  If
>>> I disable CONFIG_VDSO I encounter this during final link:
>>>
>>>   LD      init/built-in.o
>>> arm-angstrom-linux-gnueabi-ld: --pic-veneer: unknown option
>>> arm-angstrom-linux-gnueabi-ld: use the --help option for usage information
>>>
>>>
>>> I've managed to recreate this locally now, and I'll see what can be
>>> done.  Commit 378ed3ccd2a0 "x86, vdso: Make the vdso linker script
>>> compatible with Gold" looks relevant.
>>
>> This error does not look like the error I have:
> 
> I can see I perhaps phrased that badly; when I said I can recreate "this"
> I meant I could recreate the issue you reported.  I was intending to
> point out that even if the vdso build with gold is fixed or worked
> around, the kernel still won't link because gold doesn't implement
> --pic-veneer.
> 
> 
>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>> build command. Interestingly thought, I had the same issue when using
>> gold linker...
> 
> When I try this, ld.gold is used regardless.
> 
> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
> 
> $ readelf -n arch/arm/vdso/vdso.so.raw                            
> 
> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>   Owner                 Data size       Description
>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
> ID bitstring)
>     Build ID: f025409550acb7f955c61d95691291da078e6688

Hm, you are right. When deleting moving ld.bfd to ld, which makes the
BFD linker as default, compiling works flawless.

However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
objects, at least those do not have this note. For instance time.o does
not return anything:

$ readelf -n arch/arm/kernel/time.o

So it seems to be a vdso.so.raw specific problem not using the specified
linker...?

--
Stefan

Bulid regression with VDSO enabled

[Nathan Lynch]

On 05/08/2015 11:08 AM, Stefan Agner wrote:
> On 2015-05-08 17:27, Nathan Lynch wrote:
>> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>>> build command. Interestingly thought, I had the same issue when using
>>> gold linker...
>>
>> When I try this, ld.gold is used regardless.
>>
>> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
>>
>> $ readelf -n arch/arm/vdso/vdso.so.raw                            
>>
>> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>>   Owner                 Data size       Description
>>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
>> ID bitstring)
>>     Build ID: f025409550acb7f955c61d95691291da078e6688
> 
> Hm, you are right. When deleting moving ld.bfd to ld, which makes the
> BFD linker as default, compiling works flawless.
> 
> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
> objects, at least those do not have this note. For instance time.o does
> not return anything:
> 
> $ readelf -n arch/arm/kernel/time.o

This is just an object file that hasn't been linked, but your general
point stands -- LD is honored by other parts of the build.


> So it seems to be a vdso.so.raw specific problem not using the specified
> linker...?

Yes, since it's produced by an invocation of $(CC) which is expected
to call the linker implicitly, and this is how the toolchain's default
linker ends up getting used even when you set LD on the command line.  

If I'm not mistaken, implicitly performing a link through the compiler
seems to be 1) conventional for all VDSOs, not just ARM's, but 2) unusual
for other parts of the kernel build.

I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
may suffice:

diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index 8aa791051029..da0ce897edde 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
       cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) \
                    $(call cc-ldoption, -Wl$(comma)--build-id) \
                    -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
+                   $(call cc-option, -fuse-ld=bfd) \
                    -Wl,-z,common-page-size=4096 -o $@
 
 quiet_cmd_vdsomunge = MUNGE   $@

Bulid regression with VDSO enabled

[Ard Biesheuvel]

Hi Nathan,

On 8 May 2015 at 23:15, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
> On 05/08/2015 11:08 AM, Stefan Agner wrote:
>> On 2015-05-08 17:27, Nathan Lynch wrote:
>>> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>>>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>>>> build command. Interestingly thought, I had the same issue when using
>>>> gold linker...
>>>
>>> When I try this, ld.gold is used regardless.
>>>
>>> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
>>>
>>> $ readelf -n arch/arm/vdso/vdso.so.raw
>>>
>>> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>>>   Owner                 Data size       Description
>>>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>>>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
>>> ID bitstring)
>>>     Build ID: f025409550acb7f955c61d95691291da078e6688
>>
>> Hm, you are right. When deleting moving ld.bfd to ld, which makes the
>> BFD linker as default, compiling works flawless.
>>
>> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
>> objects, at least those do not have this note. For instance time.o does
>> not return anything:
>>
>> $ readelf -n arch/arm/kernel/time.o
>
> This is just an object file that hasn't been linked, but your general
> point stands -- LD is honored by other parts of the build.
>
>

I suppose this implies that the build only chokes on --pic-veneer when
linking the VDSO, and nowhere else?

>> So it seems to be a vdso.so.raw specific problem not using the specified
>> linker...?
>
> Yes, since it's produced by an invocation of $(CC) which is expected
> to call the linker implicitly, and this is how the toolchain's default
> linker ends up getting used even when you set LD on the command line.
>
> If I'm not mistaken, implicitly performing a link through the compiler
> seems to be 1) conventional for all VDSOs, not just ARM's, but 2) unusual
> for other parts of the kernel build.
>
> I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
> would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
> may suffice:
>
> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
> index 8aa791051029..da0ce897edde 100644
> --- a/arch/arm/vdso/Makefile
> +++ b/arch/arm/vdso/Makefile
> @@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
>        cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) \
>                     $(call cc-ldoption, -Wl$(comma)--build-id) \
>                     -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
> +                   $(call cc-option, -fuse-ld=bfd) \
>                     -Wl,-z,common-page-size=4096 -o $@
>
>  quiet_cmd_vdsomunge = MUNGE   $@
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Bulid regression with VDSO enabled

[Nathan Lynch]

On 05/09/2015 03:31 AM, Ard Biesheuvel wrote:
> Hi Nathan,
> 
> On 8 May 2015 at 23:15, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>> On 05/08/2015 11:08 AM, Stefan Agner wrote:
>>> On 2015-05-08 17:27, Nathan Lynch wrote:
>>>> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>>>>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>>>>> build command. Interestingly thought, I had the same issue when using
>>>>> gold linker...
>>>>
>>>> When I try this, ld.gold is used regardless.
>>>>
>>>> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
>>>>
>>>> $ readelf -n arch/arm/vdso/vdso.so.raw
>>>>
>>>> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>>>>   Owner                 Data size       Description
>>>>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>>>>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
>>>> ID bitstring)
>>>>     Build ID: f025409550acb7f955c61d95691291da078e6688
>>>
>>> Hm, you are right. When deleting moving ld.bfd to ld, which makes the
>>> BFD linker as default, compiling works flawless.
>>>
>>> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
>>> objects, at least those do not have this note. For instance time.o does
>>> not return anything:
>>>
>>> $ readelf -n arch/arm/kernel/time.o
>>
>> This is just an object file that hasn't been linked, but your general
>> point stands -- LD is honored by other parts of the build.
>>
>>
> 
> I suppose this implies that the build only chokes on --pic-veneer when
> linking the VDSO, and nowhere else?

No, --pic-veneer isn't the problem for the VDSO; for better or worse
LDFLAGS is ignored for the VDSO build.

Bulid regression with VDSO enabled

[Russell King - ARM Linux]

On Fri, May 08, 2015 at 04:15:25PM -0500, Nathan Lynch wrote:
> I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
> would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
> may suffice:
> 
> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
> index 8aa791051029..da0ce897edde 100644
> --- a/arch/arm/vdso/Makefile
> +++ b/arch/arm/vdso/Makefile
> @@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
>        cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) \
>                     $(call cc-ldoption, -Wl$(comma)--build-id) \
>                     -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
> +                   $(call cc-option, -fuse-ld=bfd) \
>                     -Wl,-z,common-page-size=4096 -o $@

Would it make more sense to have something like:

VDSO_LDFLAGS := -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
		-Wl,-z,common-page-size=4096 \
		$(call cc-ldoption, -Wl$(comma)--build-id) \
		$(call cc-option, -fuse-ld=bfd)

and then have:

	 cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) \
		      -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@

Also, do we want to use cc-option or cc-ldoption (this question becomes
obvious when you arrange the Makefile as I have above...)?  cc-option
doesn't result in the linker actually being run, where as cc-ldoption
does.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

Bulid regression with VDSO enabled

[Nathan Lynch]

Sorry for the delay in following up on this.

On 05/09/2015 04:26 AM, Russell King - ARM Linux wrote:
> On Fri, May 08, 2015 at 04:15:25PM -0500, Nathan Lynch wrote:
>> I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
>> would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
>> may suffice:
>>
>> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
>> index 8aa791051029..da0ce897edde 100644
>> --- a/arch/arm/vdso/Makefile
>> +++ b/arch/arm/vdso/Makefile
>> @@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
>>        cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) \
>>                     $(call cc-ldoption, -Wl$(comma)--build-id) \
>>                     -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
>> +                   $(call cc-option, -fuse-ld=bfd) \
>>                     -Wl,-z,common-page-size=4096 -o $@
> 
> Would it make more sense to have something like:
> 
> VDSO_LDFLAGS := -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
> 		-Wl,-z,common-page-size=4096 \
> 		$(call cc-ldoption, -Wl$(comma)--build-id) \
> 		$(call cc-option, -fuse-ld=bfd)
> 
> and then have:
> 
> 	 cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) \
> 		      -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@

Yes, I can see value in grouping linker directives into one variable;
there are a few in ccflags-y that could be moved to VDSO_LDFLAGS as well.


> Also, do we want to use cc-option or cc-ldoption (this question becomes
> obvious when you arrange the Makefile as I have above...)?  cc-option
> doesn't result in the linker actually being run, where as cc-ldoption
> does.

It's a corner case, but cc-option and cc-ldoption result in different
failure modes when gold is the default linker and the bfd linker is not
in $PATH:

With $(call cc-option, -fuse-ld=bfd):
  VDSO    arch/arm/vdso/vdso.so.raw
collect2: fatal error: cannot find 'ld'

With $(call cc-ldoption, -fuse-ld=bfd):
  VDSO    arch/arm/vdso/vdso.so.raw
  HOSTCC  arch/arm/vdso/vdsomunge
  MUNGE   arch/arm/vdso/vdso.so.dbg
  OBJCOPY arch/arm/vdso/vdso.so
BFD: arch/arm/vdso/vdso.so: Not enough room for program headers, try
linking with -N

That is, since using cc-ldoption invokes the linker, and the bfd linker
isn't found, -fuse-ld=bfd is omitted from the command and gold gets run,
resulting in the same symptom as the report that started this thread.

I think the first one gives more of a clue as to what the user can do to
remedy the situation, so that's what I'd slightly prefer.

Bulid regression with VDSO enabled

[Stefan Agner]

On 2015-05-08 23:15, Nathan Lynch wrote:
> On 05/08/2015 11:08 AM, Stefan Agner wrote:
>> On 2015-05-08 17:27, Nathan Lynch wrote:
>>> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>>>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>>>> build command. Interestingly thought, I had the same issue when using
>>>> gold linker...
>>>
>>> When I try this, ld.gold is used regardless.
>>>
>>> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
>>>
>>> $ readelf -n arch/arm/vdso/vdso.so.raw
>>>
>>> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>>>   Owner                 Data size       Description
>>>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>>>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
>>> ID bitstring)
>>>     Build ID: f025409550acb7f955c61d95691291da078e6688
>>
>> Hm, you are right. When deleting moving ld.bfd to ld, which makes the
>> BFD linker as default, compiling works flawless.
>>
>> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
>> objects, at least those do not have this note. For instance time.o does
>> not return anything:
>>
>> $ readelf -n arch/arm/kernel/time.o
> 
> This is just an object file that hasn't been linked, but your general
> point stands -- LD is honored by other parts of the build.
> 
> 
>> So it seems to be a vdso.so.raw specific problem not using the specified
>> linker...?
> 
> Yes, since it's produced by an invocation of $(CC) which is expected
> to call the linker implicitly, and this is how the toolchain's default
> linker ends up getting used even when you set LD on the command line.  
> 
> If I'm not mistaken, implicitly performing a link through the compiler
> seems to be 1) conventional for all VDSOs, not just ARM's, but 2) unusual
> for other parts of the kernel build.
> 
> I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
> would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
> may suffice:
> 
> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
> index 8aa791051029..da0ce897edde 100644
> --- a/arch/arm/vdso/Makefile
> +++ b/arch/arm/vdso/Makefile
> @@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
>        cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^)
> $(filter %.o,$^) \
>                     $(call cc-ldoption, -Wl$(comma)--build-id) \
>                     -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
> +                   $(call cc-option, -fuse-ld=bfd) \
>                     -Wl,-z,common-page-size=4096 -o $@
>  
>  quiet_cmd_vdsomunge = MUNGE   $@


Thanks Nathan, your solution looks reasonable. I figured that the option
is available since GCC 4.8, so it should work probably for most
toolchains which try to make use of gold linker as default.

Unfortunately, I hit another problem with the configuration on my build
host. With your patch applied, I get:
collect2: fatal error: cannot find 'ld'
compilation terminated.
make[2]: *** [arch/arm/vdso/vdso.so.raw] Error 1
make[1]: *** [arch/arm/vdso] Error 2
make[1]: *** Waiting for unfinished jobs....

OpenEmbedded distinguish between three toolchains for the target:
- Native (on the target)
- Host (on the host which builds the whole image, usually x86_64)
- SDK (on a third architecture for application development, usually x86
or x86_64, the toolchain which I published the link to)

I use x86_64 as host and SDK architecture, nevertheless it seems that
the toolchains are configured differently. The Host toolchain (the one
which shows the error above) uses "--enable-gold=default" when
configuring binutils. With that option, ld.bfd is still being built, but
ld is by default gold, without any additional doing (cp/ln -f..).
Currently it seems to me that this difference triggers a problem with
-fuse-ld=bfd.

However, I don't think that this problem invalidates your proposed
solution.

--
Stefan

Bulid regression with VDSO enabled

[Stefan Agner]

On 2015-05-09 14:41, Stefan Agner wrote:
> On 2015-05-08 23:15, Nathan Lynch wrote:
>> On 05/08/2015 11:08 AM, Stefan Agner wrote:
>>> On 2015-05-08 17:27, Nathan Lynch wrote:
>>>> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>>>>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>>>>> build command. Interestingly thought, I had the same issue when using
>>>>> gold linker...
>>>>
>>>> When I try this, ld.gold is used regardless.
>>>>
>>>> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
>>>>
>>>> $ readelf -n arch/arm/vdso/vdso.so.raw
>>>>
>>>> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>>>>   Owner                 Data size       Description
>>>>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>>>>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
>>>> ID bitstring)
>>>>     Build ID: f025409550acb7f955c61d95691291da078e6688
>>>
>>> Hm, you are right. When deleting moving ld.bfd to ld, which makes the
>>> BFD linker as default, compiling works flawless.
>>>
>>> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
>>> objects, at least those do not have this note. For instance time.o does
>>> not return anything:
>>>
>>> $ readelf -n arch/arm/kernel/time.o
>>
>> This is just an object file that hasn't been linked, but your general
>> point stands -- LD is honored by other parts of the build.
>>
>>
>>> So it seems to be a vdso.so.raw specific problem not using the specified
>>> linker...?
>>
>> Yes, since it's produced by an invocation of $(CC) which is expected
>> to call the linker implicitly, and this is how the toolchain's default
>> linker ends up getting used even when you set LD on the command line.
>>
>> If I'm not mistaken, implicitly performing a link through the compiler
>> seems to be 1) conventional for all VDSOs, not just ARM's, but 2) unusual
>> for other parts of the kernel build.
>>
>> I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
>> would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
>> may suffice:
>>
>> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
>> index 8aa791051029..da0ce897edde 100644
>> --- a/arch/arm/vdso/Makefile
>> +++ b/arch/arm/vdso/Makefile
>> @@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
>>        cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^)
>> $(filter %.o,$^) \
>>                     $(call cc-ldoption, -Wl$(comma)--build-id) \
>>                     -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
>> +                   $(call cc-option, -fuse-ld=bfd) \
>>                     -Wl,-z,common-page-size=4096 -o $@
>>
>>  quiet_cmd_vdsomunge = MUNGE   $@
> 
> 
> Thanks Nathan, your solution looks reasonable. I figured that the option
> is available since GCC 4.8, so it should work probably for most
> toolchains which try to make use of gold linker as default.
> 
> Unfortunately, I hit another problem with the configuration on my build
> host. With your patch applied, I get:
> collect2: fatal error: cannot find 'ld'
> compilation terminated.
> make[2]: *** [arch/arm/vdso/vdso.so.raw] Error 1
> make[1]: *** [arch/arm/vdso] Error 2
> make[1]: *** Waiting for unfinished jobs....

Sorry for the noise, it turns out when using no paths in CROSS_COMPILE
and instead add the location of the toolchain to PATH properly, this
second issue is fixed.

--
Stefan

Bulid regression with VDSO enabled

[Nathan Lynch]

On 05/09/2015 08:25 AM, Stefan Agner wrote:
> On 2015-05-09 14:41, Stefan Agner wrote:
>> On 2015-05-08 23:15, Nathan Lynch wrote:
>>> On 05/08/2015 11:08 AM, Stefan Agner wrote:
>>>> On 2015-05-08 17:27, Nathan Lynch wrote:
>>>>> On 05/08/2015 06:13 AM, Stefan Agner wrote:
>>>>>> Also, I used the BFD linker by adding LD=${CROSS_COMPILE}ld.bfd to the
>>>>>> build command. Interestingly thought, I had the same issue when using
>>>>>> gold linker...
>>>>>
>>>>> When I try this, ld.gold is used regardless.
>>>>>
>>>>> You can tell by doing and checking for a NT_GNU_GOLD_VERSION note:
>>>>>
>>>>> $ readelf -n arch/arm/vdso/vdso.so.raw
>>>>>
>>>>> Displaying notes found at file offset 0x000001bc with length 0x00000040:
>>>>>   Owner                 Data size       Description
>>>>>   GNU                  0x00000009       NT_GNU_GOLD_VERSION (gold version)
>>>>>   GNU                  0x00000014       NT_GNU_BUILD_ID (unique build
>>>>> ID bitstring)
>>>>>     Build ID: f025409550acb7f955c61d95691291da078e6688
>>>>
>>>> Hm, you are right. When deleting moving ld.bfd to ld, which makes the
>>>> BFD linker as default, compiling works flawless.
>>>>
>>>> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
>>>> objects, at least those do not have this note. For instance time.o does
>>>> not return anything:
>>>>
>>>> $ readelf -n arch/arm/kernel/time.o
>>>
>>> This is just an object file that hasn't been linked, but your general
>>> point stands -- LD is honored by other parts of the build.
>>>
>>>
>>>> So it seems to be a vdso.so.raw specific problem not using the specified
>>>> linker...?
>>>
>>> Yes, since it's produced by an invocation of $(CC) which is expected
>>> to call the linker implicitly, and this is how the toolchain's default
>>> linker ends up getting used even when you set LD on the command line.
>>>
>>> If I'm not mistaken, implicitly performing a link through the compiler
>>> seems to be 1) conventional for all VDSOs, not just ARM's, but 2) unusual
>>> for other parts of the kernel build.
>>>
>>> I suppose it's possible to make arch/arm/vdso/Makefile honor LD, but it
>>> would basically entail a rewrite.  In the meantime, using -fuse-ld=bfd
>>> may suffice:
>>>
>>> diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
>>> index 8aa791051029..da0ce897edde 100644
>>> --- a/arch/arm/vdso/Makefile
>>> +++ b/arch/arm/vdso/Makefile
>>> @@ -43,6 +43,7 @@ quiet_cmd_vdsold = VDSO    $@
>>>        cmd_vdsold = $(CC) $(c_flags) -Wl,-T $(filter %.lds,$^)
>>> $(filter %.o,$^) \
>>>                     $(call cc-ldoption, -Wl$(comma)--build-id) \
>>>                     -Wl,-Bsymbolic -Wl,-z,max-page-size=4096 \
>>> +                   $(call cc-option, -fuse-ld=bfd) \
>>>                     -Wl,-z,common-page-size=4096 -o $@
>>>
>>>  quiet_cmd_vdsomunge = MUNGE   $@
>>
>>
>> Thanks Nathan, your solution looks reasonable. I figured that the option
>> is available since GCC 4.8, so it should work probably for most
>> toolchains which try to make use of gold linker as default.
>>
>> Unfortunately, I hit another problem with the configuration on my build
>> host. With your patch applied, I get:
>> collect2: fatal error: cannot find 'ld'
>> compilation terminated.
>> make[2]: *** [arch/arm/vdso/vdso.so.raw] Error 1
>> make[1]: *** [arch/arm/vdso] Error 2
>> make[1]: *** Waiting for unfinished jobs....
> 
> Sorry for the noise, it turns out when using no paths in CROSS_COMPILE
> and instead add the location of the toolchain to PATH properly, this
> second issue is fixed.

I should have mentioned that this is the tradeoff with using -fuse-ld...
the requested linker needs to be in $PATH.

So, you lose if:
1. You're using e.g. CROSS_COMPILE=/usr/local/bin/arm-linux-gnu-gcc;
2. /usr/local/bin isn't in PATH;
3. Your toolchain is configured to use gold by default;
4. And you enable CONFIG_VDSO.

I am okay with accepting this as a limitation if Russell is.

Bulid regression with VDSO enabled

[Russell King - ARM Linux]

On Fri, May 08, 2015 at 06:08:11PM +0200, Stefan Agner wrote:
> However, so far LD=${CROSS_COMPILE}ld.bfc seems to work for other build
> objects, at least those do not have this note. For instance time.o does
> not return anything:
> 
> $ readelf -n arch/arm/kernel/time.o
> 
> So it seems to be a vdso.so.raw specific problem not using the specified
> linker...?

Remember that most of the kernel source is about building the kernel,
where we explicitly call the linker directly.

This is about building a user visible object, and we go through the
compiler front end for that.

My only worry is if we were to pick up (eg) libgcc.so* as a dependency,
but that's highly unlikely to go unnoticed if it were to happen, as my
cross-toolchain doesn't have libgcc.so available. ;)

Given that there's gold out there which doesn't work for the VDSO, I
think the solution Nathan has come up with in his reply to your email
is the best resolution to this.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.