Link failure with Debian toolchain

Link failure with Debian toolchain

[Joel Stanley]

Hi Stafford,

I'm not sure if you were aware, but Debian now has a or1k cross compile:

$ apt policy gcc-or1k-elf
gcc-or1k-elf:
  Installed: 12.1.0-2+1.0.2
$ apt policy binutils-or1k-elf
binutils-or1k-elf:
  Installed: 2.38-2+1.0.3

I've used it to build a mork1x litex soc so I think it works in some
situations at least.

I was trying to build your latest Linux kernel changes, applied on top
of v5.18, however the bug reproduces with plain v5.18.

$ CROSS_COMPILE=or1k-elf- ARCH=openrisc make O=or1k or1klitex_defconfig
$ CROSS_COMPILE=or1k-elf- ARCH=openrisc make O=or1k -j8

This fails at the link step:

  LD      vmlinux.o
+ or1k-elf-ld -r -o vmlinux.o --whole-archive
arch/openrisc/kernel/head.o init/built-in.a usr/built-in.a
arch/openrisc/built-in.a kernel/built-in.a certs/built-in.a
mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a
crypto/built-in.a block/built-in.a lib/built-in.a drivers/built-in.a
sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive
--start-group lib/lib.a /usr/lib/gcc/or1k-elf/12/libgcc.a --end-group
or1k-elf-ld: /usr/lib/gcc/or1k-elf/12/libgcc.a: error adding symbols:
archive has no index; run ranlib to add one

The libgcc.a appears to have the expected files inside it.

I tried running ar -a libgcc.a, and that changed the error to look like this:

or1k-elf-ld: init/main.o: in function `do_one_initcall':
main.c:(.init.text+0x12ec): undefined reference to `__muldi3'
main.c:(.init.text+0x12ec): relocation truncated to fit:
R_OR1K_INSN_REL_26 against undefined symbol `__muldi3'

With a lot more undefined reference errors for __muldi3, __ashldi3,
and __lshrdi3.

I'm not sure if this is a toolchain error or a kernel one. The kernel
appears to be doing the right thing.

I've cc'd Nicholas who is the Debian maintainer for the package in
case he has any ideas.

Cheers,

Joel

Re: Link failure with Debian toolchain

[Stafford Horne]

Hi Joel,

On Thu, May 26, 2022 at 11:26:45PM +0000, Joel Stanley wrote:
> Hi Stafford,
> 
> I'm not sure if you were aware, but Debian now has a or1k cross compile:
> 
> $ apt policy gcc-or1k-elf
> gcc-or1k-elf:
>   Installed: 12.1.0-2+1.0.2
> $ apt policy binutils-or1k-elf
> binutils-or1k-elf:
>   Installed: 2.38-2+1.0.3

I didn't know about this. but it looks right.

> I've used it to build a mork1x litex soc so I think it works in some
> situations at least.
> 
> I was trying to build your latest Linux kernel changes, applied on top
> of v5.18, however the bug reproduces with plain v5.18.
> 
> $ CROSS_COMPILE=or1k-elf- ARCH=openrisc make O=or1k or1klitex_defconfig
> $ CROSS_COMPILE=or1k-elf- ARCH=openrisc make O=or1k -j8

This looks good too. What is the meaning of O=or1k? I never have used that
before.

> This fails at the link step:
> 
>   LD      vmlinux.o
> + or1k-elf-ld -r -o vmlinux.o --whole-archive
> arch/openrisc/kernel/head.o init/built-in.a usr/built-in.a
> arch/openrisc/built-in.a kernel/built-in.a certs/built-in.a
> mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a
> crypto/built-in.a block/built-in.a lib/built-in.a drivers/built-in.a
> sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive
> --start-group lib/lib.a /usr/lib/gcc/or1k-elf/12/libgcc.a --end-group
> or1k-elf-ld: /usr/lib/gcc/or1k-elf/12/libgcc.a: error adding symbols:
> archive has no index; run ranlib to add one
> 
> The libgcc.a appears to have the expected files inside it.
> 
> I tried running ar -a libgcc.a, and that changed the error to look like this:
> 
> or1k-elf-ld: init/main.o: in function `do_one_initcall':
> main.c:(.init.text+0x12ec): undefined reference to `__muldi3'
> main.c:(.init.text+0x12ec): relocation truncated to fit:
> R_OR1K_INSN_REL_26 against undefined symbol `__muldi3'
> 
> With a lot more undefined reference errors for __muldi3, __ashldi3,
> and __lshrdi3.

This seems to mean that the builtins for 64-bit ops are not being found. So
something is wrong with libgcc symbols.  Maybe something strange with the
multilib args used when building the toolchain.

I'll have a look at it. It would be nice if I can used this toolchain for my CI.

-Stafford
> I'm not sure if this is a toolchain error or a kernel one. The kernel
> appears to be doing the right thing.
> 
> I've cc'd Nicholas who is the Debian maintainer for the package in
> case he has any ideas.
> 
> Cheers,
> 
> Joel

Re: Link failure with Debian toolchain

[Nicolas Boulenguez]

> This seems to mean that the builtins for 64-bit ops are not being found. So
> something is wrong with libgcc symbols.  Maybe something strange with the
> multilib args used when building the toolchain.
> I'll have a look at it. It would be nice if I can used this toolchain for my CI.

In case it helps, the options passed to ./configure are visible here:
https://salsa.debian.org/debian/gcc-or1k-elf/-/blob/master/debian/rules#L45

Re: Link failure with Debian toolchain

[Joel Stanley]

On Fri, 27 May 2022 at 22:38, Stafford Horne <shorne@gmail.com> wrote:
>
> > I've used it to build a mork1x litex soc so I think it works in some
> > situations at least.
> >
> > I was trying to build your latest Linux kernel changes, applied on top
> > of v5.18, however the bug reproduces with plain v5.18.
> >
> > $ CROSS_COMPILE=or1k-elf- ARCH=openrisc make O=or1k or1klitex_defconfig
> > $ CROSS_COMPILE=or1k-elf- ARCH=openrisc make O=or1k -j8
>
> This looks good too. What is the meaning of O=or1k? I never have used that
> before.

O is the objdir, where the output is placed. It's handy for using the
one source tree to do multiple builds:

$ make help | grep O
   make O=dir [targets] Locate all output files in "dir", including .config

Re: Link failure with Debian toolchain

[Joel Stanley]

On Sat, 28 May 2022 at 09:50, Nicolas Boulenguez <nicolas@debian.org> wrote:
>
> > This seems to mean that the builtins for 64-bit ops are not being found. So
> > something is wrong with libgcc symbols.  Maybe something strange with the
> > multilib args used when building the toolchain.
> > I'll have a look at it. It would be nice if I can used this toolchain for my CI.
>
> In case it helps, the options passed to ./configure are visible here:
> https://salsa.debian.org/debian/gcc-or1k-elf/-/blob/master/debian/rules#L45

Thanks for the link. I did a local build of gcc 12 and binutils 2.38
with the same arguments as the Debian build and I was able to
successfully build the kernel.

Taking a closer look, it's not the configure line but another step. In
the gcc rules we have:

$(gnu_type)-strip --remove-section=.comment --remove-section=.note \
 debian/gcc-*/usr/lib/gcc/*/*/lib*.a

Stripping libgcc.a breaks it. I could reproduce by running:

  or1k-elf-strip lib/gcc/or1k-elf/12/libgcc.a

If I then reinstated the original libgcc.a the build succeeded again.
I suggest we drop that line from the Debian package.

Cheers,

Joel

Re: Link failure with Debian toolchain

[Joel Stanley]

On Mon, 30 May 2022 at 08:27, Joel Stanley <joel@jms.id.au> wrote:
> Stripping libgcc.a breaks it. I could reproduce by running:
>
>   or1k-elf-strip lib/gcc/or1k-elf/12/libgcc.a
>
> If I then reinstated the original libgcc.a the build succeeded again.
> I suggest we drop that line from the Debian package.

Nicolas merged my change to remove the strip from the package. Thanks!

I was able to build and boot a kernel with the new version of the
package 12.1.0-2+1.0.3.

I've proposed a change that enables the or1k target in the Debian
gdb-multiarch package:

 https://salsa.debian.org/gdb-team/gdb/-/merge_requests/11

With a local build of that I'm able to connect to qemu's gdb interface
and poke at the running machine.

Re: Link failure with Debian toolchain

[Stafford Horne]

On Thu, Jun 02, 2022 at 04:52:01AM +0000, Joel Stanley wrote:
> On Mon, 30 May 2022 at 08:27, Joel Stanley <joel@jms.id.au> wrote:
> > Stripping libgcc.a breaks it. I could reproduce by running:
> >
> >   or1k-elf-strip lib/gcc/or1k-elf/12/libgcc.a
> >
> > If I then reinstated the original libgcc.a the build succeeded again.
> > I suggest we drop that line from the Debian package.
> 
> Nicolas merged my change to remove the strip from the package. Thanks!
> 
> I was able to build and boot a kernel with the new version of the
> package 12.1.0-2+1.0.3.
> 
> I've proposed a change that enables the or1k target in the Debian
> gdb-multiarch package:
> 
>  https://salsa.debian.org/gdb-team/gdb/-/merge_requests/11
> 
> With a local build of that I'm able to connect to qemu's gdb interface
> and poke at the running machine.

Thanks for getting this sorted out. I have been on vacation for the last 2 weeks
and just following these on my phone but not able to get to a computer and do
any useful investigation.

-Stafford