[RFC] Force build error on undefined symbols

[RFC] Force build error on undefined symbols

[Russell King]

Hi,

I've checked the date, and it isn't April Fools day.  I wish it was
though.

It appears that all binutils versions for ARM which are capable of
building 2.6 kernels which have been tested so far contain a serious
bug - it is possible to successfully link an object and still have
various symbols undefined.  Currently, these binutils have been
tested on ARM (as cross-compilers and/or native) and found wanting:

GNU assembler 2.13.90.0.18 20030206
GNU assembler 2.14 20030612
GNU assembler 2.14.90 20031229
GNU ld version 2.14.90.0.7 20031029 Debian GNU/Linux
Assembleur GNU 2.15.90.0.1 20040303

So far, we have discovered two cases:

1. When building a certain file, an undefined symbolic constant
   (TI_USED_CP) ended up in the symbol table without a relocation,
   and the value the assembler decided to use was '0'.  The effect
   of this is that we ended up setting bits in thread_info->flags.

   This appears to be a binutils "as" error.

2. When building the decompressor for ARM kernels, GCC appears to
   inexplicably emit ".global" directives for symbols not defined
   in the files being built, even though the symbols themselves are
   not actually used.  I'm not sure whether this is a real bug;
   binutils on x86 appears to accept and link such objects.

In both cases, the linker successfully created executable programs
which ran.  In the first case, it is a silent error; the kernel had
been linked, and able to run, but the program is not correct.

Obviously, the one true correct solution is to fix the toolchain and
upgrade to the latest version.  However, since we have potentially
multiple binutils versions spread across more than a year affected,
I think we need to detect such errors as well.

Therefore, I propose the following patch to detect undefined symbols
in the final image and force an error if this is the case.

Comments?

--- orig/Makefile	Sat Apr 10 12:31:36 2004
+++ linux/Makefile	Sat Apr 10 13:01:05 2004
@@ -502,7 +502,8 @@ define cmd_vmlinux__
 	$(net-y) \
 	--end-group \
 	$(filter .tmp_kallsyms%,$^) \
-	-o $@
+	-o $@; \
+	$(NM) $@ | egrep -q '^ +U ' && { echo "Link failed: undefined symbols found in final object."; $(NM) $@ | egrep '^ +U '; rm -f $@; exit 1; } || :
 endef
 
 #	set -e makes the rule exit immediately on error
--- orig/arch/arm/boot/compressed/Makefile	Sat Apr 10 12:31:36 2004
+++ linux/arch/arm/boot/compressed/Makefile	Sat Apr 10 13:01:13 2004
@@ -68,6 +68,7 @@ LDFLAGS_vmlinux := -p -X \
 $(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.o \
 	 	$(addprefix $(obj)/, $(OBJS)) FORCE
 	$(call if_changed,ld)
+	@$(NM) $@ | egrep -q '^ +U ' && { echo "Link failed: undefined symbols found in final object."; $(NM) $@ | egrep '^ +U '; rm -f $@; exit 1; } || :
 	@:
 
 

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Re: [RFC] Force build error on undefined symbols

[Sam Ravnborg]

> 
> Therefore, I propose the following patch to detect undefined symbols
> in the final image and force an error if this is the case.
> 
> Comments?

Do we really want to do this for all architectures?
You could use something like the attached to restrict it to arm.

	Sam

--- v2.6/arch/arm/Makefile	2004-04-05 23:23:54.000000000 +0200
+++ mm4/arch/arm/Makefile	2004-04-10 16:21:29.000000000 +0200
@@ -140,14 +143,19 @@
 maketools: include/asm-arm/constants.h include/linux/version.h FORCE
 	$(Q)$(MAKE) $(build)=arch/arm/tools include/asm-arm/mach-types.h
 
+check-undefsyms = $(NM) $^ | egrep -q '^ +U ' && { echo "Link failed: undefined symbols found in $^:"; $(NM) $^ | egrep '^ +U '; rm -f $^; exit 1; } || : endef
+
 # Convert bzImage to zImage
 bzImage: vmlinux
+	$(check-undefsyms)
 	$(Q)$(MAKE) $(build)=$(boot) $(boot)/zImage
 
 zImage Image bootpImage uImage: vmlinux
+	$(check-undefsyms)
 	$(Q)$(MAKE) $(build)=$(boot) $(boot)/$@
 
 zinstall install: vmlinux
+	$(check-undefsyms)
 	$(Q)$(MAKE) $(build)=$(boot) $@
 
 CLEAN_FILES += include/asm-arm/constants.h* include/asm-arm/mach-types.h \

Re: [RFC] Force build error on undefined symbols

[Russell King]

On Sat, Apr 10, 2004 at 04:24:01PM +0200, Sam Ravnborg wrote:
> > 
> > Therefore, I propose the following patch to detect undefined symbols
> > in the final image and force an error if this is the case.
> > 
> > Comments?
> 
> Do we really want to do this for all architectures?
> You could use something like the attached to restrict it to arm.

Unfortunately, some people got it into their heads to use the top
level vmlinux directly, so this wouldn't always catch the problem.
It really needs to be done immediately after vmlinux is generated
to ensure that all cases are caught.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Re: [RFC] Force build error on undefined symbols

[Sam Ravnborg]

On Sat, Apr 10, 2004 at 03:35:19PM +0100, Russell King wrote:
> On Sat, Apr 10, 2004 at 04:24:01PM +0200, Sam Ravnborg wrote:
> > > 
> > > Therefore, I propose the following patch to detect undefined symbols
> > > in the final image and force an error if this is the case.
> > > 
> > > Comments?
> > 
> > Do we really want to do this for all architectures?
> > You could use something like the attached to restrict it to arm.
> 
> Unfortunately, some people got it into their heads to use the top
> level vmlinux directly, so this wouldn't always catch the problem.
> It really needs to be done immediately after vmlinux is generated
> to ensure that all cases are caught.
OK.

How does output from your nm look?
My version (GNU nm 2.14.90.0.5 20030722 (SuSE Linux)) looks like this:
c04e56ac B zone_table

So I assume an undefined symbol would look like this:
00000000 U undef_symbol

This is not matched by "egrep -q '^ +U '"

	Sam

Re: [RFC] Force build error on undefined symbols

[Russell King]

On Sat, Apr 10, 2004 at 07:28:27PM +0200, Sam Ravnborg wrote:
> How does output from your nm look?
> My version (GNU nm 2.14.90.0.5 20030722 (SuSE Linux)) looks like this:
> c04e56ac B zone_table
> 
> So I assume an undefined symbol would look like this:
> 00000000 U undef_symbol

No, because an undefined symbol does not have an address associated with
it.  So, it looks like this:

         U symbol

More than one blank space at the beginning, U, then one space and the
symbol.  On binutils built on 32-bit architectures, 9 spaces prefixing
the 'U'.  On 64-bit architectures, 17 spaces prefixing the 'U'.

(Yes, even a cross-built binutils building for a 32-bit architecture
on a 64-bit architecture gives you 64-bit addresses.)

Does the format _really_ matter this much?

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core

Re: [RFC] Force build error on undefined symbols

[Brian Gerst]

Russell King wrote:
> Hi,
> 
> I've checked the date, and it isn't April Fools day.  I wish it was
> though.
> 
> It appears that all binutils versions for ARM which are capable of
> building 2.6 kernels which have been tested so far contain a serious
> bug - it is possible to successfully link an object and still have
> various symbols undefined.  Currently, these binutils have been
> tested on ARM (as cross-compilers and/or native) and found wanting:
> 
> GNU assembler 2.13.90.0.18 20030206
> GNU assembler 2.14 20030612
> GNU assembler 2.14.90 20031229
> GNU ld version 2.14.90.0.7 20031029 Debian GNU/Linux
> Assembleur GNU 2.15.90.0.1 20040303
> 
> So far, we have discovered two cases:
> 
> 1. When building a certain file, an undefined symbolic constant
>    (TI_USED_CP) ended up in the symbol table without a relocation,
>    and the value the assembler decided to use was '0'.  The effect
>    of this is that we ended up setting bits in thread_info->flags.
> 
>    This appears to be a binutils "as" error.
> 
> 2. When building the decompressor for ARM kernels, GCC appears to
>    inexplicably emit ".global" directives for symbols not defined
>    in the files being built, even though the symbols themselves are
>    not actually used.  I'm not sure whether this is a real bug;
>    binutils on x86 appears to accept and link such objects.
> 
> In both cases, the linker successfully created executable programs
> which ran.  In the first case, it is a silent error; the kernel had
> been linked, and able to run, but the program is not correct.
> 
> Obviously, the one true correct solution is to fix the toolchain and
> upgrade to the latest version.  However, since we have potentially
> multiple binutils versions spread across more than a year affected,
> I think we need to detect such errors as well.
> 
> Therefore, I propose the following patch to detect undefined symbols
> in the final image and force an error if this is the case.
> 
> Comments?

How about adding --no-undefined to LDFLAGS_vmlinux instead?

--
				Brian Gerst

Re: [RFC] Force build error on undefined symbols

[Russell King]

On Sat, Apr 10, 2004 at 03:33:07PM -0400, Brian Gerst wrote:
> How about adding --no-undefined to LDFLAGS_vmlinux instead?

Excellent. 8)

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core