CRC32=y && 8193TOO=m unresolved symbols

CRC32=y && 8193TOO=m unresolved symbols

[Stewart Smith]

this problem disappears when you have CONFIG_CRC32=y and 
CONFIG_8139TOO=y. It only happens when CRC32=y and 8139TOO=m

Occurs on all 2.5.70 through bk6 (haven't tried later). Also on -mm3

I get unresolved symbols for bitreverse and crc32_le.

I've tried mucking around a bit with EXPORT_SYMBOL and the like, but I 
have to admit, i'm stumped. Help! :)
------------
Stewart Smith
Vice President, Linux Australia
stewart@linux.org.au
http://www.linux.org.au

Re: CRC32=y && 8193TOO=m unresolved symbols

[David Woodhouse]

On Tue, 2003-06-03 at 02:13, Stewart Smith wrote:
> this problem disappears when you have CONFIG_CRC32=y and 
> CONFIG_8139TOO=y. It only happens when CRC32=y and 8139TOO=m
> 
> Occurs on all 2.5.70 through bk6 (haven't tried later). Also on -mm3
> 
> I get unresolved symbols for bitreverse and crc32_le.
> 
> I've tried mucking around a bit with EXPORT_SYMBOL and the like, but I 
> have to admit, i'm stumped. Help! :)

My current solution to this, which was sent to Alan but which didn't
appear in 2.4.21-rc6-ac2, is to export the symbols from kernel/ksyms.c
#ifdef CONFIG_CRC32 and from lib/crc32.c #ifndef CONFIG_CRC32.

That should work in all circumstances.


-- 
dwmw2

[PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols

[Stewart Smith]

Linus,
please apply - this fixes unresolved symbols when CONFIG_CRC32=y and 
CONFIG_8139TOO=m (it also appeared on some other ethernet device 
drivers). I think this is the right way to fix this problem. It at 
least now builds, links and boots (and hey, even my ethernet works so 
it can't all be bad :)

patches cleanly against 2.5.70 and 2.5.70-bk8

--- linux-2.5.70-orig/include/linux/crc32.h	2003-05-05 
09:53:08.000000000 +1000
+++ linux-2.5.70-stew3/include/linux/crc32.h	2003-06-04 
15:27:34.000000000 +1000
@@ -6,6 +6,7 @@
  #define _LINUX_CRC32_H

  #include <linux/types.h>
+#include <linux/module.h>

  extern u32  crc32_le(u32 crc, unsigned char const *p, size_t len);
  extern u32  crc32_be(u32 crc, unsigned char const *p, size_t len);
@@ -21,7 +22,16 @@
   * is in bit nr 0], thus it must be reversed before use. Except for
   * nics that bit swap the result internally...
   */
-#define ether_crc(length, data)    bitreverse(crc32_le(~0, data, 
length))
-#define ether_crc_le(length, data) crc32_le(~0, data, length)
+static inline u32 ether_crc(size_t length, unsigned char const *data)
+{
+  return bitreverse(crc32_le(~0, data, length));
+}
+EXPORT_SYMBOL(ether_crc);
+
+static inline u32 ether_crc_le(size_t length, unsigned char const 
*data)
+{
+  return crc32_le(~0, data, length);
+}
+EXPORT_SYMBOL(ether_crc_le);

  #endif /* _LINUX_CRC32_H */
--- linux-2.5.70-orig/kernel/ksyms.c	2003-06-02 23:28:32.000000000 +1000
+++ linux-2.5.70-stew3/kernel/ksyms.c	2003-06-04 15:11:37.000000000 
+1000
@@ -58,6 +58,7 @@
  #include <linux/time.h>
  #include <linux/backing-dev.h>
  #include <linux/percpu_counter.h>
+#include <linux/crc32.h>
  #include <asm/checksum.h>

  #if defined(CONFIG_PROC_FS)


------------
Stewart Smith
Vice President, Linux Australia
stewart@linux.org.au
http://www.linux.org.au

Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols

[Jeff Garzik]

On Wed, Jun 04, 2003 at 03:56:09PM +1000, Stewart Smith wrote:
> +{
> +  return bitreverse(crc32_le(~0, data, length));
> +}
> +EXPORT_SYMBOL(ether_crc);
> +
> +static inline u32 ether_crc_le(size_t length, unsigned char const 
> *data)
> +{
> +  return crc32_le(~0, data, length);
> +}
> +EXPORT_SYMBOL(ether_crc_le);

You can't EXPORT_SYMBOL from a header.

This sounds like Kconfig or Makefile bugs to me... all the
export-symbol stuff should already be in place.

Can you post your .config and the exact build errors you are getting?

	Jeff

Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols

[David Woodhouse]

On Wed, 2003-06-04 at 16:28, Jeff Garzik wrote:

> You can't EXPORT_SYMBOL from a header.
> 
> This sounds like Kconfig or Makefile bugs to me... all the
> export-symbol stuff should already be in place.
> 
> Can you post your .config and the exact build errors you are getting?

It's because lib/crc32.o isn't actually _referenced_ by anything, hence
isn't actually pulled into vmlinux from lib/lib.a.

My fix in the 2.4 tree is to export its symbols from kernel/ksyms.c
#ifdef CONFIG_CRC32, and to export its symbols from lib/crc32.c 
#ifndef CONFIG_CRC32.

-- 
dwmw2

Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols

[Jeff Garzik]

On Wed, Jun 04, 2003 at 04:30:49PM +0100, David Woodhouse wrote:
> On Wed, 2003-06-04 at 16:28, Jeff Garzik wrote:
> 
> > You can't EXPORT_SYMBOL from a header.
> > 
> > This sounds like Kconfig or Makefile bugs to me... all the
> > export-symbol stuff should already be in place.
> > 
> > Can you post your .config and the exact build errors you are getting?
> 
> It's because lib/crc32.o isn't actually _referenced_ by anything, hence
> isn't actually pulled into vmlinux from lib/lib.a.
> 
> My fix in the 2.4 tree is to export its symbols from kernel/ksyms.c
> #ifdef CONFIG_CRC32, and to export its symbols from lib/crc32.c 
> #ifndef CONFIG_CRC32.

That makes sense.

Any opinions on moving it out of lib/lib.a?

We have our own conditional linking system, essentially, so that's what
I would prefer.

	Jeff

Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols

[Linus Torvalds]

On Wed, 4 Jun 2003, Jeff Garzik wrote:
> 
> Any opinions on moving it out of lib/lib.a?
> 
> We have our own conditional linking system, essentially, so that's what
> I would prefer.

That makes sense. lib/lib.a wasn't ever _that_ sensible, since we only 
really include object files in it that we know should be linked in. The 
linker really does know less than the build system, and in this case that 
seems to be causing a real bug.

		Linus

[EVIL-PATCH] getting rid of lib/lib.a and breaking many archs in the processes (was Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols)

[Stewart Smith]

On Wed, Jun 04, 2003 at 08:41:59AM -0700, Linus Torvalds wrote:
> On Wed, 4 Jun 2003, Jeff Garzik wrote:
> > Any opinions on moving it out of lib/lib.a?
>
> That makes sense. lib/lib.a wasn't ever _that_ sensible, since we only 
> really include object files in it that we know should be linked in. The 
> linker really does know less than the build system, and in this case that 
> seems to be causing a real bug.

This patch needs a couple of extra things before it drops it's evil status, and I'm not sure how to do them exactly.

Instead of going the lib/lib.a route (which I presume was used so that the linker could decide weather to go with arch specific functions or the generic lib/ ones) it leaves it up to the arch to sort it all out. This could be fair enough, as most (14/20) archs seem to implement their own dump_stack anyway. However, bust_spinlocks could be a problem as only i386, mips64, x86_64 and s390 specfically implement it.

Is it a good idea to make the archs themselves include the generic implementation if they don't do it themselves? Or is there a way to detect this in the build system (this would be more elegant, but I have no idea how to do it).

Evil patch that will probably break 16 archs from compiling follows:

diff -urN linux-2.5.70-bk11-orig/Makefile linux-2.5.70-bk11stew1/Makefile
--- linux-2.5.70-bk11-orig/Makefile	2003-06-06 23:55:42.000000000 +1000
+++ linux-2.5.70-bk11stew1/Makefile	2003-06-07 01:09:05.000000000 +1000
@@ -288,7 +288,7 @@
 core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
 drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
 net-y		:= $(patsubst %/, %/built-in.o, $(net-y))
-libs-y		:= $(patsubst %/, %/lib.a, $(libs-y))
+libs-y		:= $(patsubst %/, %/built-in.o, $(libs-y))
 
 ifdef include_config
 
diff -urN linux-2.5.70-bk11-orig/arch/i386/lib/Makefile linux-2.5.70-bk11stew1/arch/i386/lib/Makefile
--- linux-2.5.70-bk11-orig/arch/i386/lib/Makefile	2003-05-05 09:53:01.000000000 +1000
+++ linux-2.5.70-bk11stew1/arch/i386/lib/Makefile	2003-06-07 17:22:39.000000000 +1000
@@ -2,8 +2,6 @@
 # Makefile for i386-specific library files..
 #
 
-L_TARGET = lib.a
-
 obj-y = checksum.o delay.o \
 	usercopy.o getuser.o \
 	memcpy.o strstr.o
diff -urN linux-2.5.70-bk11-orig/lib/Makefile linux-2.5.70-bk11stew1/lib/Makefile
--- linux-2.5.70-bk11-orig/lib/Makefile	2003-06-02 23:28:32.000000000 +1000
+++ linux-2.5.70-bk11stew1/lib/Makefile	2003-06-07 17:21:44.000000000 +1000
@@ -6,10 +6,8 @@
 # unless it's something special (ie not a .c file).
 #
 
-L_TARGET := lib.a
-
 obj-y := errno.o ctype.o string.o vsprintf.o cmdline.o \
-	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
+	 rbtree.o radix-tree.o \
 	 kobject.o idr.o
 
 obj-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o

Re: [EVIL-PATCH] getting rid of lib/lib.a and breaking many archs in the processes (was Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols)

[Christoph Hellwig]

On Sat, Jun 07, 2003 at 05:33:21PM +1000, Stewart Smith wrote:
> Is it a good idea to make the archs themselves include the generic implementation if they don't do it themselves? Or is there a way to detect this in the build system (this would be more elegant, but I have no idea how to do it).

You can always add a HAVE_ARCH_FOO #define.  Btw, we have so many
of those these days that an <asm/config.h> for them might be a better
choice than polluting random other header.  Opinions?

Re: [EVIL-PATCH] getting rid of lib/lib.a and breaking many archs in the processes (was Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols)

[Linus Torvalds]

On Sat, 7 Jun 2003, Stewart Smith wrote:
> 
> This patch needs a couple of extra things before it drops it's evil
> status, and I'm not sure how to do them exactly.

How about making the lib/ directory separate out the "these are optional"  
pieces from the "these are basic libs that got explicitly enabled by the
user", and leave the archive usage for the truly optional stuff?

There's nothing inherently wrong with using the archive format per se, and
it still makes sense to just let the linker do the simple stuff instead of
adding unnecessary code to the configuration management to do stuff that
the linker would give us for free.

The _clean_ way to do this (I think) might be to make the normal build 
rules understand a "obj-l" for "library objects", and always build a 
"lib.a" for those, instead of having the magic "if there is a L_TARGET, 
use obj-y and make a library of them" special case rule.

Sam, can you try to sprinkle the proper Makefile magic pixie-dust around?


		Linus

Re: [EVIL-PATCH] getting rid of lib/lib.a and breaking many archs in the processes (was Re: [PATCH] fixed: CRC32=y && 8193TOO=m unresolved symbols)

[Sam Ravnborg]

On Sat, Jun 07, 2003 at 10:34:01AM -0700, Linus Torvalds wrote:
> The _clean_ way to do this (I think) might be to make the normal build 
> rules understand a "obj-l" for "library objects", and always build a 
> "lib.a" for those, instead of having the magic "if there is a L_TARGET, 
> use obj-y and make a library of them" special case rule.
> 
> Sam, can you try to sprinkle the proper Makefile magic pixie-dust around?

New variable introduced: lib-y
The lib-y syntax allows you to do the usual tricks such as:
lib-$(CONFIG_SMP) += percpu_counter.o

A built-in.o is always present in a directory that list .o files in
either obj-* or lib-*.
lib.a is made only when lib-y is defined.
In the top-level Makefile I now always have a lib.a and built-in.o for
directories listed in libs-y (libs-y is only used in architecture
specific makefiles).

I am considering removing the requirement to list directories
that uses lib-y in libs-y. Kbuild has the information, so no need
to do it. It just looks a bit too ugly at first glance.

I updated lib/makefile  [crc32.o is now always build-in if selected].
And i386/lib as well.

Does this meet what you had in mind?

	Sam

 Makefile               |    4 +++-
 arch/i386/lib/Makefile |    9 ++++-----
 lib/Makefile           |   11 +++++------
 scripts/Makefile.build |   39 ++++++++++++++++++---------------------
 scripts/Makefile.lib   |    7 +++++++
 5 files changed, 37 insertions(+), 33 deletions(-)

===== Makefile 1.410 vs edited =====
--- 1.410/Makefile	Tue Jun  3 23:27:14 2003
+++ edited/Makefile	Sat Jun  7 19:57:35 2003
@@ -288,7 +288,9 @@
 core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
 drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
 net-y		:= $(patsubst %/, %/built-in.o, $(net-y))
-libs-y		:= $(patsubst %/, %/lib.a, $(libs-y))
+libs-y1		:= $(patsubst %/, %/lib.a, $(libs-y))
+libs-y2		:= $(patsubst %/, %/built-in.o, $(libs-y))
+libs-y		:= $(libs-y1) $(libs-y2)
 
 ifdef include_config
 
===== arch/i386/lib/Makefile 1.11 vs edited =====
--- 1.11/arch/i386/lib/Makefile	Sat Dec 14 13:38:56 2002
+++ edited/arch/i386/lib/Makefile	Sat Jun  7 20:05:00 2003
@@ -2,12 +2,11 @@
 # Makefile for i386-specific library files..
 #
 
-L_TARGET = lib.a
 
-obj-y = checksum.o delay.o \
+lib-y = checksum.o delay.o \
 	usercopy.o getuser.o \
 	memcpy.o strstr.o
 
-obj-$(CONFIG_X86_USE_3DNOW) += mmx.o
-obj-$(CONFIG_HAVE_DEC_LOCK) += dec_and_lock.o
-obj-$(CONFIG_DEBUG_IOVIRT)  += iodebug.o
+lib-$(CONFIG_X86_USE_3DNOW) += mmx.o
+lib-$(CONFIG_HAVE_DEC_LOCK) += dec_and_lock.o
+lib-$(CONFIG_DEBUG_IOVIRT)  += iodebug.o
===== lib/Makefile 1.22 vs edited =====
--- 1.22/lib/Makefile	Mon May 12 22:20:36 2003
+++ edited/lib/Makefile	Sat Jun  7 20:36:14 2003
@@ -6,18 +6,17 @@
 # unless it's something special (ie not a .c file).
 #
 
-L_TARGET := lib.a
 
-obj-y := errno.o ctype.o string.o vsprintf.o cmdline.o \
+lib-y := errno.o ctype.o string.o vsprintf.o cmdline.o \
 	 bust_spinlocks.o rbtree.o radix-tree.o dump_stack.o \
 	 kobject.o idr.o
 
-obj-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o
-obj-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
-obj-$(CONFIG_SMP) += percpu_counter.o
+lib-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o
+lib-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
+lib-$(CONFIG_SMP) += percpu_counter.o
 
 ifneq ($(CONFIG_HAVE_DEC_LOCK),y) 
-  obj-y += dec_and_lock.o
+  lib-y += dec_and_lock.o
 endif
 
 obj-$(CONFIG_CRC32)	+= crc32.o
===== scripts/Makefile.build 1.37 vs edited =====
--- 1.37/scripts/Makefile.build	Tue Jun  3 23:14:57 2003
+++ edited/scripts/Makefile.build	Sat Jun  7 21:01:41 2003
@@ -32,9 +32,7 @@
 endif
 
 ifdef L_TARGET
-ifneq ($(L_TARGET),lib.a)
-$(warning kbuild: $(obj)/Makefile - L_TARGET := $(L_TARGET) target shall be renamed to lib.a. Please fix!)
-endif
+$(error kbuild: $(obj)/Makefile - Use of L_TARGET is replaced by lib-y in 2.5. Please fix!)
 endif
 
 ifdef list-multi
@@ -47,21 +45,19 @@
 
 # ===========================================================================
 
-# If a Makefile does not define a L_TARGET, link an object called "built-in.o"
-
-ifdef L_TARGET
-L_TARGET := $(obj)/$(L_TARGET)
-else
-ifneq ($(strip $(obj-y) $(obj-m) $(obj-n) $(obj-)),)
-O_TARGET := $(obj)/built-in.o
+ifneq ($(strip $(lib-y) $(lib-m) $(lib-n) $(lib-)),)
+lib-target := $(obj)/lib.a
 endif
+
+ifneq ($(strip $(obj-y) $(obj-m) $(obj-n) $(obj-) $(lib-target)),)
+builtin-target := $(obj)/built-in.o
 endif
 
 # We keep a list of all modules in $(MODVERDIR)
 
 touch-module = @echo $(@:.o=.ko) > $(MODVERDIR)/$(@F:.o=.mod)
 
-__build: $(if $(KBUILD_BUILTIN),$(O_TARGET) $(L_TARGET) $(extra-y)) \
+__build: $(if $(KBUILD_BUILTIN),$(builtin-target) $(lib-target) $(extra-y)) \
 	 $(if $(KBUILD_MODULES),$(obj-m)) \
 	 $(subdir-ym) $(always)
 	@:
@@ -203,7 +199,8 @@
 %.o: %.S FORCE
 	$(call if_changed_dep,as_o_S)
 
-targets += $(real-objs-y) $(real-objs-m) $(extra-y) $(MAKECMDGOALS) $(always)
+targets += $(real-objs-y) $(real-objs-m) $(lib-y)
+targets += $(extra-y) $(MAKECMDGOALS) $(always)
 
 # Build the compiled-in targets
 # ---------------------------------------------------------------------------
@@ -214,30 +211,30 @@
 #
 # Rule to compile a set of .o files into one .o file
 #
-ifdef O_TARGET
+ifdef builtin-target
 quiet_cmd_link_o_target = LD      $@
-# If the list of objects to link is empty, just create an empty O_TARGET
+# If the list of objects to link is empty, just create an empty built-in.o
 cmd_link_o_target = $(if $(strip $(obj-y)),\
 		      $(LD) $(ld_flags) -r -o $@ $(filter $(obj-y), $^),\
 		      rm -f $@; $(AR) rcs $@)
 
-$(O_TARGET): $(obj-y) FORCE
+$(builtin-target): $(obj-y) FORCE
 	$(call if_changed,link_o_target)
 
-targets += $(O_TARGET)
-endif # O_TARGET
+targets += $(builtin-target)
+endif # builtin-target
 
 #
 # Rule to compile a set of .o files into one .a file
 #
-ifdef L_TARGET
+ifdef lib-target
 quiet_cmd_link_l_target = AR      $@
-cmd_link_l_target = rm -f $@; $(AR) $(EXTRA_ARFLAGS) rcs $@ $(obj-y)
+cmd_link_l_target = rm -f $@; $(AR) $(EXTRA_ARFLAGS) rcs $@ $(lib-y)
 
-$(L_TARGET): $(obj-y) FORCE
+$(lib-target): $(lib-y) FORCE
 	$(call if_changed,link_l_target)
 
-targets += $(L_TARGET)
+targets += $(lib-target)
 endif
 
 #
===== scripts/Makefile.lib 1.18 vs edited =====
--- 1.18/scripts/Makefile.lib	Mon Mar 10 22:03:33 2003
+++ edited/scripts/Makefile.lib	Sat Jun  7 20:55:00 2003
@@ -18,6 +18,12 @@
 
 obj-m := $(filter-out $(obj-y),$(obj-m))
 
+# Libraries are always collected in one lib file.
+# Filter out objects already built-in
+
+lib-y := $(filter-out $(obj-y), $(sort $(lib-y) $(lib-m)))
+
+
 # Handle objects in subdirs
 # ---------------------------------------------------------------------------
 # o if we encounter foo/ in $(obj-y), replace it by foo/built-in.o
@@ -91,6 +97,7 @@
 targets		:= $(addprefix $(obj)/,$(targets))
 obj-y		:= $(addprefix $(obj)/,$(obj-y))
 obj-m		:= $(addprefix $(obj)/,$(obj-m))
+lib-y		:= $(addprefix $(obj)/,$(lib-y))
 subdir-obj-y	:= $(addprefix $(obj)/,$(subdir-obj-y))
 real-objs-y	:= $(addprefix $(obj)/,$(real-objs-y))
 real-objs-m	:= $(addprefix $(obj)/,$(real-objs-m))