[parisc-linux] kernel module relocation bug

[parisc-linux] kernel module relocation bug

[Randolph Chung]

currently the kernel module loader does not support long branch stubs.
using the debian 2.6.8-1 kernel, when loading the ipv6 module, i see:

legolas[6:43] linux-2.6% sudo modprobe ipv6
FATAL: Error inserting ipv6 (/lib/modules/2.6.8-1-32-smp/kernel/net/ipv6/ipv6.ko): Invalid module format

in syslog, i get:
module ipv6 relocation of symbol fold_prot_inuse is out of range (0x3ffeffe2 in 17 bits)

these are the other modules i have loaded at the moment:

legolas[6:26] linux-2.6% lsmod
Module                  Size  Used by
ehci_hcd               58788  0
ohci_hcd               41924  0
usbcore               121084  2 ehci_hcd,ohci_hcd
tulip                  65468  0
ad1889                 19524  0
soundcore              13828  1 ad1889
ac97_codec             24876  1 ad1889
tg3                   107760  0
ext3                  135588  2
jbd                   103960  1 ext3
sd_mod                 24416  5
sym53c8xx             105004  4

the kernel i have is:
Linux legolas 2.6.8-1-32-smp #1 SMP Tue Nov 2 13:07:05 MST 2004 parisc
GNU/Linux

to fix this we will probably need to introduce long branch stubs into 
into the kernel module loader. anybody want to give this a shot?

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[James Bottomley]

On Sat, 2004-11-27 at 00:47, Randolph Chung wrote:
> currently the kernel module loader does not support long branch stubs.
> using the debian 2.6.8-1 kernel, when loading the ipv6 module, i see:
> 
> legolas[6:43] linux-2.6% sudo modprobe ipv6
> FATAL: Error inserting ipv6 (/lib/modules/2.6.8-1-32-smp/kernel/net/ipv6/ipv6.ko): Invalid module format
> 
> in syslog, i get:
> module ipv6 relocation of symbol fold_prot_inuse is out of range (0x3ffeffe2 in 17 bits)


_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[James Bottomley]

On Sat, 2004-11-27 at 00:47, Randolph Chung wrote:
> currently the kernel module loader does not support long branch stubs.
> using the debian 2.6.8-1 kernel, when loading the ipv6 module, i see:
> 
> legolas[6:43] linux-2.6% sudo modprobe ipv6
> FATAL: Error inserting ipv6 (/lib/modules/2.6.8-1-32-smp/kernel/net/ipv6/ipv6.ko): Invalid module format

This is pretty nasty.  The problem isn't a bad relocation, its that the
relocation target is too far away (i.e. ipv6 is too big).

We get around this in the kernel by using -ffunction-sections so that
the real linker can insert stubs between functions.  However,
calculating how to do that in the in-kernel module loader would not be a
whole lot of fun.

James


_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[Ulrich Teichert]

Hi,

>> currently the kernel module loader does not support long branch stubs.
>> using the debian 2.6.8-1 kernel, when loading the ipv6 module, i see:
>> 
>> legolas[6:43] linux-2.6% sudo modprobe ipv6
>> FATAL: Error inserting ipv6 (/lib/modules/2.6.8-1-32-smp/kernel/net/ipv6/ipv6.ko): Invalid module format
>
>This is pretty nasty.  The problem isn't a bad relocation, its that the
>relocation target is too far away (i.e. ipv6 is too big).

I've seen this quite often in userspace on hppa-linux, lately while building
TAO (http://www.cs.wustl.edu/~schmidt/TAO.html).

>We get around this in the kernel by using -ffunction-sections so that
>the real linker can insert stubs between functions.  However,
>calculating how to do that in the in-kernel module loader would not be a
>whole lot of fun.

Wouldn't there a way to fix that somewhere else in the toolchain? Or is
there a parisc machine code limitation in the way? I would really get rid of
-ffunction-sections as it seems to generate slower executables. Could
you please explain what needs to be done?

TIA,
Uli
-- 
Dipl. Inf. Ulrich Teichert|e-mail: Ulrich.Teichert@gmx.de
Stormweg 24               |listening to: Suicide Drive (The Deep Eynde)
24539 Neumuenster, Germany|Public Pervert (Interpol) Clé De Contact (Metal Urbain)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[Randolph Chung]

> I've seen this quite often in userspace on hppa-linux, lately while building
> TAO (http://www.cs.wustl.edu/~schmidt/TAO.html).

what error do you see? the one that recommends rebuilding with
-ffunction-sections?

in your application, do you first build some .o's, link them together
into a bigger .o, and then use those to create your final application
image? (otherwise i don't think this problem should occur; if it does
there's a bug somewhere that we should fix).

> Wouldn't there a way to fix that somewhere else in the toolchain? Or is
> there a parisc machine code limitation in the way? I would really get rid of
> -ffunction-sections as it seems to generate slower executables. Could
> you please explain what needs to be done?

we have (or, rather, Dave has) been fixing this in various places. This
problem happens much less frequently now than it used to. Fixes may be
required either in gcc or in the linker to detect when a branch target
is too far away and needs to be converted to a long branch.

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[Ulrich Teichert]

Hi,

>> I've seen this quite often in userspace on hppa-linux, lately while building
>> TAO (http://www.cs.wustl.edu/~schmidt/TAO.html).
>
>what error do you see? the one that recommends rebuilding with
>-ffunction-sections?

Yes, exactly. While linking a huge C++ executable against dynamic
C++ libraries build with -fPIC.

>in your application, do you first build some .o's, link them together
>into a bigger .o, and then use those to create your final application
>image? (otherwise i don't think this problem should occur; if it does
>there's a bug somewhere that we should fix).
[TAO isn't really *my* application, it's more Douglas C. Schmidt's one,
I'm just a happy user]

Well, the linking of libraries as it's done in TAO is IMHO pretty much
standard:

g++ -D_REENTRANT -DACE_HAS_AIO_CALLS -D_GNU_SOURCE   -I/opt/ACE_wrappers -DACE_HAS_EXCEPTIONS -D__ACE_INLINE__ -DACE_BUILD_DLL -DACE_OS_BUILD_DLL -shared -Wl,-h -Wl,libACE.so.5.4.0 -o libACE.so.5.4.0 .shobj/ARGV.o
...
[countless list of objects]
...
.shobj/NT_Service.o .shobj/UUID.o -Wl,-E -L/opt/ACE_wrappers/ace -L./  -ldl -lpthread -lrt
rm -f libACE.so
ln -s libACE.so.5.4.0 libACE.so
chmod a+rx libACE.so.5.4.0

and all the other libs are created more or less the same way.

>> Wouldn't there a way to fix that somewhere else in the toolchain? Or is
>> there a parisc machine code limitation in the way? I would really get rid of
>> -ffunction-sections as it seems to generate slower executables. Could
>> you please explain what needs to be done?
>
>we have (or, rather, Dave has) been fixing this in various places. This
>problem happens much less frequently now than it used to. Fixes may be
>required either in gcc or in the linker to detect when a branch target
>is too far away and needs to be converted to a long branch.

Ah, then most likely, my toolchain is too old. I'm using debian woody right
now:

GNU ld version 2.12.90.0.1 20020307 Debian/GNU Linux
GNU assembler version 2.12.90.0.1 (hppa-linux) using BFD version 2.12.90.0.1 20020307 Debian/GNU Linux
Reading specs from /usr/lib/gcc-lib/hppa-linux/3.0.4/specs
Configured with: ../src/configure -v --enable-languages=c,c++,f77,proto,objc --prefix=/usr --infodir=/share/info --mandir=/share/man --enable-shared --with-gnu-as --with-gnu-ld --with-system-zlib --enable-long-long --enable-nls --without-included-gettext --disable-checking --enable-threads=posix --with-cpp-install-dir=bin hppa-linux
Thread model: posix
gcc version 3.0.4

What ld/gas/gcc versions would you recommend? Do I need extra patches?

TIA,
Uli
-- 
Dipl. Inf. Ulrich Teichert|e-mail: Ulrich.Teichert@gmx.de
Stormweg 24               |listening to: Suicide Drive (The Deep Eynde)
24539 Neumuenster, Germany|Public Pervert (Interpol) Clé De Contact (Metal Urbain)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[John David Anglin]

> Ah, then most likely, my toolchain is too old. I'm using debian woody =

Yes.

> What ld/gas/gcc versions would you recommend? Do I need extra patches?

Binutils 2.15, gcc 3.3.4.  No.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Userland relocation problems resolved (was: kernel module relocation bug)

[Ulrich Teichert]

Hi,

[del]
>Binutils 2.15, gcc 3.3.4.  No.

Right. Now it builds OK and passes most of the tests. The build finished
about two hours ago (on a B1000 - I don't have enough RAM).
I too have the problem of hanging 'expect' processes - now that I have a
new compiler, I should go for a newer kernel as well, I suppose ;-)

Thanks to all,
Uli
-- 
Dipl. Inf. Ulrich Teichert|e-mail: Ulrich.Teichert@gmx.de
Stormweg 24               |listening to: Suicide Drive (The Deep Eynde)
24539 Neumuenster, Germany|Public Pervert (Interpol) Clé De Contact (Metal Urbain)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Userland relocation problems resolved (was: kernel module relocat

[John David Anglin]

> I too have the problem of hanging 'expect' processes - now that I have =
> a
> new compiler, I should go for a newer kernel as well, I suppose ;-)

dave@hiauly6:~$ runtest --version
WARNING: Couldn't find the global config file.
Expect version is       5.42.1
Tcl version is          8.3
Framework version is    1.4.4

This is the most reliable configuration that I have found.  Tcl 8.4
is substantially worse.  Expect SEGVs on recent 64-bit SMP kernels in
the GCC testsuite.  I don't see the problem on 32-bit non-SMP kernels.
Sometimes java processes don't terminate although the testsuite
completes.  There is an issue with dejagnu asking tcl to trap SEGVs.
This causes tcl to try to restart expect after a SEGV.  The behavior
of signal is undefined in this situation and may cause the SEGV to
be repeated in an infinite loop.  You can fix this by editing dejagnu.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[John David Anglin]

> to fix this we will probably need to introduce long branch stubs into 
> into the kernel module loader. anybody want to give this a shot?

Another option which is probably less work is to build your kernel
and modules with -mpa-risc-2-0.  This should give you 22-bit branches.

There is also -mlong-calls.  Not too well tested but in theory it
should always avoid the above problem.  It's much more expensive
than -mpa-risc-2-0.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[Randolph Chung]

> Another option which is probably less work is to build your kernel
> and modules with -mpa-risc-2-0.  This should give you 22-bit branches.

well, except these 32-bit kernel/modules need to work on pa11 systems
too....

> There is also -mlong-calls.  Not too well tested but in theory it
> should always avoid the above problem.  It's much more expensive
> than -mpa-risc-2-0.

mmm.. maybe this is an option for pa11 systems.

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[Randolph Chung]

In reference to a message from John David Anglin, dated Nov 29:
> > to fix this we will probably need to introduce long branch stubs into 
> > into the kernel module loader. anybody want to give this a shot?
> 
> Another option which is probably less work is to build your kernel
> and modules with -mpa-risc-2-0.  This should give you 22-bit branches.

btw, Dave, pls correct me if i'm wrong:
is it correct that the problem only occurs because we are producing a .o
from multiple .o's? inside a single .o gcc should ensure that all
branches can reach the beginning and end of that .o using a 17-bit
relative branches, right?

randolph
-- 
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[John David Anglin]

> btw, Dave, pls correct me if i'm wrong:
> is it correct that the problem only occurs because we are producing a .o
> from multiple .o's? inside a single .o gcc should ensure that all
> branches can reach the beginning and end of that .o using a 17-bit
> relative branches, right?

If you don't use -ffunction-sections, gcc ensure that branches can reach
the beginning of an individual .o.  All bets are off if you use relinking
to combine .o's.

With -ffunction-sections, gcc guarantees that calls can reach the
beginning of a function plus some margin for the stub table.  The linker
inserts stubs on a per function basis.  This should work better with
relinking as the functions remain in their own sections.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[James Bottomley]

On Mon, 2004-11-29 at 18:08, John David Anglin wrote:
> With -ffunction-sections, gcc guarantees that calls can reach the
> beginning of a function plus some margin for the stub table.  The linker
> inserts stubs on a per function basis.  This should work better with
> relinking as the functions remain in their own sections.

So what we really need is the equivalent of -ffunction-sections for the
linker where, when we combine a bunch of .o's it sees if the branches
are getting too far a way and drops a stub in between the function
sections (so that the in-kernel linker doesn't have to bother about
doing this).

James


_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[John David Anglin]

> On Mon, 2004-11-29 at 18:08, John David Anglin wrote:
> > With -ffunction-sections, gcc guarantees that calls can reach the
> > beginning of a function plus some margin for the stub table.  The linker
> > inserts stubs on a per function basis.  This should work better with
> > relinking as the functions remain in their own sections.
> 
> So what we really need is the equivalent of -ffunction-sections for the
> linker where, when we combine a bunch of .o's it sees if the branches
> are getting too far a way and drops a stub in between the function
> sections (so that the in-kernel linker doesn't have to bother about
> doing this).

Then, in the final or in-kernel link, the long branch stubs get
fixed.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] kernel module relocation bug

[James Bottomley]

On Mon, 2004-11-29 at 18:23, John David Anglin wrote:
> > On Mon, 2004-11-29 at 18:08, John David Anglin wrote:
> > > With -ffunction-sections, gcc guarantees that calls can reach the
> > > beginning of a function plus some margin for the stub table.  The linker
> > > inserts stubs on a per function basis.  This should work better with
> > > relinking as the functions remain in their own sections.
> > 
> > So what we really need is the equivalent of -ffunction-sections for the
> > linker where, when we combine a bunch of .o's it sees if the branches
> > are getting too far a way and drops a stub in between the function
> > sections (so that the in-kernel linker doesn't have to bother about
> > doing this).
> 
> Then, in the final or in-kernel link, the long branch stubs get
> fixed.

That's the general idea.  Obviously any two pass linker (like the one in
binutils) can place the stubs optimally in between the sections (as long
as the code was compiled with -ffunction-sections) as part of the final
link, but the in-kernel linker is small and simple and I'd rather not
have to introduce it to the concept of multiple passes, so if there were
a flag to get binutils to do the hard work for us it would be much
appreciated.

James


_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux