Re: [Linux-ia64] [RFC] /proc/ksyms change for IA64 (fwd)

Re: [Linux-ia64] [RFC] /proc/ksyms change for IA64 (fwd)

[Jack Steiner]

> with old modutils, modules will oops.  I don't see this as a problem,
> the IA64 population is fairly small, in any case gcc cross compile for
> IA64 has problems right now.
> 

The proposal looks fine. I have another question.

What problems exist with cross compiling.

We are still using the cross-compiler for all of our building & testing.
We use:
	gcc version 2.96-ia64-000717 snap 001117 (plus some patches that Ralf added last Dec).


I know that at some point we need to convert to native builds but right now we
dont have sufficient bigsur/lion boxes to do that.

We have not seen any problems with the compiler we are using - at least we 
have not attributed a problem to the compiler.

Should we upgrade to gcc3.0 yet???


Any info on this subject would be appreciated......

-- 
Thanks

Jack Steiner    (651-683-5302)   (vnet 233-5302)      steiner@sgi.com

Re: [RFC] /proc/ksyms change for IA64

[Keith Owens]

On Thu, 2 Aug 2001 07:42:45 -0500 (CDT), 
Jack Steiner <steiner@sgi.com> wrote:
>What problems exist with cross compiling.
>We are still using the cross-compiler for all of our building & testing.
>We use:
>	gcc version 2.96-ia64-000717 snap 001117 (plus some patches that Ralf added last Dec).
>
>I know that at some point we need to convert to native builds but right now we
>dont have sufficient bigsur/lion boxes to do that.
>
>We have not seen any problems with the compiler we are using - at least we 
>have not attributed a problem to the compiler.

The changes to the copy_user pipeline code in the latest 2.4.7-ia64
patch break snap 001117 in some circumstances.  There are workarounds
but they require changes to code that is not IA64 related, not
satisfactory.

>Should we upgrade to gcc3.0 yet???

I have not succeeded building a cross compiler ix86 to ia64 for gcc
3.0, using a current CVS tree.  The flow insn code breaks on some type
conversions in cross compile mode, even when using gcc 3.0 as the base
compiler.  Debugging is still in progress.

Re: [Linux-ia64] [RFC] /proc/ksyms change for IA64 (fwd)

[Jes Sorensen]

>>>>> "Jack" == Jack Steiner <steiner@sgi.com> writes:

Jack> I know that at some point we need to convert to native builds
Jack> but right now we dont have sufficient bigsur/lion boxes to do
Jack> that.

Jack> We have not seen any problems with the compiler we are using -
Jack> at least we have not attributed a problem to the compiler.

Jack> Should we upgrade to gcc3.0 yet???

Probably not yet. I would stick to gcc-2.96 based on what you can find
in Red Hat's (or one of the other distributions') ia64 SRPMS.

Jes

[RFC] /proc/ksyms change for IA64

[Keith Owens]

The IA64 use of descriptors for function pointers has bitten ksymoops.
For those not familiar with IA64, &func points to a descriptor
containing { &code, &data_context }.  System.map contains the address
of the code, /proc/ksyms contains the address of the descriptor.
insmod needs the descriptor address, ksymoops and debuggers need the
code address, /proc/ksyms needs to contain both addresses, with one of
them prefixed by a special character.

EXPORT_SYMBOL() cannot distinguish between &function and &data.
Telling everybody that they must type EXPORT_SYMBOL_FUNCTION or
EXPORT_SYMBOL_DATA is not an option, the second symbol has to be
automatically added after linking vmlinux and modules.  I want to fix
this problem in user space, with no changes to to kernel code.  My
proposal for kernel 2.5 is :-

* In System.map and /proc/ksyms, label foo addresses the code, not the
  descriptor.  Label &foo addresses the function descriptor.

* foo has symbol versions, &foo does not.

* In kernel build, all objects that export symbols are post processed
  immediately after they are compiled.  On most architectures the post
  process is a no-op.  On IA64 it runs a program from modutils that
  identifies exported function pointers and modifies the ksymtab and
  kstrtab, see below.

* At run time, insmod does its normal address fixup processing first,
  including checking for symbol versions.  If the existing code
  succeeds without unresolved symbols, insmod modifies the address
  fixup so a reference to foo is resolved to &foo, iff both foo and
  &foo are in /proc/ksyms.

This has the benefit that all the changes are in modutils plus a small
change to the kernel build, no other user space tools are affected.
Most user space programs expect foo to address the code, not the
descriptor, AFAIK it is only insmod that needs the descriptor.  This
change will also affect any code that tries to front end functions from
modules but IMNSHO code should not be doing that.  If this breaks
binary only modules - tough.

One problem that always has to be addressed with modutils is version
skew between user space and the kernel.  With this approach, if you
build a new kernel with old modutils then the post processing program
will not be available so ksyms has the old format and the old insmod
behaviour applies, ksymoops will get errors but the kernel will still
run.  With a new modutils but an old kernel you get the old behaviour.
The only way you can get problems with version skew is to compile a
kernel on one machine with new modutils and install on another machine
with old modutils, modules will oops.  I don't see this as a problem,
the IA64 population is fairly small, in any case gcc cross compile for
IA64 has problems right now.

The new modutils supplied program (/usr/bin/modules_post) will look
through the __ksymtab section for an object.  For each exported symbol,
look at the data it references.  If that data contains two relocatable
addresses, one pointing to a text symbol, the other to a data symbol
and the text symbol has the same name as the exported symbol then this
is a descriptor.  Change the exported symbol name to &foo and add an
exported symbol foo that points to the code.

Note: This is modutils 2.5 stuff.  modules_post needs to use BFD which
      current modutils does not use, this adds a new requirement when
      compiling modutils.  Using BFD is the only way I can handle all
      the relocation types, especially in cross compile mode.  It may
      or may not get backported to modutils 2.4, probably not.
      ksymoops on IA64 2.4 will just have to live with lots of
      warnings.

Re: [RFC] /proc/ksyms change for IA64

[Rusty Russell]

In message <22165.996722560@kao2.melbourne.sgi.com> you write:
> The IA64 use of descriptors for function pointers has bitten ksymoops.
> For those not familiar with IA64, &func points to a descriptor
> containing { &code, &data_context }.  System.map contains the address
> of the code, /proc/ksyms contains the address of the descriptor.
> insmod needs the descriptor address, ksymoops and debuggers need the
> code address, /proc/ksyms needs to contain both addresses, with one of
> them prefixed by a special character.

Eewwww....

	How about just adding /proc/ksyms-ia64 with the code pointers
which contains the ia64 code addresses required by ksymoops and
debuggers.  These are, after all, less vital than insmod.

Cheers,
Rusty.
--
Premature optmztion is rt of all evl. --DK

Re: [RFC] /proc/ksyms change for IA64

[Keith Owens]

On Sun, 05 Aug 2001 15:44:45 +1000, 
Rusty Russell <rusty@rustcorp.com.au> wrote:
>In message <22165.996722560@kao2.melbourne.sgi.com> you write:
>> The IA64 use of descriptors for function pointers has bitten ksymoops.
>> For those not familiar with IA64, &func points to a descriptor
>> containing { &code, &data_context }.  System.map contains the address
>> of the code, /proc/ksyms contains the address of the descriptor.
>> insmod needs the descriptor address, ksymoops and debuggers need the
>> code address, /proc/ksyms needs to contain both addresses, with one of
>> them prefixed by a special character.
>
>Eewwww....
>
>	How about just adding /proc/ksyms-ia64 with the code pointers
>which contains the ia64 code addresses required by ksymoops and
>debuggers.  These are, after all, less vital than insmod.

That requires changes to every kernel debugger, oops decoder etc.  I
don't control all of Linux debugging yet ;).  It is also more work
because it requires kernel changes as well as lots of user space.

BTW, do you know that you are sending from 144.137.82.79 which has no reverse
DNS?  Oh wait, it's Telstra :(

Welcome to Telstra country, on any day you can hear the customers being
ripped off.

Re: [RFC] /proc/ksyms change for IA64

[Rusty Russell]

In message <13470.996995793@ocs3.ocs-net> you write:
> >Eewwww....
> >
> >	How about just adding /proc/ksyms-ia64 with the code pointers
> >which contains the ia64 code addresses required by ksymoops and
> >debuggers.  These are, after all, less vital than insmod.
> 
> That requires changes to every kernel debugger, oops decoder etc.  I
> don't control all of Linux debugging yet ;).  It is also more work
> because it requires kernel changes as well as lots of user space.

For ia64 only.  IMHO, that's a better line to draw.

Rusty.
--
Premature optmztion is rt of all evl. --DK

Re: [RFC] /proc/ksyms change for IA64

[Keith Owens]

On Mon, 06 Aug 2001 00:02:51 +1000, 
Rusty Russell <rusty@rustcorp.com.au> wrote:
>In message <13470.996995793@ocs3.ocs-net> you write:
>> >Eewwww....
>> >
>> >	How about just adding /proc/ksyms-ia64 with the code pointers
>> >which contains the ia64 code addresses required by ksymoops and
>> >debuggers.  These are, after all, less vital than insmod.
>> 
>> That requires changes to every kernel debugger, oops decoder etc.  I
>> don't control all of Linux debugging yet ;).  It is also more work
>> because it requires kernel changes as well as lots of user space.
>
>For ia64 only.  IMHO, that's a better line to draw.

My approach preserves the notion that a ksyms entry points to the data
or code, changing the function descriptor to a different name in ksyms.
Your approach breaks that notion and introduces a special case for ia64
ksyms.  My approach only needs changes to modutils.  Your approach
requires changes to the kernel and to every utility that understands
ksyms.  Guess which approach I am doing?

Re: [RFC] /proc/ksyms change for IA64

[Richard Henderson]

On Thu, Aug 02, 2001 at 01:22:40PM +1000, Keith Owens wrote:
> Using BFD is the only way I can handle all
> the relocation types, especially in cross compile mode.

What the hell?  You've got everything you need right there in 
the obj subdirectory.  Please don't bring libbfd back to life
in modutils.


r~

Re: [RFC] /proc/ksyms change for IA64

[Keith Owens]

On Sat, 18 Aug 2001 18:27:56 -0700, 
Richard Henderson <rth@twiddle.net> wrote:
>On Thu, Aug 02, 2001 at 01:22:40PM +1000, Keith Owens wrote:
>> Using BFD is the only way I can handle all
>> the relocation types, especially in cross compile mode.
>
>What the hell?  You've got everything you need right there in 
>the obj subdirectory.  Please don't bring libbfd back to life
>in modutils.

Cross compile mode.  Nothing in modutils works unless it is running on
the machine it was compiled for.  As more modutil functions get pushed
back into kbuild time, this is getting to be a problem.  I could do all
my own code for endianess and size differences between host and target,
but why bother when bfd already does it for me?

Re: [RFC] /proc/ksyms change for IA64

[Richard Henderson]

On Tue, Aug 21, 2001 at 04:53:20PM +1000, Keith Owens wrote:
> ... but why bother when bfd already does it for me?

How about bfd is a bloated monstrosity.  Right now libobj is
just over 10k; you'll be looking at a _minimum_ of 150k to
pull in only the linking code from libbfd.a.

How about bfd is an unmaintainable nightmare?  If you've not
spent the last couple of years working on bfd, save your
sanity and don't start.

It would take only a day or three to fix up libobj to cross
compile.  If you don't want to do it, let someone else.


r~