Re: [parisc-linux] userspace function pointers in the kernel

Re: [parisc-linux] userspace function pointers in the kernel

[Cary Coutant]

>It looks like the IA-64 runtime achieves this by generating official
>procedure descriptors for every defined function in an object and
>placing them in the .opd section, then using these descriptors
>whenever the address of a function is taken.  Presumably external
>functions are compared using their .IA_64.pltoff entries?  I'm not
>clear on how that works and don't have an IA-64 handy to experiment.

I can't say for sure how Linux/IA-64 uses the .opd section, but the 
runtime and the psABI recommend that the "official" function descriptors 
be created by the dynamic loader as needed in response to dynamic FPTR 
relocations. A function pointer is always loaded from the GOT or from a 
statically-initialized variable, either of which would have been tagged 
with a dynamic FPTR relocation. The dynamic loader will allocate one and 
only one OFD for each function whose address is taken, and resolve each 
function pointer to the address of that OFD. Function pointers can then 
be compared as simple 32- or 64-bit quantities, without worrying about 
the contents of the function descriptor that they point to.

One way of avoiding the dynamic allocation by the dynamic loader is for 
the linker to allocate OFDs statically for each exported function, and 
for each hidden or static function whose address is taken. It looks like 
this may be what Linux is doing on IA-64. The disadvantage is that you 
end up with far more OFDs than you'll ever need, since few functions ever 
really have their addresses taken.

The .pltoff section is for local copies of OFDs that are used by inlined 
import stubs. When the compiler decides that a function is unlikely to 
resolve within the load module, it generates an inlined external call 
sequence, rather than rely on the linker to resolve a direct call to an 
import stub. This inlined sequence needs a local copy of the function 
descriptor to avoid an extra indirection. Since these function 
descriptors never serve as an official function descriptor, it doesn't 
matter that they don't have the same address as an OFD.

Note that the same argument applies to C++ vtables -- we can put copies 
of the function descriptors there, too, since we don't need the pointers 
to vtable entries to be unique.

As an aside, the problem we had on HP-UX/PA was in statically allocating 
the official function descriptors -- the linker allocated OFD 
"candidates" whenever it saw the address of a function get taken. This 
produced multiple candidates for the OFD, and it was possible that none 
of those candidates were actually in the load module where the function 
was defined. Thus, the loader had to choose one arbitrarily, and 
sometimes could be tricked into choosing different ones at different 
times. Hence the problem.

-cary

Re: [parisc-linux] userspace function pointers in the kernel

[David Huggins-Daines]

Cary Coutant <cary@cup.hp.com> writes:

> I can't say for sure how Linux/IA-64 uses the .opd section, but the 
> runtime and the psABI recommend that the "official" function descriptors 
> be created by the dynamic loader as needed in response to dynamic FPTR 
> relocations.

Ah, okay, that makes sense.  This is what Linux/IA-64 does - dynamic
FPTR relocations result in OFDs being created dynamically (always, for
both local and global symbols).  And yes, the code in GNU libc does
ensure that they are mapped one-to-one with symbols in each process.

As mentioned earlier we have been trying to decide whether to do this
on PA-RISC as well.  Originally I threw it in so that (you guessed it)
comparisons of function pointers with NULL would work properly :-)

But as Alan says, it looks like we need it for other reasons as well.

> One way of avoiding the dynamic allocation by the dynamic loader is for 
> the linker to allocate OFDs statically for each exported function, and 
> for each hidden or static function whose address is taken.

Hm.  I'm not nearly as familiar with the linker code as I am with the
dynamic linker code, so I'm not sure what it does.  But, as noted
above, GNU libc does dynamic allocation, so it's probably not relevant.

-- 
dhd@linuxcare.com, http://www.linuxcare.com/
Linuxcare. Support for the revolution.

Re: [parisc-linux] userspace function pointers in the kernel

[Alan Modra]

On 12 Sep 2000, David Huggins-Daines wrote:

> Cary Coutant <cary@cup.hp.com> writes:
> 
> > One way of avoiding the dynamic allocation by the dynamic loader is for 
> > the linker to allocate OFDs statically for each exported function, and 
> > for each hidden or static function whose address is taken.
> 
> Hm.  I'm not nearly as familiar with the linker code as I am with the
> dynamic linker code, so I'm not sure what it does.  But, as noted
> above, GNU libc does dynamic allocation, so it's probably not relevant.

Well, I'm more familiar with the linker than the dynamic linker :)

The model I'm thinking of for elf32-hppa is along these lines:

o  The linker creates a plt entry for all plabel relocs.  plt entries for
   elf32-hppa are a function address, linkage table pointer pair, so
   there's no need for the dynamic linker to allocate fptrs.
o  A dynamic plabel reloc will have the function symbol, and an addend
   into the plt.  This is a rather unusual reloc because the function
   symbol value is ignored when calculating the final value.
o  The dynamic linker builds a list or hash table of function symbols
   versus plt offsets, and adjusts plabels so that only one plt entry is
   ever used per function.

Alan Modra
-- 
Linuxcare.  Support for the Revolution.

Function descriptors, etc. (was Re: [parisc-linux] userspace function pointers in the kernel)

[David Huggins-Daines]

[-- Attachment #1: Type: text/plain, Size: 4770 bytes --]

Alan Modra <alan@linuxcare.com.au> writes:

> The model I'm thinking of for elf32-hppa is along these lines:
> 
> o  The linker creates a plt entry for all plabel relocs.  plt entries for
>    elf32-hppa are a function address, linkage table pointer pair, so
>    there's no need for the dynamic linker to allocate fptrs.
> o  A dynamic plabel reloc will have the function symbol, and an addend
>    into the plt.  This is a rather unusual reloc because the function
>    symbol value is ignored when calculating the final value.
> o  The dynamic linker builds a list or hash table of function symbols
>    versus plt offsets, and adjusts plabels so that only one plt entry is
>    ever used per function.

Reasons why the IA-64 scheme is better:

        a) In the scheme above, we cannot index the list/table by the
        code address of the function in the PLT slot, because the PLT
        will usually not have been relocated at the point when PLABEL
        relocations are being processed.  Thus we have to:

                1) Relocate PLT slots as we encounter them in PLABEL
                   relocations, or
                2) Store a copy of the code address in the table apart
                   from the PLT slot, or
                3) Index the table by the address of the Elf32_Sym
                   structure, by the symbol name (I think this is
                   fraught with peril), or by something else.  This
                   also requires another field in the table apart from
                   the PLT slot.

        b) The IA-64 code uses a one-to-one mapping of function code
        addresses to function descriptors, not symbols to function
        descriptors, and indexes the list/table by the code address.
        I'm not sure whether this is important or not, but something
        tells me it is.

        c) We will still have to dynamically allocate function
        descriptors for symbols obtained with dlopen() and dlsym().
        First of all, we won't have a special PLABEL relocation, so we
        would have to search the PLT of the loaded object "manually".
        Also, think about the case where:

                - libfoo.so defines foo() strongly
                - libfoo2.so defines foo() weakly
                - libfoo2.so defines bar() which calls foo()
                - our program links against libfoo.so and libfoo2.so
                - our program calls foo()
                - our program calls bar()
                - our program dlopen()s libfoo2.so
                - our program calls dlsym(libfoo2, "foo")
                - oops!  the PLT slot for foo() in libfoo2.so points
                  to the strong definition in libfoo.so, so it's
                  useless as a function descriptor.
                - I've attached a tarball of this testcase.  Run it on
                  i386 and IA-64 and see what it does :-)

        d) If we have to dynamically allocate function descriptors as
        well as uniquify relocations to PLT slots, then we basically
        have to duplicate the code in dl-fptr.c, and we lose.
        
Can we please use the IA-64 scheme or something very close to it?
(i.e. closer than what you've proposed, which, in attempting to
implement it in ld.so, does not appear to be close enough) It is
already implemented and debugged, I had it successfully working last
week modulo problems with initializers and finalizers, and I don't see
there being a significant gain in performance or memory usage with the
scheme outlined above.

What I suggest is the following:

* The linker still creates a PLT entry for all PLABEL relocs.  This is
  undoubtedly useful for local symbols, and allows us to take
  advantage of lazy linking for them (potentially a big win in libc.so
  which uses lots and lots of vtables).

* We don't have to generate dynamic function descriptors for PLABEL
  relocations to local symbols because we know that their PLT entries
  are and will remain unique to their code addresses.  Again, a win.

* Dynamic PLABEL relocs will either be *ABS* or .plt relative and
  pointing to the PLT slot for local symbols, or contain the function
  symbol and no addend for global symbols.  For the latter we will
  generate unique function descriptors dynamically using the same code
  as on IA-64.  Yes, we lose 12 bytes of memory per global symbol this
  way.  I don't think that's a big deal given that it allows us to use
  the same code path for these symbols and for dlsym().

* dlsym() will always generate a unique function descriptor dynamically.

I've implemented this and checked it in (it kludges around the addends
on global PLABEL relocations, that is easy to fix :-).  It passes my
other tests but the one below causes the linker to segfault.


[-- Attachment #2: Interaction of dlsym() and weak dynamic symbols --]
[-- Type: application/x-gtar, Size: 1134 bytes --]

[-- Attachment #3: Type: text/plain, Size: 89 bytes --]


-- 
dhd@linuxcare.com, http://www.linuxcare.com/
Linuxcare. Support for the revolution.

Re: Function descriptors, etc. (was Re: [parisc-linux] userspace function pointers in the kernel)

[Alan Modra]

On 13 Sep 2000, David Huggins-Daines wrote:

> Reasons why the IA-64 scheme is better:
>[snip]

Hrmm, you've clearly had more time than I thinking about dynamic linker
issues.  :-)  My scheme of having the linker allocate space for the ld.so
structures necessary to generate unique function pointers can still
work;  It's just that plabels need a full struct hppa_fptr in addition
to our normal plt slot, rather than the measely single word I gave you.
This could have been accomplished simply by changing PLABEL_PLT_ENTRY_SIZE
to 20, and then ld.so wouldn't need those extra mmaps.

However, I guess we should go with what you have David - the time saved by
having the linker allocate space isn't huge, and in fact we might not even
need to mmap since ld.so has 256 fptr slots pre-allocated.

I just ran the following, and found libc.so only needs 63 global fptrs.

readelf -r libc.so | grep PLABEL \
| sed -e 's/.*R_PARISC/R_PARISC/' -e '/PLABEL      /d' -e 's/ *+.*//' \
| sort | uniq

Alan
-- 
Linuxcare.  Support for the Revolution.