[parisc-linux] Use of the EI_OSABI field

[parisc-linux] Use of the EI_OSABI field

[John Marvin]

> Another glibc issue (which is why I'm sending this back to the list) is
> that there has been quite some discussion on the binutils list over the
> use of the EI_OSABI field.  The conclusion being that it isn't really
> correct to use this field to discern the intendend execution platform.
> It's purpose is really to tell ELF tools that a file contains fields and
> values that need to be interpreted in a non-standard way.  Since both
> elf32-hppa and elf64-hppa follow the standards, I'm inclined to think that
> the gnu tools should set EI_OSABI to zero for both HPUX and Linux targets.

I don't read the binutils mailing list, so I checked the archive.  In my
opinion I didn't see a discussion, I saw H.J.  Lu repeatedly asserting
this "fact".  He may or may not be correct about original intention, but
the implementation so far has been that EI_OSABI is used to tell what the
target platform is.  That is what FreeBSD is doing, that is what HP-UX is
doing, and that is what the IA-64 Unix System V Application Binary
Interface specifies:

  http://developer.intel.com/design/IA-64/Downloads/24537002.pdf

Note that the second paragraph in section 1.1 of that specification
includes the following:

    This document is the result of consensus among operating system
    vendors intending to provide UNIX and UNIX workalike operating
    systems on the IA-64 architecture. The vendors participating in
    this effort include Intel, Sun Microsystems, SCO, IBM, SGI,
    Cygnus Solutions, VA Linux Systems, HP, and Compaq.

So, If the specification is wrong according to H.J. Lu, why did both
Cygnus and VA Linux Systems not object to this:

    Section 4.1.1.4 Operating System Identification

    The e_ident[EI_OSABI] value identifies the operating system and
    ABI to which the object is targeted, as listed in Table 4-1.

Table 4-1 lists the various ELFOSABI_* fields, e.g. ELFOSABI_HPUX,
ELFOSABI_NETBSD, ELFOSABI_LINUX, etc.

Note also, that the current mechanism for us to be able to differentiate
elf executables in the Linux kernel is the machine dependent
elf_check_arch() macro, whose only argument is a pointer to a
elf32_hdr or elf64_hdr. I think it would be ugly to try to get the
.note.ABI-tag section first for every exec of a new binary in order
to determine what the target ABI is.

In my opinion, unless H.J. Lu can get the IA-64 ABI changed, it is too
late to assert his view of what that field was supposed to be. Please
leave the EI_OSABI field set to ELFOSABI_LINUX until a better consensus
on this issue is reached.

John Marvin
jsm@fc.hp.com

Re: [parisc-linux] Use of the EI_OSABI field

[Alan Modra]

cc to binutils because John makes some salient points.

On Mon, 20 Nov 2000, John Marvin wrote:

> > Another glibc issue (which is why I'm sending this back to the list) is
> > that there has been quite some discussion on the binutils list over the
> > use of the EI_OSABI field.  The conclusion being that it isn't really
> > correct to use this field to discern the intendend execution platform.
> > It's purpose is really to tell ELF tools that a file contains fields and
> > values that need to be interpreted in a non-standard way.  Since both
> > elf32-hppa and elf64-hppa follow the standards, I'm inclined to think that
> > the gnu tools should set EI_OSABI to zero for both HPUX and Linux targets.
> 
> I don't read the binutils mailing list, so I checked the archive.  In my
> opinion I didn't see a discussion, I saw H.J.  Lu repeatedly asserting
> this "fact".  He may or may not be correct about original intention, but
> the implementation so far has been that EI_OSABI is used to tell what the
> target platform is.  That is what FreeBSD is doing, that is what HP-UX is
> doing, and that is what the IA-64 Unix System V Application Binary
> Interface specifies:
> 
>   http://developer.intel.com/design/IA-64/Downloads/24537002.pdf
> 
> Note that the second paragraph in section 1.1 of that specification
> includes the following:
> 
>     This document is the result of consensus among operating system
>     vendors intending to provide UNIX and UNIX workalike operating
>     systems on the IA-64 architecture. The vendors participating in
>     this effort include Intel, Sun Microsystems, SCO, IBM, SGI,
>     Cygnus Solutions, VA Linux Systems, HP, and Compaq.
> 
> So, If the specification is wrong according to H.J. Lu, why did both
> Cygnus and VA Linux Systems not object to this:
> 
>     Section 4.1.1.4 Operating System Identification
> 
>     The e_ident[EI_OSABI] value identifies the operating system and
>     ABI to which the object is targeted, as listed in Table 4-1.
> 
> Table 4-1 lists the various ELFOSABI_* fields, e.g. ELFOSABI_HPUX,
> ELFOSABI_NETBSD, ELFOSABI_LINUX, etc.
> 
> Note also, that the current mechanism for us to be able to differentiate
> elf executables in the Linux kernel is the machine dependent
> elf_check_arch() macro, whose only argument is a pointer to a
> elf32_hdr or elf64_hdr. I think it would be ugly to try to get the
> .note.ABI-tag section first for every exec of a new binary in order
> to determine what the target ABI is.
> 
> In my opinion, unless H.J. Lu can get the IA-64 ABI changed, it is too
> late to assert his view of what that field was supposed to be. Please
> leave the EI_OSABI field set to ELFOSABI_LINUX until a better consensus
> on this issue is reached.
> 
> John Marvin
> jsm@fc.hp.com

I'm happy enough to leave things as they are in puffin CVS, but these
changes haven't been merged back to sourceware yet, partly because I had
some doubts myself as to whether setting EI_OSABI was correct.  H.J. Lu
wasn't the only one arguing that EI_OSABI should be left at zero;  Ulrich
Drepper also was quite vehement against changing sourceware FreeBSD
binutils.

BTW, it's not too hard to check .note.ABI-tag.  The linker arranges for a
PT_NOTE program header entry to point to it, and the section itself is
virtually guaranteed to be read in with the header as it's placed right
after the header along with .interp.

Alan Modra
-- 
Linuxcare.  Support for the Revolution.

Re: [parisc-linux] Use of the EI_OSABI field

[Ulrich Drepper]

Alan Modra <alan@linuxcare.com.au> writes:

> Ulrich Drepper also was quite vehement against changing sourceware
> FreeBSD binutils.

I've never said anything about any *BSD, why should I?  The *BSD
people wanted to change the Linux binutils.

Anyway, the ABI value is zero unless you implement ELF extensions.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

Re: [parisc-linux] Use of the EI_OSABI field

[Alan Modra]

On 21 Nov 2000, Ulrich Drepper wrote:

> Alan Modra <alan@linuxcare.com.au> writes:
> 
> > Ulrich Drepper also was quite vehement against changing sourceware
> > FreeBSD binutils.
> 
> I've never said anything about any *BSD, why should I?  The *BSD
> people wanted to change the Linux binutils.

Sorry, I stated that badly.

> Anyway, the ABI value is zero unless you implement ELF extensions.

Exactly what is an "ELF extension"?  Anything outside gABI or
"gABI + psABI"?  Handly the latter, as it seems to me that a processor
specific ABI can specify extensions.  There's also the awkward possibility
that a psABI may specify an extension that is later incorporated into a
new revision of the gABI (eg. hpux and DT_INIT_ARRAY)  Does that mean that
if a new revision of the gABI completely incorporates all previous
extensions, that EI_OSABI should become zero?

Yes, I'm arguing that "No ELF extensions => EI_OSABI == 0" is not
necessarily true, but I'm _not_ arguing that changing x86 Linux binutils
is wise.  Historical usage is important.  If we were to change x86 Linux
binutils to set EI_OSABI, then that can only be after a considerable
period of time to allow code such as glibc to accept a new branding.

Alan Modra
-- 
Linuxcare.  Support for the Revolution.

Re: [parisc-linux] Use of the EI_OSABI field

[Ulrich Drepper]

Alan Modra <alan@linuxcare.com.au> writes:

> Exactly what is an "ELF extension"?  Anything outside gABI or
> "gABI + psABI"?

Anything beyond the psABI that cannot be handled by the rules in the
gABI.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

Re: [parisc-linux] Use of the EI_OSABI field

[H . J . Lu]

On Wed, Nov 22, 2000 at 11:13:29AM +1100, Alan Modra wrote:
> 
> > Anyway, the ABI value is zero unless you implement ELF extensions.
> 
> Exactly what is an "ELF extension"?  Anything outside gABI or

ELF extension is something whose interpretation is not documented in
neither gABI nor psABI. HP's old ELF is a good example since it didn't
implement DT_INIT/DT_FINI, but added DT_INIT_ARRARY/DT_FINI_ARRAY. A
normal ELF tool will have a hard time to interpret those fields and
their values.


H.J.

Re: [parisc-linux] Use of the EI_OSABI field

[Alan Modra]

On Tue, 21 Nov 2000, H . J . Lu wrote:

> On Wed, Nov 22, 2000 at 11:13:29AM +1100, Alan Modra wrote:
> > 
> > > Anyway, the ABI value is zero unless you implement ELF extensions.
> > 
> > Exactly what is an "ELF extension"?  Anything outside gABI or
> 
> ELF extension is something whose interpretation is not documented in
> neither gABI nor psABI. HP's old ELF is a good example since it didn't
> implement DT_INIT/DT_FINI, but added DT_INIT_ARRARY/DT_FINI_ARRAY. A
> normal ELF tool will have a hard time to interpret those fields and
> their values.

Neither you nor Ulrich have responded to John's point that the IA64 psABI
states
  "The e_ident[EI_OSABI] value identifies the operating system and ABI to
   which the object is targeted"

Granted, this is only in a processor specific ABI, but it does flatly
contradict your assertions that the purpose of EI_OSABI is to flag the
presense of ELF extensions.

Alan Modra
-- 
Linuxcare.  Support for the Revolution.

Re: [parisc-linux] Use of the EI_OSABI field

[H . J . Lu]

On Wed, Nov 22, 2000 at 02:03:07PM +1100, Alan Modra wrote:
> On Tue, 21 Nov 2000, H . J . Lu wrote:
> 
> > On Wed, Nov 22, 2000 at 11:13:29AM +1100, Alan Modra wrote:
> > > 
> > > > Anyway, the ABI value is zero unless you implement ELF extensions.
> > > 
> > > Exactly what is an "ELF extension"?  Anything outside gABI or
> > 
> > ELF extension is something whose interpretation is not documented in
> > neither gABI nor psABI. HP's old ELF is a good example since it didn't
> > implement DT_INIT/DT_FINI, but added DT_INIT_ARRARY/DT_FINI_ARRAY. A
> > normal ELF tool will have a hard time to interpret those fields and
> > their values.
> 
> Neither you nor Ulrich have responded to John's point that the IA64 psABI
> states
>   "The e_ident[EI_OSABI] value identifies the operating system and ABI to
>    which the object is targeted"
> 
> Granted, this is only in a processor specific ABI, but it does flatly
> contradict your assertions that the purpose of EI_OSABI is to flag the
> presense of ELF extensions.
> 

When I was at the IA64 ABI meeting yesterday, we were looking at the
EI_OSABI issue. Everyone at the IA64 ABI meeting agreed that what
Ulrich and I had said was the correct understanding.

"The e_ident[EI_OSABI] value identifies the operating system and ABI
to which the object is targeted" just means if the e_ident[EI_OSABI]
value is not ELFOSABI_NONE, you have to look somewhere else in addition
to gABI and IA64 psABI to interpret the ELF fields/values specific to
that OS. Otherwise, gABI and IA64 psABI are sufficient.

-- 
H.J. Lu (hjl@valinux.com)

Re: [parisc-linux] Use of the EI_OSABI field

[Ulrich Drepper]

Alan Modra <alan@linuxcare.com.au> writes:

>   "The e_ident[EI_OSABI] value identifies the operating system and ABI to
>    which the object is targeted"
> 
> Granted, this is only in a processor specific ABI, but it does flatly
> contradict your assertions that the purpose of EI_OSABI is to flag the
> presense of ELF extensions.

The meaning of the field changed over time.  Or better said, some
people initially understood it differently.

-- 
---------------.                          ,-.   1325 Chesapeake Terrace
Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
Red Hat          `--' drepper at redhat.com   `------------------------

Re: [parisc-linux] Use of the EI_OSABI field

[John Marvin]

>
> BTW, it's not too hard to check .note.ABI-tag.  The linker arranges for a
> PT_NOTE program header entry to point to it, and the section itself is
> virtually guaranteed to be read in with the header as it's placed right
> after the header along with .interp.

I didn't say it was difficult, I said it was ugly. It means another
parisc only change to the machine independent file fs/binfmt_elf.c,
since the hook provided will not allow this check. Without a change,
binfmt_elf.c won't be smart enough to differentiate a binary produced
by Gnu binutils for HP-UX and a binary produced by Gnu binutils for
Linux, so it will claim both, and then blow up later, rather than
not claiming the HP-UX binary and allowing it to be claimed by
an arch dependent binary handler further down the list.

And binfmt_elf.c does NOT read the program headers in the same read, so
another read would have to be done (the data should be found in in cache
rather than going to disk for it). Since we now need both the program
headers and a section header to determine whether or not we should claim
the file AND binfmt_elf.c also wants to look at those headers after
the file is claimed, a small redesign is probably in order (rather than
re-reading the headers). I'm not sure whether or not Linus would buy
that.

So, I guess I'll pursue the interpreter field instead, since that is
what sparc is doing (i.e. they have their own sparc only code in
binfmt_elf.c). Since that will be an easier sell. I need to do more
research here. I suspect that statically linked binaries are not going
to allow this solution to work though.

John Marvin
jsm@fc.hp.com

Re: [parisc-linux] Use of the EI_OSABI field

[Cary Coutant]

As the original author of the proposal to add the EI_OSABI field to the 
ELF format, let me try to clarify the intent of this field. The 
authoritative definition of this field is found in SCO's gABI document, 
which is still the official specification for the ELF format (this is 
probably posted somewhere on SCO's web site). Here's what it says:

    Byte e_ident[EI_OSABI] identifies the operating system and
    ABI to which the object is targeted. Some fields in other ELF
    structures have flags and values that have operating system
    and/or ABI specific meanings; the interpretation of those
    fields is determined by the value of this byte. The value of
    this byte must be interpreted differently for each machine.
    That is, each value for the e_machine field determines a set
    of values for the EI_OSABI byte. Values are assigned by the
    ABI processor supplement for each machine. If the processor
    supplement does not specify a set of values, the value 0
    shall be used and indicates unspecified.

The first sentence is still a bit misleading, and is an artifact of the 
original proposal. Originally, the field was intended to identify the 
target ABI (hence the name of the field). As we started discussing a 
common Unix ABI for IA-64, however, it became clear that this field 
wouldn't serve that purpose, but it was still needed to identify the set 
of platform-specific ELF extensions that are used by the object file.

There are a number of fields in the ELF format for which ranges of values 
or a set of flag bits are reserved for vendor-specific use (e.g., 
SHT_LOOS through SHT_HIOS for vendor-specific section types, and 
SHF_MASKOS for vendor-specific section attributes). If an object file 
uses any of these values or flag bits, the consumer of the file must 
consult the EI_OSABI field to determine what those values or flags mean. 
It works just like the e_machine field does for attaching meaning to 
processor-specific values and flags.

The intent is that any ABI-conforming implementation will be able to 
execute an ABI-conforming binary, even if it uses certain vendor-specific 
features. In many cases, those vendor-specific features are hints for a 
particular OS that can be ignored by other implementations. Where this is 
not the case, and a vendor-specific feature must be understood by the 
system in order to process the file correctly, we have a couple of 
alternatives.

For section types and flags that a linker must understand, we have the 
SHF_OS_NONCONFORMING flag -- if set, and a linker doesn't understand a 
particular section type or flag, it must reject the file.

For executables that will execute only on a particular implementation, we 
must use an alternate interpreter (PT_INTERP) or bind to 
implementation-specific shared libraries. An ABI-conforming binary will 
use the interpreter specified in the psABI spec, and will use only system 
libraries specified there.

For statically-bound programs, I'm afraid we don't have a clear solution. 
We took the general approach that such programs are not ABI-conforming in 
the first place, and can use any mechanism they might choose to verify 
that they are executing on the appropriate implementation.

-cary

Re: [parisc-linux] Use of the EI_OSABI field

[David O'Brien]

On Tue, Nov 21, 2000 at 03:27:19PM -0800, Ulrich Drepper wrote:
> Alan Modra <alan@linuxcare.com.au> writes:
> 
> > Ulrich Drepper also was quite vehement against changing sourceware
> > FreeBSD binutils.
> 
> I've never said anything about any *BSD, why should I?  The *BSD
> people wanted to change the Linux binutils.

No (don't put words in my mouth Ulrich as you'll be wrong 99% of the
time).  I wanted the Sourceware Binutils to set the EI_OSABI to
"ELFOSABI_LINUX" when targeting Linux.
 
-- 
-- David  (obrien@FreeBSD.org)

Re: [parisc-linux] Use of the EI_OSABI field

[David O'Brien]

On Tue, Nov 21, 2000 at 07:11:29PM -0800, H . J . Lu wrote:
> When I was at the IA64 ABI meeting yesterday, we were looking at the
> EI_OSABI issue. Everyone at the IA64 ABI meeting agreed that what
> Ulrich and I had said was the correct understanding.
> 
> "The e_ident[EI_OSABI] value identifies the operating system and ABI
> to which the object is targeted" just means if the e_ident[EI_OSABI]


Then is this *extremely* misleading text going to be changed???
"the operating system and" needs to be deleted in the _official_
_published_ specification.

-- 
-- David  (obrien@FreeBSD.org)

Re: [parisc-linux] Use of the EI_OSABI field

[David O'Brien]

On Tue, Nov 21, 2000 at 07:18:21PM -0800, Ulrich Drepper wrote:
> Alan Modra <alan@linuxcare.com.au> writes:
> >   "The e_ident[EI_OSABI] value identifies the operating system and ABI to
> >    which the object is targeted"
> > 
> > Granted, this is only in a processor specific ABI, but it does flatly
> > contradict your assertions that the purpose of EI_OSABI is to flag the
> > presense of ELF extensions.
> 
> The meaning of the field changed over time.  Or better said, some
> people initially understood it differently.

Then lets get the _official_ wording changed.  Obviously my
interpretation wasn't unreasonable.   And the world does not need to have
you as a premadona keeper of the [unwritten] rules of the land.
 
-- 
-- David  (obrien@FreeBSD.org)

Re: [parisc-linux] Use of the EI_OSABI field

[H . J . Lu]

On Sat, Nov 25, 2000 at 12:28:40PM -0800, David O'Brien wrote:
> On Tue, Nov 21, 2000 at 07:11:29PM -0800, H . J . Lu wrote:
> > When I was at the IA64 ABI meeting yesterday, we were looking at the
> > EI_OSABI issue. Everyone at the IA64 ABI meeting agreed that what
> > Ulrich and I had said was the correct understanding.
> > 
> > "The e_ident[EI_OSABI] value identifies the operating system and ABI
> > to which the object is targeted" just means if the e_ident[EI_OSABI]
> 
> 
> Then is this *extremely* misleading text going to be changed???
> "the operating system and" needs to be deleted in the _official_
> _published_ specification.
> 

I will bring this issue up to ia64 psABI and gABI.

-- 
H.J. Lu (hjl@valinux.com)