Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Stanislaw Skowronek]

Hi!

It seems that gcc (with -O; -O0 fixes the issue) will generate R_MIPS_HI16
without succeeding R_MIPS_LO16 (or the other way - but it's not a real
problem that way) in '.rel.text' (not '.rela.text'). According to SGI ELF
spec this combination is invalid (well, addend AHL is created from low 16
bits from HI16 and low 16 bits from LO16, and the actual result of
relocation might depend on the LO16 part - at least this is what I
understood from the specific[a]tion); it also upsets Indy PROM when
converted into ECOFF.

Gcc 3.4.3 does not exhibit this (wanton) behavior. What the heck?

Stanislaw Skowronek

--<=>--
  "There is no pain, you are receding...
   A distant ship, smoke on the horizon.
   You are only coming through in waves,
   Your lips move, but I can't hear what you're saying."

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Richard Sandiford]

Stanislaw Skowronek <sskowron@ET.PUT.Poznan.PL> writes:
> It seems that gcc (with -O; -O0 fixes the issue) will generate R_MIPS_HI16
> without succeeding R_MIPS_LO16 (or the other way - but it's not a real
> problem that way) in '.rel.text' (not '.rela.text'). According to SGI ELF
> spec this combination is invalid (well, addend AHL is created from low 16
> bits from HI16 and low 16 bits from LO16, and the actual result of
> relocation might depend on the LO16 part - at least this is what I
> understood from the specific[a]tion); it also upsets Indy PROM when
> converted into ECOFF.
>
> Gcc 3.4.3 does not exhibit this (wanton) behavior. What the heck?

Remember that support for %hi() and %lo() on REL targets is a GNU extension.
The assembler is expected to reorder the relocations so that the HI16s
come before the LO16s.  It sounds like this isn't happening in your case,
so which version of binutils are you using?

This isn't really a change from gcc 3.4 to "gcc 3.5" (now known as 4.0 ;).
It has long been possible for gcc's assembly code to contain "out of order"
%hi()s and %lo()s.  On the other hand, the more aggressive the optimisers
are, the more likely you are to see it, so the behaviour will certainly
be different in specific testcases.

Richard

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Stanislaw Skowronek]

> Remember that support for %hi() and %lo() on REL targets is a GNU extension.

Erm. Are you sure?

SGI's ELF64 spec says:

"Any of the relocation types may appear in either a SHT_REL or a SHT_RELA
relocation section, except that relocation types involving AHL operands
are forbidden in a 64-bit SHT_REL section and discouraged in a 32-bit
SHT_REL section."

There is no word of GNU there and in any case SGI had their own tools. But
again, it is possible that the idea bounced back and forth...

> The assembler is expected to reorder the relocations so that the HI16s
> come before the LO16s.  It sounds like this isn't happening in your case,
> so which version of binutils are you using?

The user who sent the b0rked binary (`K) uses 2.15.94 or so (I'll ask him
again) / "gcc 3.5". On 2.15 / gcc 3.4.3 there is no problem like this (at
least I can't trigger it).

> This isn't really a change from gcc 3.4 to "gcc 3.5" (now known as 4.0 ;).

Well, one of %hi()s is reordered to beginning of a loop and this is what
makes it unpaired. I don't think that any assembler could fix that.

Thanks for answering,

Stanislaw

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Richard Sandiford]

Stanislaw Skowronek <sskowron@ET.PUT.Poznan.PL> writes:
>> Remember that support for %hi() and %lo() on REL targets is a GNU extension.
>
> Erm. Are you sure?
>
> SGI's ELF64 spec says:
>
> "Any of the relocation types may appear in either a SHT_REL or a SHT_RELA
> relocation section, except that relocation types involving AHL operands
> are forbidden in a 64-bit SHT_REL section and discouraged in a 32-bit
> SHT_REL section."
>
> There is no word of GNU there and in any case SGI had their own tools. But
> again, it is possible that the idea bounced back and forth...

I'm talking about the %hi() and %lo() relocation _operators_,
not the ELF relocations themselves.  The ELF spec has nothing
to say about the syntax of assembler relocation operators.

>> This isn't really a change from gcc 3.4 to "gcc 3.5" (now known as 4.0 ;).
>
> Well, one of %hi()s is reordered to beginning of a loop and this is what
> makes it unpaired. I don't think that any assembler could fix that.

What do you mean?  I'm talking about reordering the relocations in
the .rel.foo section, not reordering the code.  I.e. if you have:

    .text
    ...
    addiu $4,$4,%lo(foo)
    ...
    lui $4,%hi(foo)

the assembler is expected to output the R_MIPS_HI16 .rel.text entry
for the lui before the R_MIPS_LO16 entry for the addiu.

Richard

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Stanislaw Skowronek]

> I'm talking about the %hi() and %lo() relocation _operators_,
> not the ELF relocations themselves.  The ELF spec has nothing
> to say about the syntax of assembler relocation operators.

OK, right :)

> What do you mean?  I'm talking about reordering the relocations in
> the .rel.foo section, not reordering the code.  I.e. if you have:
> 
>     .text
>     ...
>     addiu $4,$4,%lo(foo)
>     ...
>     lui $4,%hi(foo)
> 
> the assembler is expected to output the R_MIPS_HI16 .rel.text entry
> for the lui before the R_MIPS_LO16 entry for the addiu.

If you have something like that:

	.text
	...
loop_label:
	lui $4, %hi(foo)
	addiu $4, $4, %lo(foo)
	...
	jmp loop_label
	...

the compiler might be smart and change it into:

	.text
	...
	lui $4, %hi(foo)
loop_label:
	addiu $4, $4, %lo(foo)
	...
	lui $4, %hi(foo)
	jmp loop_label
	...

for instance, to put the lui into branch delay slot (quite a smart
decision, this one). However now %hi and %lo are unpaired. What should the
tool do?

Cheers,

Stanislaw

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Richard Sandiford]

Stanislaw Skowronek <sskowron@ET.PUT.Poznan.PL> writes:
>> What do you mean?  I'm talking about reordering the relocations in
>> the .rel.foo section, not reordering the code.  I.e. if you have:
>> 
>>     .text
>>     ...
>>     addiu $4,$4,%lo(foo)
>>     ...
>>     lui $4,%hi(foo)
>> 
>> the assembler is expected to output the R_MIPS_HI16 .rel.text entry
>> for the lui before the R_MIPS_LO16 entry for the addiu.
>
> If you have something like that:
>
> 	.text
> 	...
> loop_label:
> 	lui $4, %hi(foo)
> 	addiu $4, $4, %lo(foo)
> 	...
> 	jmp loop_label
> 	...
>
> the compiler might be smart and change it into:
>
> 	.text
> 	...
> 	lui $4, %hi(foo)
> loop_label:
> 	addiu $4, $4, %lo(foo)
> 	...
> 	lui $4, %hi(foo)
> 	jmp loop_label
> 	...
>
> for instance, to put the lui into branch delay slot (quite a smart
> decision, this one). However now %hi and %lo are unpaired. What should the
> tool do?

It should generate:

    R_MIPS_HI16
    R_MIPS_HI16
    R_MIPS_LO16

And yes, the idea that several HI16s can be associated with the same
LO16 is also a GNU extension. ;)

(FWIW: as before, this extension, and indeed the whole idea of "out of
order" or "unpaired" %hi()s, isn't new.  It's been around for 10 years.)

Richard

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Stanislaw Skowronek]

> It should generate:
> 
>     R_MIPS_HI16
>     R_MIPS_HI16
>     R_MIPS_LO16
> 
> And yes, the idea that several HI16s can be associated with the same
> LO16 is also a GNU extension. ;)

Good, no problem - thanks for confirming my darkest suspicions. How can I
detect this? (I've got to emit SGI-compliant ECOFF.) I can emit sham
relocs into .rel.text that point into specially added synthetic
instructions.

Stanislaw

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Richard Sandiford]

Stanislaw Skowronek <sskowron@ET.PUT.Poznan.PL> writes:

>> It should generate:
>> 
>>     R_MIPS_HI16
>>     R_MIPS_HI16
>>     R_MIPS_LO16
>> 
>> And yes, the idea that several HI16s can be associated with the same
>> LO16 is also a GNU extension. ;)
>
> Good, no problem - thanks for confirming my darkest suspicions. How can I
> detect this? (I've got to emit SGI-compliant ECOFF.) I can emit sham
> relocs into .rel.text that point into specially added synthetic
> instructions.

Sorry if this is a dumb question, but why do you need to generate
_relocatable_ ECOFF?  If you really need to do that, I think you'll
just have to force gcc to use assembler macros, ala:

   gcc -mno-explicit-relocs -mno-split-addresses

Richard

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Stanislaw Skowronek]

> Sorry if this is a dumb question, but why do you need to generate
> _relocatable_ ECOFF?

It allows me to boot all Indys and O2s off the same binary. Nice for boot
CDs. Especially that Octanes and Origins should be bootable from another
one... just like IRIX.

> If you really need to do that, I think you'll just have to force gcc
> to use assembler macros, ala:
> 
>    gcc -mno-explicit-relocs -mno-split-addresses

Thanks!

Stanislaw

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Maciej W. Rozycki]

On Tue, 24 May 2005, Stanislaw Skowronek wrote:

> > Sorry if this is a dumb question, but why do you need to generate
> > _relocatable_ ECOFF?
> 
> It allows me to boot all Indys and O2s off the same binary. Nice for boot
> CDs. Especially that Octanes and Origins should be bootable from another
> one... just like IRIX.

 Do they use a different load address so that you keep an "almost fully 
linked" relocatable (i.e. with all objects already included, but still 
done with "-r") and do the final link differently for each of them?  If 
this is the case you should be able to keep that "almost fully linked" 
relocatable as ELF and then just do the final link using ELF and then 
`objcopy' to ECOFF.  That should work for most of the cases, although I've 
seen problems with firmware not recognizing MIPS III ECOFF binaries 
expecting a MIPS I one instead.  AFAIK, `objcopy' doesn't allow you to 
force a different magic number upon a conversion -- this is probably the 
last reason `elf2ecoff' hasn't been removed from the Linux tree yet (and 
you can use the tool indeed if you hit this problem).

 Trying to support GNU extensions in ECOFF is probably hopeless and not 
worth the hassle and the file format is likely to be obsoleted by the 
toolchain soon (if not already done), except from BFD -- which'll let you 
continue doing `objcopy', `objdump', etc.

  Maciej

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Stanislaw Skowronek]

> > It allows me to boot all Indys and O2s off the same binary. Nice for boot
> > CDs. Especially that Octanes and Origins should be bootable from another
> > one... just like IRIX.

>  Do they use a different load address so that you keep an "almost fully 
> linked" relocatable (i.e. with all objects already included, but still 
> done with "-r") and do the final link differently for each of them?

Yes, exactly.

> If this is the case you should be able to keep that "almost fully
> linked"  relocatable as ELF and then just do the final link using ELF
> and then `objcopy' to ECOFF.

I tried objcopy (it was my first thought), for one reason or another it
didn't work (internal BFD error something). Reportedly ECOFF is belly-up
in current GNU binutils - at least it is what I heard.

>  Trying to support GNU extensions in ECOFF is probably hopeless and not 
> worth the hassle and the file format is likely to be obsoleted by the 
> toolchain soon (if not already done), except from BFD -- which'll let you 
> continue doing `objcopy', `objdump', etc.

My input is elf32-tradbigmips. So I entirely don't care for binutils'
ECOFF anymore, and I consider this a good thing. As I said, objcopy didn't
work at all. Fixing BFD is not my job (it's a monster of Frankensteinian
proportions), I think that it is actually much easier to write my own
converter (well, I did it, and it works - except that funny HI/LO mismatch
I'm going to work around).

Stanislaw

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Maciej W. Rozycki]

On Tue, 24 May 2005, Stanislaw Skowronek wrote:

> I tried objcopy (it was my first thought), for one reason or another it
> didn't work (internal BFD error something). Reportedly ECOFF is belly-up
> in current GNU binutils - at least it is what I heard.

 And the bug report is...  Well, where?

> My input is elf32-tradbigmips. So I entirely don't care for binutils'
> ECOFF anymore, and I consider this a good thing. As I said, objcopy didn't
> work at all. Fixing BFD is not my job (it's a monster of Frankensteinian
> proportions), I think that it is actually much easier to write my own
> converter (well, I did it, and it works - except that funny HI/LO mismatch
> I'm going to work around).

 Well, you are not required to fix BFD, but if you don't even report 
problems they are going to be left undiscovered forever...

  Maciej

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Richard Sandiford]

"Maciej W. Rozycki" <macro@linux-mips.org> writes:
>  Trying to support GNU extensions in ECOFF is probably hopeless and not 
> worth the hassle and the file format is likely to be obsoleted by the 
> toolchain soon (if not already done), except from BFD -- which'll let you 
> continue doing `objcopy', `objdump', etc.

Yeah.  It's probably also worth noting that we might (soon?) remove the
-mno-explicit-relocs/-mno-split-addresses mode from gcc.  Having both
modes does add to the maintenance burden and the gas support for PIC
relocation operators has been around for a while now.  (It dates back
to binutils 2.14 IIRC.)

Richard

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Ralf Baechle]

On Tue, May 24, 2005 at 11:40:53AM +0100, Maciej W. Rozycki wrote:

> 
>  Do they use a different load address so that you keep an "almost fully 
> linked" relocatable (i.e. with all objects already included, but still 
> done with "-r") and do the final link differently for each of them?  If 
> this is the case you should be able to keep that "almost fully linked" 
> relocatable as ELF and then just do the final link using ELF and then 
> `objcopy' to ECOFF.  That should work for most of the cases, although I've 
> seen problems with firmware not recognizing MIPS III ECOFF binaries 
> expecting a MIPS I one instead.  AFAIK, `objcopy' doesn't allow you to 
> force a different magic number upon a conversion -- this is probably the 
> last reason `elf2ecoff' hasn't been removed from the Linux tree yet (and 
> you can use the tool indeed if you hit this problem).

The kernel ELF binary is too complicated for objcopy to cope with.  Fixing
objcopy to handle this case correctly turned out to be hopeless but with
a little linker script magic it's possible to keep the kernel vmlinux file
within what elf2ecoff can deal with.

>  Trying to support GNU extensions in ECOFF is probably hopeless and not 
> worth the hassle and the file format is likely to be obsoleted by the 
> toolchain soon (if not already done), except from BFD -- which'll let you 
> continue doing `objcopy', `objdump', etc.

If anything a real ECOFF toolchain would be needed.  Teaching ECOFF about
all the ELF magic like complex handling of certain magic symbols etc. is
a waste of time ...

  Ralf

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Maciej W. Rozycki]

On Tue, 24 May 2005, Ralf Baechle wrote:

> >  Do they use a different load address so that you keep an "almost fully 
> > linked" relocatable (i.e. with all objects already included, but still 
> > done with "-r") and do the final link differently for each of them?  If 
> > this is the case you should be able to keep that "almost fully linked" 
> > relocatable as ELF and then just do the final link using ELF and then 
> > `objcopy' to ECOFF.  That should work for most of the cases, although I've 
> > seen problems with firmware not recognizing MIPS III ECOFF binaries 
> > expecting a MIPS I one instead.  AFAIK, `objcopy' doesn't allow you to 
> > force a different magic number upon a conversion -- this is probably the 
> > last reason `elf2ecoff' hasn't been removed from the Linux tree yet (and 
> > you can use the tool indeed if you hit this problem).
> 
> The kernel ELF binary is too complicated for objcopy to cope with.  Fixing
> objcopy to handle this case correctly turned out to be hopeless but with
> a little linker script magic it's possible to keep the kernel vmlinux file
> within what elf2ecoff can deal with.

 Well, it used to work for me the few times I tried, except from that MIPS 
III magic number problem.  But that's not a binutils' fault.

  Maciej

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Ralf Baechle]

On Tue, May 24, 2005 at 03:50:16PM +0100, Maciej W. Rozycki wrote:

> > The kernel ELF binary is too complicated for objcopy to cope with.  Fixing
> > objcopy to handle this case correctly turned out to be hopeless but with
> > a little linker script magic it's possible to keep the kernel vmlinux file
> > within what elf2ecoff can deal with.
> 
>  Well, it used to work for me the few times I tried, except from that MIPS 
> III magic number problem.  But that's not a binutils' fault.

That's because you haven't booted 2.6 on DECstations yet ;-)  I'm sure
you'll have some elf2ecoff fun ...

  Ralf

Re: Unmatched R_MIPS_HI16/R_MIPS_LO16 on gcc 3.5

[Maciej W. Rozycki]

On Tue, 24 May 2005, Ralf Baechle wrote:

> >  Well, it used to work for me the few times I tried, except from that MIPS 
> > III magic number problem.  But that's not a binutils' fault.
> 
> That's because you haven't booted 2.6 on DECstations yet ;-)  I'm sure
> you'll have some elf2ecoff fun ...

 I won't unless I insist on checking -- my configuration doesn't need 
ECOFF.  It's never needed, actually, hence my lack of incentive in finding 
problems to fix with ECOFF. ;-)

  Maciej