[U-Boot] FPGA relocation/C environment

[U-Boot] FPGA relocation/C environment

[Wolfgang Wegner]

Hi,

I am using U-Boot on a Coldfire to load FPGA code (Xilinx Spartan3
and Altera Cyclone2, currently) and am just trying to update my code
base to current U-Boot for finally sending patches.

Now I recognized that commit 6385b28116f775da4771b768ba9bf93c3aaaf26e
removed FPGA relocation, which of course breaks FPGA code for my
Coldfire board.

What is the "proper relocation for PPC boards" that is mentioned
by Peter in his commit log? I found this test patch by him:
http://lists.denx.de/pipermail/u-boot/2009-September/060582.html
which leads me to the assumption that for PPC, newer compilers
do the right thing (TM) when given -mrelocatable, but
- my m68k-elf-gcc (Sourcery G++ Lite 4.3-208) 4.3.3 (that was the
  newest one I could get my hands on that is able to compile U-Boot
  at all) does not know -mrelocatable
- I do not understand how the compiler could know about the
  relocation for the FPGA (and possibly other) functions when the
  actual relocation is done during U-Boot run time?

Can anybody give some hints where I have to look at or where my
misunderstanding occurs?

Thank you,
Wolfgang

[U-Boot] FPGA relocation/C environment

[Wolfgang Denk]

Dear Wolfgang Wegner,

In message <20091029082113.GE3216@leila.ping.de> you wrote:
> 
> Now I recognized that commit 6385b28116f775da4771b768ba9bf93c3aaaf26e
> removed FPGA relocation, which of course breaks FPGA code for my
> Coldfire board.

This is an out-of-tree port, right? If your code had been in mainline,
this would not have happened.

> What is the "proper relocation for PPC boards" that is mentioned
> by Peter in his commit log? I found this test patch by him:
...
> - I do not understand how the compiler could know about the
>   relocation for the FPGA (and possibly other) functions when the
>   actual relocation is done during U-Boot run time?

The compiler / linker  create  a  Global  Offset  Table  (GOT)  which
basicly  is  a  list of offsets to the image, where address constants
need to be updated to relocate the image to a new  address.  This  is
what the U-Boot relocation code does: add a (fixed) relocation offset
(target  address - source address) to all the locations listed in the
GOT. The problem was, that certain  types  of  addresses  (statically
initialized function pointers) did not get listed in the GOT. This is
now fixed.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"Remember, Information is not knowledge;  Knowledge  is  not  Wisdom;
Wisdom is not truth; Truth is not beauty; Beauty is not love; Love is
not music; Music is the best."                          - Frank Zappa

[U-Boot] FPGA relocation/C environment

[Wolfgang Wegner]

Dear Wolfgang Denk,

On Thu, Oct 29, 2009 at 12:00:00PM +0100, Wolfgang Denk wrote:
> Dear Wolfgang Wegner,
> 
> In message <20091029082113.GE3216@leila.ping.de> you wrote:
> > 
> > Now I recognized that commit 6385b28116f775da4771b768ba9bf93c3aaaf26e
> > removed FPGA relocation, which of course breaks FPGA code for my
> > Coldfire board.
> 
> This is an out-of-tree port, right? If your code had been in mainline,
> this would not have happened.

I know, as I mentioned in the first place, I am right now updating to
get it integrated. (see below for the footnote)

> > What is the "proper relocation for PPC boards" that is mentioned
> > by Peter in his commit log? I found this test patch by him:
> ...
> > - I do not understand how the compiler could know about the
> >   relocation for the FPGA (and possibly other) functions when the
> >   actual relocation is done during U-Boot run time?
> 
> The compiler / linker  create  a  Global  Offset  Table  (GOT)  which
> basicly  is  a  list of offsets to the image, where address constants
> need to be updated to relocate the image to a new  address.  This  is
> what the U-Boot relocation code does: add a (fixed) relocation offset
> (target  address - source address) to all the locations listed in the
> GOT. The problem was, that certain  types  of  addresses  (statically
> initialized function pointers) did not get listed in the GOT. This is
> now fixed.

Thank you for the clarification! I will see if I can find out in which
state gcc and U-Boot for coldfire are in this respect.

Best regards,
Wolfgang

[At the time of the original port, MCF532x/7x was not present in U-Boot;
my own port had quite many conflicts (probably because of me being still
very unfamiliar with U-Boot and its concepts) with the upcoming official
port, which in turn had some severe problems at least with my setup. This
and the fact that I needed some time to get at least a bit familiar with
git led to me sticking with my old out-of-tree port. Now as the official
MCF537x port is settled I am in the process of doing my homework...]

[U-Boot] FPGA relocation/C environment

[Joakim Tjernlund]

>
> Dear Wolfgang Wegner,
>
> In message <20091029082113.GE3216@leila.ping.de> you wrote:
> >
> > Now I recognized that commit 6385b28116f775da4771b768ba9bf93c3aaaf26e
> > removed FPGA relocation, which of course breaks FPGA code for my
> > Coldfire board.
>
> This is an out-of-tree port, right? If your code had been in mainline,
> this would not have happened.
>
> > What is the "proper relocation for PPC boards" that is mentioned
> > by Peter in his commit log? I found this test patch by him:
> ...
> > - I do not understand how the compiler could know about the
> >   relocation for the FPGA (and possibly other) functions when the
> >   actual relocation is done during U-Boot run time?
>
> The compiler / linker  create  a  Global  Offset  Table  (GOT)  which
> basicly  is  a  list of offsets to the image, where address constants

Just to avoid any misunderstandings, the GOT does not hold offsets, it holds
absolute addresses. These addresses gets adjusted with an offset when you
relocate.

[U-Boot] FPGA relocation/C environment

[Wolfgang Wegner]

Dear Joakim Tjernlund,

On Thu, Oct 29, 2009 at 02:08:05PM +0100, Joakim Tjernlund wrote:
> 
> Just to avoid any misunderstandings, the GOT does not hold offsets, it holds
> absolute addresses. These addresses gets adjusted with an offset when you
> relocate.

thank you, I already understood this.

As far as I can see there is already some GOT handling in cpu/mcf532x/start.S
and the flags (-fPIC) in cpu/mcf532x/config.mk (and other coldfire ports, but
MCF532x/MCF537x is what I am currently working on for testing), but somehow
it does not work.
However, it takes some time for me to understand the assembler code, 
especially the powerpc counterpart I am using as a reference.

There is a __got_start and __got_end in my u-boot.map, so I would think
the linker does behave correctly - but I did not yet check if all the
symbols (better: symbols' addresses) expected to be there are really
present:

.reloc          0x00024600     0x1990
                0x00024600                __got_start = .
 *(.got)
 .got           0x00024600     0x1990 cpu/mcf532x/cpu_init.o
                0x00025f90                __got_end = .
                0x00025f90                _GOT2_TABLE_ = .
 *(.got2)
                0x00025f90                _FIXUP_TABLE_ = .
 *(.fixup)
                0x00000000                __got2_entries = ((_FIXUP_TABLE_ - _GOT2_TABLE_) >> 0x2)
                0x00000000                __fixup_entries = ((. - _FIXUP_TABLE_) >> 0x2)

(flash resides at 0x0, SDRAM at 0x4000000 and CONFIG_SYS_MONITOR_BASE is
0x40020000)

Regards,
Wolfgang

[U-Boot] FPGA relocation/C environment

[Joakim Tjernlund]

wolfgang at leila.ping.de (Wolfgang Wegner) wrote on 29/10/2009 14:41:00:
>
> Dear Joakim Tjernlund,
>
> On Thu, Oct 29, 2009 at 02:08:05PM +0100, Joakim Tjernlund wrote:
> >
> > Just to avoid any misunderstandings, the GOT does not hold offsets, it holds
> > absolute addresses. These addresses gets adjusted with an offset when you
> > relocate.
>
> thank you, I already understood this.
>
> As far as I can see there is already some GOT handling in cpu/mcf532x/start.S
> and the flags (-fPIC) in cpu/mcf532x/config.mk (and other coldfire ports, but
> MCF532x/MCF537x is what I am currently working on for testing), but somehow
> it does not work.
> However, it takes some time for me to understand the assembler code,
> especially the powerpc counterpart I am using as a reference.
>
> There is a __got_start and __got_end in my u-boot.map, so I would think
> the linker does behave correctly - but I did not yet check if all the
> symbols (better: symbols' addresses) expected to be there are really
> present:
>
> .reloc          0x00024600     0x1990
>                 0x00024600                __got_start = .
>  *(.got)
>  .got           0x00024600     0x1990 cpu/mcf532x/cpu_init.o
>                 0x00025f90                __got_end = .
>                 0x00025f90                _GOT2_TABLE_ = .
>  *(.got2)
>                 0x00025f90                _FIXUP_TABLE_ = .
>  *(.fixup)
>                 0x00000000                __got2_entries = ((_FIXUP_TABLE_ -
> _GOT2_TABLE_) >> 0x2)
>                 0x00000000                __fixup_entries = ((. - _FIXUP_TABLE_) >> 0x2)

It seems like you don't have any relocation data as both __got2_entries and
__fixup_entries are zero. Something seems broken in general, perhaps your linker script
is bust?

>
> (flash resides at 0x0, SDRAM at 0x4000000 and CONFIG_SYS_MONITOR_BASE is
> 0x40020000)

[U-Boot] FPGA relocation/C environment

[Wolfgang Wegner]

On Thu, Oct 29, 2009 at 03:22:24PM +0100, Joakim Tjernlund wrote:
> 
> It seems like you don't have any relocation data as both __got2_entries and
> __fixup_entries are zero. Something seems broken in general, perhaps your linker script
> is bust?

I took board/freescale/m5373evb/u-boot.lds without modifications first.
After noticing that the PPC linker scripts do not have this entry (at this
place), I removed "*(.fixup)" in the .text section, but this did not change
anything.
However, in my u-boot.map, I have some .data.rel entries which look as
if they would belong into the .fixup section?!
[...]
.data.rel      0x00026eb4       0x30 lib_m68k/libm68k.a(board.o)
                0x00026eb4                init_sequence
 .data.rel      0x00026ee4       0x4c common/libcommon.a(cmd_bootm.o)
                0x00026ee4                boot_os  
 .data.rel      0x00026f30       0x24 common/libcommon.a(cmd_mii.o)
                0x00026f30                desc_and_len_tbl
 .data.rel      0x00026f54       0x88 board/astro/mcf5373l/libmcf5373l.a(fpga.o)
                0x00026f88                xilinx_fns 
                0x00026fa8                xilinx_qspi_fns
                0x00026f70                altera_fpga
                0x00026f54                altera_fns
                0x00026fc8                xilinx_fpga
                0x00026fdc                . = .
                0x00026fdc                __u_boot_cmd_start = .
[...]

(From fpga.c:
[...]
Xilinx_Spartan3_Slave_Serial_fns xilinx_fns = {
	xilinx_pre_config_fn,
	xilinx_pgm_fn,
	xilinx_clk_fn,
	xilinx_init_fn,
	xilinx_done_fn,
	xilinx_wr_fn,
	0,
	xilinx_fastwr_fn
};

Xilinx_desc xilinx_fpga[CONFIG_FPGA_COUNT] = {
	{Xilinx_Spartan3,
	 slave_serial,
	 XILINX_XC3S4000_SIZE,
#if FPGA_QSPI
	 (void *) &xilinx_qspi_fns,
#else
	 (void *) &xilinx_fns,
#endif
	 0}
};
[...]
)

I did not find a special entry for .data.rel (or .data.*) in the
PPC u-boot.lds either, so maybe this is something the linker should
do automatically?

Regards,
Wolfgang

[U-Boot] FPGA relocation/C environment

[Wolfgang Wegner]

Hi,

although I have to leave for now, just some questions to see if
there is anything I understood correctly until now...

- in the PPC startup (assembly) files, the section .got2 is explicitely
  created using START_GOT etc.
- .got2 contains some vital pointers (vectors) as well as data
  pointers like a pointer to .fixup and a pointer to itself (?!)
- the ppc linker automatically puts .data.rel (and other relocatable
  sections?) into .fixup (which seems not the case with my m68k linker)
- the ppc startup code uses some obfuscated assembly code ;-) to
  relocate both the global relocation table (.got) as well as the
  sections .got2 and .fixup (.got seems to be implicitly mentioned:
  "/* First our own GOT */
	add	r14, r14, r15
	/* then the one used by the C code */
	add	r30, r30, r15"
  but I can not see where it is actually relocated...)

Is this correct so far? I am not sure because it seems more complicated
than necessary to me (especially the algorithm to relocate .got2 and
.fixup - or is this all about the possibility to do multiple relocations?),
so maybe I still have a misunderstanding here. Furthermore, the special
handling of r14 (-ffixed-r14 in config.mk) still puzzles me.

Thank you and best regards,
Wolfgang

[U-Boot] FPGA relocation/C environment

[Joakim Tjernlund]

wolfgang at leila.ping.de (Wolfgang Wegner) wrote on 29/10/2009 16:00:04:
>
> On Thu, Oct 29, 2009 at 03:22:24PM +0100, Joakim Tjernlund wrote:
> >
> > It seems like you don't have any relocation data as both __got2_entries and
> > __fixup_entries are zero. Something seems broken in general, perhaps your
> linker script
> > is bust?
>
> I took board/freescale/m5373evb/u-boot.lds without modifications first.
> After noticing that the PPC linker scripts do not have this entry (at this
> place), I removed "*(.fixup)" in the .text section, but this did not change
> anything.
> However, in my u-boot.map, I have some .data.rel entries which look as
> if they would belong into the .fixup section?!
> [...]
> .data.rel      0x00026eb4       0x30 lib_m68k/libm68k.a(board.o)
>                 0x00026eb4                init_sequence
>  .data.rel      0x00026ee4       0x4c common/libcommon.a(cmd_bootm.o)
>                 0x00026ee4                boot_os
>  .data.rel      0x00026f30       0x24 common/libcommon.a(cmd_mii.o)
>                 0x00026f30                desc_and_len_tbl
>  .data.rel      0x00026f54       0x88 board/astro/mcf5373l/libmcf5373l.a(fpga.o)
>                 0x00026f88                xilinx_fns
>                 0x00026fa8                xilinx_qspi_fns
>                 0x00026f70                altera_fpga
>                 0x00026f54                altera_fns
>                 0x00026fc8                xilinx_fpga
>                 0x00026fdc                . = .
>                 0x00026fdc                __u_boot_cmd_start = .

Compile one small file into asm (use -S to gcc) with and without -fPIC
then compare where the relocation data goes. Normally
they will be in .got2 but perhaps your compiler does something else?

[U-Boot] FPGA relocation/C environment

[Joakim Tjernlund]

wolfgang at leila.ping.de (Wolfgang Wegner) wrote on 29/10/2009 16:44:55:
>
> Hi,
>
> although I have to leave for now, just some questions to see if
> there is anything I understood correctly until now...
>
> - in the PPC startup (assembly) files, the section .got2 is explicitely
>   created using START_GOT etc.

It adds a few entries to .got2 as the asm needs to access a few entries vis the
GOT. If you look at the asm generated by gcc, it does the same for global data.

> - .got2 contains some vital pointers (vectors) as well as data
>   pointers like a pointer to .fixup and a pointer to itself (?!)

The first entry in got2 host a pointer to itself, this is needed by relocation later.
.fixup entries is created by -mrelocatable and WD has mentioned what these are for.

> - the ppc linker automatically puts .data.rel (and other relocatable
>   sections?) into .fixup (which seems not the case with my m68k linker)
> - the ppc startup code uses some obfuscated assembly code ;-) to
>   relocate both the global relocation table (.got) as well as the
>   sections .got2 and .fixup (.got seems to be implicitly mentioned:
>   "/* First our own GOT */
>    add   r14, r14, r15
>    /* then the one used by the C code */
>    add   r30, r30, r15"
>   but I can not see where it is actually relocated...)

As I recall that code is a leftover, at least the C version.
Relocation happens directly after the in_ram: label.
However, this is not you problem. You don't have any relocation entries
so there is nothing to relocate. Your linker/linker script is broken.

>
> Is this correct so far? I am not sure because it seems more complicated
> than necessary to me (especially the algorithm to relocate .got2 and
> .fixup - or is this all about the possibility to do multiple relocations?),
> so maybe I still have a misunderstanding here. Furthermore, the special
> handling of r14 (-ffixed-r14 in config.mk) still puzzles me.

The -ffixed-r14 is for IRQ procressing. The irq code needs to
jump to transfer_to_handler via the GOT and needs r14 to hold
a ptr to the GOT at all times. This can be avoided and I have a patch
to get rid of ffixed-r14 but only for mpc83xx so I or someone else
needs to do the other boards too before it can be committed.

  Jocke

[U-Boot] FPGA relocation/C environment

[Wolfgang Wegner]

Hi,

thank you all for all the comments so far.

It seems I have at least 2 problems:
- my gcc (m68k-elf-gcc (Sourcery G++ Lite 4.3-208) 4.3.3) does not
  produce correctly relocatable code; no matter if I give -fPIC or
  not (-mrelocatable is not accepted at all), I do not get any
  entries in .got2 or .fixup, only .got
  However, there still may be anything wrong in my conclusion,
  because I can use this compiler to compile uClinux-dist-20071107
  and get working applications (which, as far as I understand, have
  to be relocated at runtime, too, but I do not know if any of these
  use such features as statically initialized function pointers that
  I have in the U-Boot fpga code...).
- the current coldfire startup code (at least for MCF532x/MCF537x)
  would not handle these sections even if they existed - however,
  this seems to be rather easily (i.e. maybe even I could do it)
  fixed, looking at the ppc examples.

I will first live with the problem (relocating the fpga struct pointers
in my own board code, where it is filled anyways - for coldfire,
gd->reloc_off is still present) and see if I can get hold of a better
gcc version.
I tried gcc-4.4.1, but was unable to compile one of U-Boot or uClinux,
so switched back to this one which I found to compile (old and current)
U-Boot, old uClinux _and_ support mcf5445x, which I am designing in
now...

Thank you and sorry for my stupid questions.

Regards,
Wolfgang

[U-Boot] FPGA relocation/C environment

[Graeme Russ]

Wolfgang Wegner wrote:
> Hi,
> 
> thank you all for all the comments so far.
> 
> It seems I have at least 2 problems:
> - my gcc (m68k-elf-gcc (Sourcery G++ Lite 4.3-208) 4.3.3) does not
>   produce correctly relocatable code; no matter if I give -fPIC or
>   not (-mrelocatable is not accepted at all), I do not get any
>   entries in .got2 or .fixup, only .got

-mreloctable only applies to PPC CPUs - not version of gcc will give you
.got2 and .fixup unless you are building for PPC

>   However, there still may be anything wrong in my conclusion,
>   because I can use this compiler to compile uClinux-dist-20071107
>   and get working applications (which, as far as I understand, have
>   to be relocated at runtime, too, but I do not know if any of these
>   use such features as statically initialized function pointers that
>   I have in the U-Boot fpga code...).

They are build and linked as pic or pie (position independent executable)
This puts information in the elf binary that the dynamic loader/linker
can user to modify the executable as it is loaded into memory to enable
it to run at the target load address

> - the current coldfire startup code (at least for MCF532x/MCF537x)
>   would not handle these sections even if they existed - however,
>   this seems to be rather easily (i.e. maybe even I could do it)
>   fixed, looking at the ppc examples.

You might want to look at the x86 example instead. I posted a patch a
while ago (http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/68923)

> 
> I will first live with the problem (relocating the fpga struct pointers
> in my own board code, where it is filled anyways - for coldfire,
> gd->reloc_off is still present) and see if I can get hold of a better
> gcc version.

Yes, this manual 'fixup' will work, but the long term goal is to remove
this

> I tried gcc-4.4.1, but was unable to compile one of U-Boot or uClinux,
> so switched back to this one which I found to compile (old and current)
> U-Boot, old uClinux _and_ support mcf5445x, which I am designing in
> now...
> 
> Thank you and sorry for my stupid questions.

Not stupid at all - Proper relocation can be very tricky to achieve. I
spent a _long_ time getting it to work for x86, and in the end the
solution is rather trivial - Have a search through the u-boot mailing
list ;)

> 
> Regards,
> Wolfgang
> 

Regards,

Graeme