[U-Boot-Users] Changing u-boot relocation scheme

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

Wolfgang,

I wonder if you would consider (and accept a patch) changing the
u-boot architecture as described below.

As we all are well aware u-boot is a great software package which
speeds up bringing up new hardware immensely, its popularity is
obviously growing very fast. One thing which (IMO anyway) slows down
its acceptance is the way it handles relocating of itself into an
arbitrary DRAM area. (Arbitrary meaning that it depends on many
factors and the exact address can't be assigned ahead of time).

While this relocation is seamless for u-boot in its released form, it
becomes a pain each time a module needs to be added (not necessarily
for following release to the Open Source community, for instance used
for inhouse installations which include tens of thousand units for
some companies). If these added modules were not written in position
independent manner (namely, using structures with multiple stage
indirect pointers interleaved with data), the effort to make these
modules work in u-boot is very exhausting.

Here is the suggested alternative. I will talk about recent PPC
platforms, but it could be done very similar for most of the other
platforms I believe.

In general the PPC u-boot image consists of three parts: the bulk of
code and data, the startup page (the top 4k mapped out of reset), and
the reset vector (the top four bytes in the address space). The bulk
is usually aligned at a certain address matching the bootprom size,
and there is a gap between the bulk and the startup page.

Now, let's say we link and purge the bulk section twice, once locating
it at offset 0 and once at the normal u-boot TEXT_BASE, building two
binary images (not including the top 4k in both cases).

Then we compare the two images and identify all locations which are
different. All these locations are 4 byte aligned, and all differ by
the same amount - equal TEXT_BASE. Now we can build a table of offsets
which need to be changed when the program is moved into a different
location, include this table in the resulting image between the bulk
and the startup page, and use this table when relocating code.

Basically, this implements a poor man's loader, and it will be
sufficient in case the image we are dealing with is a flat binary
(which is of course the case with u-boot). This will allow to add any
code to u-boot without much worries about being able to relocate it
properly. We would do away with storing certain structures in certain
sections, dealing with GOT, etc.

I made some experiments, and this seems feasible, this could be done
as a local customization, but I would much prefer to release it to the
u-boot community and make it part of mainline - will you consider such
a patch?

Thank you for reading this far, please let me know what you think,

Vadim

[U-Boot-Users] Changing u-boot relocation scheme

[Jerry Van Baren]

vb wrote:
> Wolfgang,
> 
> I wonder if you would consider (and accept a patch) changing the
> u-boot architecture as described below.
> 
> As we all are well aware u-boot is a great software package which
> speeds up bringing up new hardware immensely, its popularity is
> obviously growing very fast. One thing which (IMO anyway) slows down
> its acceptance is the way it handles relocating of itself into an
> arbitrary DRAM area. (Arbitrary meaning that it depends on many
> factors and the exact address can't be assigned ahead of time).

One thing (IMO anyway) that allows u-boot to Just Work[tm] is that it 
relocates itself to the best location available, automatically adjusting 
to the probed memory configuration.

There.  Fixed that.  ;-)

> While this relocation is seamless for u-boot in its released form, it
> becomes a pain each time a module needs to be added (not necessarily
> for following release to the Open Source community, for instance used
> for inhouse installations which include tens of thousand units for
> some companies). If these added modules were not written in position
> independent manner (namely, using structures with multiple stage
> indirect pointers interleaved with data), the effort to make these
> modules work in u-boot is very exhausting.

You lost me here.

What is a module?
How does a it relate to u-boot?
Why does it need to be relocated?
Why isn't it written in position independent manner?
How is your relocation methodology going to fixup a module's data 
structures?
What pointers are in your data structures?  Why do they need to be 
relocated?

> Here is the suggested alternative. I will talk about recent PPC
> platforms, but it could be done very similar for most of the other
> platforms I believe.
> 
> In general the PPC u-boot image consists of three parts: the bulk of
> code and data, the startup page (the top 4k mapped out of reset), and
> the reset vector (the top four bytes in the address space). The bulk
> is usually aligned at a certain address matching the bootprom size,
> and there is a gap between the bulk and the startup page.
> 
> Now, let's say we link and purge the bulk section twice, once locating
> it at offset 0 and once at the normal u-boot TEXT_BASE, building two
> binary images (not including the top 4k in both cases).
> 
> Then we compare the two images and identify all locations which are
> different. All these locations are 4 byte aligned, and all differ by
> the same amount - equal TEXT_BASE. Now we can build a table of offsets
> which need to be changed when the program is moved into a different
> location, include this table in the resulting image between the bulk
> and the startup page, and use this table when relocating code.


Gcc supports "proper" relocation, if only we knew how to make it work 
for all "reasonable" versions of gcc (and Grant's conclusion is that 
many versions of gcc in use today do *not* support the relocation).
   <http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/>

Grant Likely created a patch that did this in the 1.3.0 timeframe, but 
ended up reverting it due to toolset problems - it was a patch before 
its time.

To read up on the history, follow this search:
<http://search.gmane.org/?query=relocation&author=grant+likely&group=gmane.comp.boot-loaders.u-boot&sort=date&DEFAULTOP=and&xP=Zreloc%09Zchang%09Zgrant%09Zlike&xFILTERS=Gcomp.boot-loaders.u-boot---A>

> Basically, this implements a poor man's loader, and it will be
> sufficient in case the image we are dealing with is a flat binary
> (which is of course the case with u-boot). This will allow to add any
> code to u-boot without much worries about being able to relocate it
> properly. We would do away with storing certain structures in certain
> sections, dealing with GOT, etc.
> 
> I made some experiments, and this seems feasible, this could be done
> as a local customization, but I would much prefer to release it to the
> u-boot community and make it part of mainline - will you consider such
> a patch?
> 
> Thank you for reading this far, please let me know what you think,

On one hand, your proposal sounds fairly simple and independent of the 
gcc version.  The primary cost would be extra build steps
1) Link at TEXT_BASE
2) Link at address 0
3) Binary diff the two to find addresses that need relocation
4) Rebuild with the diff table (compile the diff table and relink)

On the other hand, the skeptic in me says it ain't that easy.  Go ahead, 
prove me wrong.  ;-)

> Vadim

Best regards,
gvb

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

Jerry, thank you for your reply, please see below:

On Wed, Jul 23, 2008 at 2:46 PM, Jerry Van Baren <gerald.vanbaren@ge.com> wrote:
> vb wrote:
>>  One thing which (IMO anyway) slows down
>> its acceptance is the way it handles relocating of itself into an
>> arbitrary DRAM area. (Arbitrary meaning that it depends on many
>> factors and the exact address can't be assigned ahead of time).
>
> One thing (IMO anyway) that allows u-boot to Just Work[tm] is that it
> relocates itself to the best location available, automatically adjusting to
> the probed memory configuration.
>
> There.  Fixed that.  ;-)
>


Sorry, I was not clear - it indeed is a great property of u-boot that
it can run on any probed memory config. What I meant to say was that
the way it relocates itself is not perfect, and this is why u-boot is
not becoming more ubiquitous.

>> While this relocation is seamless for u-boot in its released form, it
>> becomes a pain each time a module needs to be added (not necessarily
>> for following release to the Open Source community, for instance used
>> for inhouse installations which include tens of thousand units for
>> some companies). If these added modules were not written in position
>> independent manner (namely, using structures with multiple stage
>> indirect pointers interleaved with data), the effort to make these
>> modules work in u-boot is very exhausting.
>
> You lost me here.
>
> What is a module?

for the purposes of this discussion any .o file or any library of .o files.

> How does a it relate to u-boot?

I need to extend u-boot with certain functionality available from those modules.

> Why does it need to be relocated?

because the  modules are now part of u-boot, they need to be relocated
along with the rest of the image.

> Why isn't it written in position independent manner?

because they come from different sources, not concerned with u-boot,
and just work and need to be ported as is.

> How is your relocation methodology going to fixup a module's data
> structures?

The relocation will adjust only pointers to absolute addresses stored
in text/data segments.

> What pointers are in your data structures?  Why do they need to be
> relocated?
>

in some cases data structures include pointers to other data
structures, etc. (This is called  'tree' btw, sorry couldn't resist
:-)

[snip]
>
> Gcc supports "proper" relocation, if only we knew how to make it work for
> all "reasonable" versions of gcc (and Grant's conclusion is that many
> versions of gcc in use today do *not* support the relocation).
>  <http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/>
>

well, if there is a pointer to a data element or to a function stored
in a data structure, the .bin image is built and fixed using actual
addresses set by the linker. How can gcc help here?

> Grant Likely created a patch that did this in the 1.3.0 timeframe, but ended
> up reverting it due to toolset problems - it was a patch before its time.
>
> To read up on the history, follow this search:
> <http://search.gmane.org/?query=relocation&author=grant+likely&group=gmane.comp.boot-loaders.u-boot&sort=date&DEFAULTOP=and&xP=Zreloc%09Zchang%09Zgrant%09Zlike&xFILTERS=Gcomp.boot-loaders.u-boot---A>
>

I will read up on this.

[snip]

>> I made some experiments, and this seems feasible, this could be done
>> as a local customization, but I would much prefer to release it to the
>> u-boot community and make it part of mainline - will you consider such
>> a patch?
>>
>> Thank you for reading this far, please let me know what you think,
>
> On one hand, your proposal sounds fairly simple and independent of the gcc
> version.  The primary cost would be extra build steps
> 1) Link at TEXT_BASE
> 2) Link at address 0
> 3) Binary diff the two to find addresses that need relocation
> 4) Rebuild with the diff table (compile the diff table and relink)
>

add here inserting code to process the table and modify pointers after
relocation (which is already being done, but suing a different
algorithm)

> On the other hand, the skeptic in me says it ain't that easy.  Go ahead,
> prove me wrong.  ;-)
>

I am sure it is doable, I am just trying to understand if this would
be accepted in general, because it can be done differently depending
if one needs to push it upstream or not.

cheers,
/vb

>> Vadim
>
> Best regards,
> gvb
>

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <f608b67d0807231039i434e96dbvda86590776db2cb0@mail.gmail.com> you wrote:
> 
> I wonder if you would consider (and accept a patch) changing the
> u-boot architecture as described below.

Consider: always. Accept: if it makes more sense than the current
implementation.

> As we all are well aware u-boot is a great software package which
> speeds up bringing up new hardware immensely, its popularity is
> obviously growing very fast. One thing which (IMO anyway) slows down
> its acceptance is the way it handles relocating of itself into an
> arbitrary DRAM area. (Arbitrary meaning that it depends on many
> factors and the exact address can't be assigned ahead of time).

The address is not arbitrary, and in fact it CANNOT be determined in
advance, especially on boards with varying RAM sized (for example
using standard DIMMs). Since U-Boot will relocate itself normally to
the end of the available RAM, you cannot know the relocation address
in advance. Also, for example things like the PRAM size can be set by
the user through environment variables - again this will cause the
relocation address to become variable.

> While this relocation is seamless for u-boot in its released form, it
> becomes a pain each time a module needs to be added (not necessarily
> for following release to the Open Source community, for instance used
> for inhouse installations which include tens of thousand units for
> some companies). If these added modules were not written in position
> independent manner (namely, using structures with multiple stage
> indirect pointers interleaved with data), the effort to make these
> modules work in u-boot is very exhausting.

I don't understand what you mean. Either you link statically with the
U-Boot image, or you use standalone programs. In both situations no
such problem as described by you exists.

> Here is the suggested alternative. I will talk about recent PPC
> platforms, but it could be done very similar for most of the other
> platforms I believe.

I don't think so.

> In general the PPC u-boot image consists of three parts: the bulk of

No, this is by far not general, not even for PPC.

> Now, let's say we link and purge the bulk section twice, once locating
> it at offset 0 and once at the normal u-boot TEXT_BASE, building two
> binary images (not including the top 4k in both cases).
> 
> Then we compare the two images and identify all locations which are
> different. All these locations are 4 byte aligned, and all differ by
> the same amount - equal TEXT_BASE. Now we can build a table of offsets
> which need to be changed when the program is moved into a different
> location, include this table in the resulting image between the bulk
> and the startup page, and use this table when relocating code.

Why would we do that? The compiler already generates such a table for
free - the GOT.

> Basically, this implements a poor man's loader, and it will be
> sufficient in case the image we are dealing with is a flat binary
> (which is of course the case with u-boot). This will allow to add any
> code to u-boot without much worries about being able to relocate it
> properly. We would do away with storing certain structures in certain
> sections, dealing with GOT, etc.

I don't see how your "poor man's" GOT would be any better than the
compiler generated one - except that it's horribly non-standard and
glumpsy?

> I made some experiments, and this seems feasible, this could be done
> as a local customization, but I would much prefer to release it to the
> u-boot community and make it part of mainline - will you consider such
> a patch?

I don't understand which problem you're trying to fix, and I don't see
that your solution would be any better than what we have now.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
I object to intellect without discipline;  I object to power without
constructive purpose.
	-- Spock, "The Squire of Gothos", stardate 2124.5

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <f608b67d0807231546t44ef5a6ak68d98b26beb5d8fc@mail.gmail.com> you wrote:
> 
> > What is a module?
> 
> for the purposes of this discussion any .o file or any library of .o files.

Neither of these is executable - you either have to link these with
the U-Boot code or into a standalone application - and in both cases
none of the problems you describe happens.

> > How does a it relate to u-boot?
> 
> I need to extend u-boot with certain functionality available from those modules.

OK - as mentioned: either statically linked or as standalone app.

> > Why does it need to be relocated?
> 
> because the  modules are now part of u-boot, they need to be relocated
> along with the rest of the image.

U-Boot does this (if statically linked) the very same way like all
other U-Boot code gets relocated.

> > Why isn't it written in position independent manner?
> 
> because they come from different sources, not concerned with u-boot,
> and just work and need to be ported as is.

If you want to link against U-Boot, you have to use at least
compatible compiler options and the U-Boot linker script.

> > How is your relocation methodology going to fixup a module's data
> > structures?
> 
> The relocation will adjust only pointers to absolute addresses stored
> in text/data segments.

No.

> > What pointers are in your data structures?  Why do they need to be
> > relocated?
> 
> in some cases data structures include pointers to other data
> structures, etc. (This is called  'tree' btw, sorry couldn't resist
> :-)

This should get sorted out correctly when linking.

> > Gcc supports "proper" relocation, if only we knew how to make it work for
> > all "reasonable" versions of gcc (and Grant's conclusion is that many
> > versions of gcc in use today do *not* support the relocation).
> >  <http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/>
> 
> well, if there is a pointer to a data element or to a function stored
> in a data structure, the .bin image is built and fixed using actual
> addresses set by the linker. How can gcc help here?

By adding the respective entries to the GOT.

> I am sure it is doable, I am just trying to understand if this would
> be accepted in general, because it can be done differently depending
> if one needs to push it upstream or not.

If you invest time in solving such problems, than your time willbe
much better iinvested if you try to help solving the remaining issues
with  Grant's code.

What Grant suggests is the way to go. I do not think your approach has
chances for mainline.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Nothing in progression can rest on its original plan. We may as  well
think  of  rocking  a grown man in the cradle of an infant.
- Edmund Burke

[U-Boot-Users] Changing u-boot relocation scheme

[Robert Schwebel]

On Wed, Jul 23, 2008 at 10:39:00AM -0700, vb wrote:
> While this relocation is seamless for u-boot in its released form, it
> becomes a pain each time a module needs to be added (not necessarily
> for following release to the Open Source community, for instance used
> for inhouse installations which include tens of thousand units for
> some companies). If these added modules were not written in position
> independent manner (namely, using structures with multiple stage
> indirect pointers interleaved with data), the effort to make these
> modules work in u-boot is very exhausting.

Note that in U-Boot-v2 we have loadable module support; code can
EXPORT_SYMBOL() "public" APIs to other components, which, for example,
can be such custom modules. The mechanism was invented for the scenario
you describe above.

It comes for a cost (footprint), but the overhead is not so big as one
might think. You can play around with the feature by building the
"sandbox" target, which is a user-mode-u-boot (port of u-boot to Linux
userspace).

rsc
-- 
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
     Hannoversche Str. 2, 31134 Hildesheim, Germany
   Phone: +49-5121-206917-0 |  Fax: +49-5121-206917-9

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

Wolfgang, thank you for your reply, let me try to explain myself a bit clearer:

On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk <wd@denx.de> wrote:
> In message <f608b67d0807231039i434e96dbvda86590776db2cb0@mail.gmail.com> you wrote:
>
>> some companies). If these added modules were not written in position
>> independent manner (namely, using structures with multiple stage
>> indirect pointers interleaved with data), the effort to make these
>> modules work in u-boot is very exhausting.
>
> I don't understand what you mean. Either you link statically with the
> U-Boot image, or you use standalone programs. In both situations no
> such problem as described by you exists.
>

we talk here about modules statically linked into the u-boot image.
Allow me to illustrate the problem I am trying to solve. Consider
adding this code to a u-boot source file on a ppc460gt platform:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
87a88,105
>
> int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);
>
> int  (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
>
> int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
> {
>       printf ("pointer is %p\n", pf);
>       printf ("function is %p\n", do_ptrt);
>       return 0;
> }
>
> U_BOOT_CMD(
>       ptrt, CFG_MAXARGS, 1,   do_ptrt,
>       "ptrt\n",
>       ""
> );
>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

And this is what happens when this command is invoked:

vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
=> ptrt
pointer is fffb2754
function is 0ffb7754
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So, the value of 'pf' is equal to the address of do_ptrt() *before*
relocation. The fact that there is a GOT and a sophisticated linker
script did not prevent this from happening.

>> Here is the suggested alternative. I will talk about recent PPC
>> platforms, but it could be done very similar for most of the other
>> platforms I believe.
>
> I don't think so.
>

Admittedly I don't have experience with nearly as many different
platforms as you do, but I don't see why not, is there a specific
fundamental reason on any of the platforms? But even if this does not
work on some, it will work on most.

>> In general the PPC u-boot image consists of three parts: the bulk of
>
> No, this is by far not general, not even for PPC.
>

I used that particular layout to emphasize that care needs to be taken
to remove the fixed address sections (the last 4k memory page in this
case) from the set of modules to link. On other architectures this
would be a different part of the image linked to the reset vector, the
idea remains the same.

>> Then we compare the two images and identify all locations which are
>> different. All these locations are 4 byte aligned, and all differ by
>> the same amount - equal TEXT_BASE. Now we can build a table of offsets
>> which need to be changed when the program is moved into a different
>> location, include this table in the resulting image between the bulk
>> and the startup page, and use this table when relocating code.
>
> Why would we do that? The compiler already generates such a table for
> free - the GOT.
>

well, didn't the example above show that the compiler will not do that
to an arbitrary pointer to a text object?

>> Basically, this implements a poor man's loader, and it will be
>> sufficient in case the image we are dealing with is a flat binary
>> (which is of course the case with u-boot). This will allow to add any
>> code to u-boot without much worries about being able to relocate it
>> properly. We would do away with storing certain structures in certain
>> sections, dealing with GOT, etc.
>
> I don't see how your "poor man's" GOT would be any better than the
> compiler generated one - except that it's horribly non-standard and
> glumpsy?
>

Quite frankly, GOT is also clumsy and even ugly IMHO - it requires a
lot of special treatment, explicit data definitions and linker
scripts, and does not work for arbitrary data as we saw above.

>> I made some experiments, and this seems feasible, this could be done
>> as a local customization, but I would much prefer to release it to the
>> u-boot community and make it part of mainline - will you consider such
>> a patch?
>
> I don't understand which problem you're trying to fix, and I don't see
> that your solution would be any better than what we have now.
>

I beg to differ, the solution I am suggesting will allow to stop
worrying about placing data in special sections, and will allow to
link in third party  modules which use cross segment pointers.

[snip]
>>
>> because the  modules are now part of u-boot, they need to be relocated
>> along with the rest of the image.
>
> U-Boot does this (if statically linked) the very same way like all
> other U-Boot code gets relocated.
>

not for pointers to functions as we saw above, right?

>> > Why isn't it written in position independent manner?
>>
>> because they come from different sources, not concerned with u-boot,
>> and just work and need to be ported as is.
>
> If you want to link against U-Boot, you have to use at least
> compatible compiler options and the U-Boot linker script.
>

well, the example above was inserted in one of u-boot source files, so
it was compiled exactly the way other u-boot modules were compiled.

>> > How is your relocation methodology going to fixup a module's data
>> > structures?
>>
>> The relocation will adjust only pointers to absolute addresses stored
>> in text/data segments.
>
> No.
>

But I think it will and I can demonstrate that.

>> > What pointers are in your data structures?  Why do they need to be
>> > relocated?
>>
>> in some cases data structures include pointers to other data
>> structures, etc. (This is called  'tree' btw, sorry couldn't resist
>> :-)
>
> This should get sorted out correctly when linking.
>

but it does not.

>> > Gcc supports "proper" relocation, if only we knew how to make it work for
>> > all "reasonable" versions of gcc (and Grant's conclusion is that many
>> > versions of gcc in use today do *not* support the relocation).
>> >  <http://article.gmane.org/gmane.comp.boot-loaders.u-boot/36343/>
>>
>> well, if there is a pointer to a data element or to a function stored
>> in a data structure, the .bin image is built and fixed using actual
>> addresses set by the linker. How can gcc help here?
>
> By adding the respective entries to the GOT.
>

and this is what I am trying to avoid, because in my case I need to
link in large chunks of third party code which can't be massaged to
follow the u-boot conventions, and can't be used as downloadable
applications for some platform specific reasons.

>> I am sure it is doable, I am just trying to understand if this would
>> be accepted in general, because it can be done differently depending
>> if one needs to push it upstream or not.
>
> If you invest time in solving such problems, than your time willbe
> much better iinvested if you try to help solving the remaining issues
> with  Grant's code.
>
> What Grant suggests is the way to go. I do not think your approach has
> chances for mainline.
>

I am sorry to hear this, especially since what I suggest would be
completely compiler agnostic and would allow to avoid some of the
limitations one must follow today while adding stuff to u-boot.

cheers,
/vb


> Best regards,
>
> Wolfgang Denk
>
> --
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
> Nothing in progression can rest on its original plan. We may as  well
> think  of  rocking  a grown man in the cradle of an infant.
> - Edmund Burke
>

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

vb <vb@vsbe.com> wrote:
> > int  (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
> >
> > int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
> > {
> >       printf ("pointer is %p\n", pf);
> >       printf ("function is %p\n", do_ptrt);
> >       return 0;
> > }

Just do this instead:

int  (*pf)(struct cmd_tbl_s *, int, int, char *[]);

int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
	if (!pf)
		pf = do_ptrt;

        printf ("pointer is %p\n", pf);
        printf ("function is %p\n", do_ptrt);
        return 0;
}

IMO, it's best to avoid such pointers in the first place, especially
ones that are statically initialized.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Jerry Van Baren]

vb wrote:
> Wolfgang, thank you for your reply, let me try to explain myself a bit clearer:
> 
> On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk <wd@denx.de> wrote:
>> In message <f608b67d0807231039i434e96dbvda86590776db2cb0@mail.gmail.com> you wrote:
>>
>>> some companies). If these added modules were not written in position
>>> independent manner (namely, using structures with multiple stage
>>> indirect pointers interleaved with data), the effort to make these
>>> modules work in u-boot is very exhausting.
>> I don't understand what you mean. Either you link statically with the
>> U-Boot image, or you use standalone programs. In both situations no
>> such problem as described by you exists.
> 
> we talk here about modules statically linked into the u-boot image.
> Allow me to illustrate the problem I am trying to solve. Consider
> adding this code to a u-boot source file on a ppc460gt platform:
> 
> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
> 87a88,105
>> int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);
>>
>> int  (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
>>
>> int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
>> {
>>       printf ("pointer is %p\n", pf);
>>       printf ("function is %p\n", do_ptrt);
>>       return 0;
>> }
>>
>> U_BOOT_CMD(
>>       ptrt, CFG_MAXARGS, 1,   do_ptrt,
>>       "ptrt\n",
>>       ""
>> );
>>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> And this is what happens when this command is invoked:
> 
> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
> => ptrt
> pointer is fffb2754
> function is 0ffb7754
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> So, the value of 'pf' is equal to the address of do_ptrt() *before*
> relocation. The fact that there is a GOT and a sophisticated linker
> script did not prevent this from happening.

OK, now I'm curious: what happens if you make the pf() pointer constant? 
  This will (should) change it from an initialized variable in the .data 
section to a constant in the .rodata section.  I'm wondering if the 
.rodata section gets relocation information where the .data section doesn't.

int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);

const int (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;

int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
{
       printf ("pointer is %p\n", pf);
       printf ("function is %p\n", do_ptrt);
       return 0;
}

Thanks,
gvb

[U-Boot-Users] Changing u-boot relocation scheme

[Kenneth Johansson]

On Wed, 2008-07-23 at 23:45 -0700, vb wrote:
> Wolfgang, thank you for your reply, let me try to explain myself a bit clearer:
> 
> On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk <wd@denx.de> wrote:
> > In message <f608b67d0807231039i434e96dbvda86590776db2cb0@mail.gmail.com> you wrote:
> >
> >> some companies). If these added modules were not written in position
> >> independent manner (namely, using structures with multiple stage
> >> indirect pointers interleaved with data), the effort to make these
> >> modules work in u-boot is very exhausting.
> >
> > I don't understand what you mean. Either you link statically with the
> > U-Boot image, or you use standalone programs. In both situations no
> > such problem as described by you exists.
> >
> 
> we talk here about modules statically linked into the u-boot image.
> Allow me to illustrate the problem I am trying to solve. Consider
> adding this code to a u-boot source file on a ppc460gt platform:
> 
> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
> 87a88,105
> >
> > int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[]);
> >
> > int  (*pf)(struct cmd_tbl_s *, int, int, char *[]) = do_ptrt;
> >
> > int do_ptrt (cmd_tbl_t *cmdtp, int flag, int argc, char *argv[])
> > {
> >       printf ("pointer is %p\n", pf);
> >       printf ("function is %p\n", do_ptrt);
> >       return 0;
> > }
> >
> > U_BOOT_CMD(
> >       ptrt, CFG_MAXARGS, 1,   do_ptrt,
> >       "ptrt\n",
> >       ""
> > );
> >
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> And this is what happens when this command is invoked:
> 
> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
> => ptrt
> pointer is fffb2754
> function is 0ffb7754
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> So, the value of 'pf' is equal to the address of do_ptrt() *before*
> relocation. The fact that there is a GOT and a sophisticated linker
> script did not prevent this from happening.


This is due to a u-boot on powerpc not doing the right things when it is
relocating itself. The code to do the right things exist but is
currently disabled.  

What is missing right now is that to properly fix function pointers
stored in data section we have to compile with -mrelocatable flag to get
gcc to create a fixup table so we later at runtime can just go over that
table and correct all function pointers. 

This is currently not done as there was some problems with it that
people thought was due to toolchain issue. I'm not so sure it is, the
code to generate this fixup has been in gcc since mid 90's and has not
seen many changes and the linker should not have to know about it as
it's just a normal relocataion for it. My take is that some targets did
something special with trying to fixup some pointers themselves and fail
when it's done correctly.

this is the only issue I'm aware of regarding relocation and if you
enable this for your board all your problems goes away. 

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <f608b67d0807232345r482792b6lbc9bd6804f48e597@mail.gmail.com> you wrote:
> 
> So, the value of 'pf' is equal to the address of do_ptrt() *before*
> relocation. The fact that there is a GOT and a sophisticated linker
> script did not prevent this from happening.

Yes, statically initialized function pointers are known to be not
relocated at the moment; they need manual fixing after relocation.
Note that this is exactly what Grant's patch fixed.

> > Why would we do that? The compiler already generates such a table for
> > free - the GOT.
> 
> well, didn't the example above show that the compiler will not do that
> to an arbitrary pointer to a text object?

It just shows that we don't know (yet) how to handle this correctly on
a range of given toolchains.

> Quite frankly, GOT is also clumsy and even ugly IMHO - it requires a
> lot of special treatment, explicit data definitions and linker
> scripts, and does not work for arbitrary data as we saw above.

It is standard, documented by the ABI, and works fine if done
correctly. We just don't do it correctly yet.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The only person who always got his work done by Friday
                                                 was Robinson Crusoe.

[U-Boot-Users] Changing u-boot relocation scheme

[Kenneth Johansson]

On Thu, 2008-07-24 at 15:01 +0200, Wolfgang Denk wrote:
> In message <f608b67d0807232345r482792b6lbc9bd6804f48e597@mail.gmail.com> you wrote:
> > Quite frankly, GOT is also clumsy and even ugly IMHO - it requires a
> > lot of special treatment, explicit data definitions and linker
> > scripts, and does not work for arbitrary data as we saw above.
> 
> It is standard, documented by the ABI, and works fine if done
> correctly. We just don't do it correctly yet.
> 
> Best regards,
> 
> Wolfgang Denk
> 

Well the type of runtime relocation that we do in u-boot is not really
specified in any documentation. What is in the abi is assuming a dynamic
linker and a proper elf file.

Gcc can be nice and create a fixup table for us if we ask for it but
that is not something that exist in any abi document I ever read. That
also explains why when this reloaction code was initially done we ended
up with the extra fixup code that tries to catch a few pointer that the
GOT table is not containing, nobody know the fixup table even existed
and it sure as hell is not mentioned in the documentation you have to
read the gcc code to understand what the -mrelocatable option really do.

[U-Boot-Users] Changing u-boot relocation scheme

[Jon Loeliger]

vb wrote:

>> What Grant suggests is the way to go. I do not think your approach has
>> chances for mainline.
>>
> 
> I am sorry to hear this, especially since what I suggest would be
> completely compiler agnostic and would allow to avoid some of the
> limitations one must follow today while adding stuff to u-boot.
> 
> cheers,
> /vb

As an alternate approach, perhaps this is something
that might be explored in the framework of the so-called
"U-Boot V2" effort?  Specifically, if you can demonstrate
it working, that would hold much more weight.  You know,
"Show me the code."! :-)

jdl

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Thu, Jul 24, 2008 at 2:58 AM, Haavard Skinnemoen
<haavard.skinnemoen@atmel.com> wrote:
>
> IMO, it's best to avoid such pointers in the first place, especially
> ones that are statically initialized.
>
> Haavard
>

On Thu, Jul 24, 2008 at 5:23 AM, Jerry Van Baren <gerald.vanbaren@ge.com> wrote:
>
> OK, now I'm curious: what happens if you make the pf() pointer constant?
>  This will (should) change it from an initialized variable in the .data

Haavard, Jerry, your both suggestions are very right, but you are
missing an important point: I am talking about porting code to u-boot,
libraries with hundreds of files, written and working in other
environments - they can't be compiled in as is, and changing them is
prohibitively inefficient.

cheers,
/vb

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Thu, Jul 24, 2008 at 5:47 AM, Kenneth Johansson <kenneth@southpole.se> wrote:
>
> What is missing right now is that to properly fix function pointers
> stored in data section we have to compile with -mrelocatable flag to get
> gcc to create a fixup table so we later at runtime can just go over that
> table and correct all function pointers.
>
Interesting, this is exactly what I am talking about, but done by the
compiler, let me check this out.
Thanks for the tip!

cheers,
/vb

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Thu, Jul 24, 2008 at 6:45 AM, Jon Loeliger <jdl@freescale.com> wrote:
>
> As an alternate approach, perhaps this is something
> that might be explored in the framework of the so-called
> "U-Boot V2" effort?  Specifically, if you can demonstrate
> it working, that would hold much more weight.  You know,
> "Show me the code."! :-)
>
> jdl
>
now you are talking :-)! I will do something along these lines, either
the compiler option or this "poor man's locator" thing and show you -
u-boot definitely needs this enhancement.

cheers,
/vb

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

Kenneth Johansson <kenneth@southpole.se> wrote:
> you have to
> read the gcc code to understand what the -mrelocatable option really do.

Plus it's only defined for PowerPC. What do we do on the 11 other
architectures?

We could build u-boot as a shared library I guess, but that feels a bit
weird...

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Kenneth Johansson]

On Thu, 2008-07-24 at 18:57 +0200, Haavard Skinnemoen wrote:
> Kenneth Johansson <kenneth@southpole.se> wrote:
> > you have to
> > read the gcc code to understand what the -mrelocatable option really do.
> 
> Plus it's only defined for PowerPC. What do we do on the 11 other
> architectures?

Well other architectures works in different ways and thus has other
flags.

> We could build u-boot as a shared library I guess, but that feels a bit
> weird...

What do you mean by that ? u-boot is already compiled with the -fPIC
option. 

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Thu, Jul 24, 2008 at 9:57 AM, Haavard Skinnemoen
<haavard.skinnemoen@atmel.com> wrote:
> Kenneth Johansson <kenneth@southpole.se> wrote:
>> you have to
>> read the gcc code to understand what the -mrelocatable option really do.
>
> Plus it's only defined for PowerPC. What do we do on the 11 other
> architectures?
>
> We could build u-boot as a shared library I guess, but that feels a bit
> weird...
>

I ran a quick experiment with -mrelocate, compiled a module with it
and then objdump -D its contents, it seems to do exactly what I wanted
- stores addresses of pointers to adjust into the fixit section.

I guess the least intrusive alternative would be to compile the ported
code with this option, and then add a check to the u-boot function
dealing with GOT adjustments to see if the fixit section is nonempty.
If so - relocate .fixit in addition to GOT.

But as you pointed out, this would work on ppc only (with a 'good'
compiler), and still remains to be proven, I will get to it a bit
later.

cheers,
/vb

> Haavard
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> U-Boot-Users mailing list
> U-Boot-Users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/u-boot-users
>

[U-Boot-Users] Changing u-boot relocation scheme

[Kenneth Johansson]

On Thu, 2008-07-24 at 10:37 -0700, vb wrote:
> On Thu, Jul 24, 2008 at 9:57 AM, Haavard Skinnemoen
> <haavard.skinnemoen@atmel.com> wrote:
> > Kenneth Johansson <kenneth@southpole.se> wrote:
> >> you have to
> >> read the gcc code to understand what the -mrelocatable option really do.
> >
> > Plus it's only defined for PowerPC. What do we do on the 11 other
> > architectures?
> >
> > We could build u-boot as a shared library I guess, but that feels a bit
> > weird...
> >
> 
> I ran a quick experiment with -mrelocate, compiled a module with it
> and then objdump -D its contents, it seems to do exactly what I wanted
> - stores addresses of pointers to adjust into the fixit section.
> 
> I guess the least intrusive alternative would be to compile the ported
> code with this option, and then add a check to the u-boot function
> dealing with GOT adjustments to see if the fixit section is nonempty.
> If so - relocate .fixit in addition to GOT.
> 
> But as you pointed out, this would work on ppc only (with a 'good'
> compiler), and still remains to be proven, I will get to it a bit
> later.

I have run u-boot with everything compiled with -mrelocatable and normal
relocation fixup code removed so I know it works fine. So consider it
proven. But since it apparently is a problem in some situation the best
is probably to make it a configuration option. 

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Thu, Jul 24, 2008 at 11:09 AM, Kenneth Johansson
<kenneth@southpole.se> wrote:
>

>
> I have run u-boot with everything compiled with -mrelocatable and normal
> relocation fixup code removed so I know it works fine. So consider it
> proven. But since it apparently is a problem in some situation the best
> is probably to make it a configuration option.
>
>

From what I saw in my limited experiment, there is no code (as in CPU
instructions) generated by gcc, just a data section containing
offsets.

Maybe the linker is supposed to add some code to the startup file to
handle that section in case it is invoked with -mrelocatable (on't
know if it would recognize this an option) or detects .fixup section
in the .o files being linked. It is a mute point for embedded
development anyways.

I *think* the only thing missing is a simple function to scan the
.fixup table and fix the pointers after relocation.

cheers,
/vb

>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> U-Boot-Users mailing list
> U-Boot-Users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/u-boot-users
>

[U-Boot-Users] Changing u-boot relocation scheme

[Joakim Tjernlund]

> -----Original Message-----
> From: u-boot-users-bounces at lists.sourceforge.net [mailto:u-boot-users-bounces at lists.sourceforge.net]
> On Behalf Of vb
> Sent: den 24 juli 2008 20:26
> To: Kenneth Johansson
> Cc: u-boot-users at lists.sourceforge.net; Haavard Skinnemoen; Wolfgang Denk
> Subject: Re: [U-Boot-Users] Changing u-boot relocation scheme
> 
> On Thu, Jul 24, 2008 at 11:09 AM, Kenneth Johansson
> <kenneth@southpole.se> wrote:
> >
> 
> >
> > I have run u-boot with everything compiled with -mrelocatable and normal
> > relocation fixup code removed so I know it works fine. So consider it
> > proven. But since it apparently is a problem in some situation the best
> > is probably to make it a configuration option.
> >
> >
> 
> >From what I saw in my limited experiment, there is no code (as in CPU
> instructions) generated by gcc, just a data section containing
> offsets.
> 
> Maybe the linker is supposed to add some code to the startup file to
> handle that section in case it is invoked with -mrelocatable (on't
> know if it would recognize this an option) or detects .fixup section
> in the .o files being linked. It is a mute point for embedded
> development anyways.
> 
> I *think* the only thing missing is a simple function to scan the
> .fixup table and fix the pointers after relocation.

You don't need that much, just reapply Grants patch(mentioned early in the debate)
that were reverted because it did not work for everyone, probably due to some
other strange relocation scheme. I too have run u-boot with full .fixup applied
and it works. U-boot gets bigger, but that is most likely due to the old reloc code
is just nullified instead of removed.

     Jocke 

[U-Boot-Users] Changing u-boot relocation scheme

[Kenneth Johansson]

On Thu, 2008-07-24 at 11:26 -0700, vb wrote:
> On Thu, Jul 24, 2008 at 11:09 AM, Kenneth Johansson
> <kenneth@southpole.se> wrote:
> >
> 
> >
> > I have run u-boot with everything compiled with -mrelocatable and normal
> > relocation fixup code removed so I know it works fine. So consider it
> > proven. But since it apparently is a problem in some situation the best
> > is probably to make it a configuration option.
> >
> >
> 
> From what I saw in my limited experiment, there is no code (as in CPU
> instructions) generated by gcc, just a data section containing
> offsets.
> 
> Maybe the linker is supposed to add some code to the startup file to
> handle that section in case it is invoked with -mrelocatable (on't
> know if it would recognize this an option) or detects .fixup section
> in the .o files being linked. It is a mute point for embedded
> development anyways.

I think the code exist in libgcc __eabi_convert() but code already exist
in u-boot at least in several start.S files. 

> I *think* the only thing missing is a simple function to scan the
> .fixup table and fix the pointers after relocation.

The code should be there
find . -name "*.S" | xargs grep FIXUP_TABLE

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

Dear Kenneth,

in message <1216905458.27201.38.camel@localhost.localdomain> you wrote:
> 
> > It is standard, documented by the ABI, and works fine if done
> > correctly. We just don't do it correctly yet.
> 
> Well the type of runtime relocation that we do in u-boot is not really
> specified in any documentation. What is in the abi is assuming a dynamic
> linker and a proper elf file.
> 
> Gcc can be nice and create a fixup table for us if we ask for it but
> that is not something that exist in any abi document I ever read. That
> also explains why when this reloaction code was initially done we ended
> up with the extra fixup code that tries to catch a few pointer that the
> GOT table is not containing, nobody know the fixup table even existed
> and it sure as hell is not mentioned in the documentation you have to
> read the gcc code to understand what the -mrelocatable option really do.

Thanks for providing such a friendly explanation that the code I came
up is so crappy :-)

Anyway - let's fix it, rather than do something that is *completely*
non-standard.

Thanks.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Beware of the Turing Tar-pit in  which  everything  is  possible  but
nothing of interest is easy.

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <20080724185734.6d79c419@hskinnemo-gx745.norway.atmel.com> you wrote:
> 
> Plus it's only defined for PowerPC. What do we do on the 11 other
> architectures?

Fix them in the first place to do reloction at all?

> We could build u-boot as a shared library I guess, but that feels a bit
> weird...

Shared? Shared by what?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"To take a significant step forward, you must make a series of finite
improvements." - Donald J. Atwood, General Motors

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <1216922981.27201.66.camel@localhost.localdomain> you wrote:
> 
> I have run u-boot with everything compiled with -mrelocatable and normal
> relocation fixup code removed so I know it works fine. So consider it
> proven. But since it apparently is a problem in some situation the best
> is probably to make it a configuration option. 

Ahem. What exactly did you prove? What were your tests? Which
toolchains did you try? Did you apply Grant's patch or not?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"The question of whether a computer can think is no more  interesting
than the question of whether a submarine can swim"
                                                - Edsgar W.  Dijkstra

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Fri, 25 Jul 2008 06:28:16 +0200
Wolfgang Denk <wd@denx.de> wrote:

> In message <20080724185734.6d79c419@hskinnemo-gx745.norway.atmel.com> you wrote:
> > 
> > Plus it's only defined for PowerPC. What do we do on the 11 other
> > architectures?
> 
> Fix them in the first place to do reloction at all?

Er. How? The only thing this thread has come up with is a powerpc-only
-mrelocatable option which apparently gives the magic word to the
linker to have it emit dynamic relocations. We need similar options
for 11 other architectures before we can even think about starting to
utilize this in a generic manner.

> > We could build u-boot as a shared library I guess, but that feels a bit
> > weird...
> 
> Shared? Shared by what?

An ELF shared library has the dynamic relocations we need. So if we
build u-boot as an .so file, it should work in theory on most
architectures.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Thu, 24 Jul 2008 19:12:20 +0200
Kenneth Johansson <kenneth@southpole.se> wrote:

> > We could build u-boot as a shared library I guess, but that feels a bit
> > weird...  
> 
> What do you mean by that ? u-boot is already compiled with the -fPIC
> option. 

If that's sufficient, why would you need the -mrelocatable option on
PowerPC?

-fPIC makes the compiler generate position-independent code, which is
necessary but not sufficient in order to get a fully relocatable
binary. The second part of the puzzle is dynamic relocations. Shared
libraries have them, and apparently, PowerPC binaries have them if you
link with -mrelocatable.

Currently u-boot is compiled with -fPIC, so the result is almost fully
relocatable. We work around the lack of dynamic relocations by manually
fixing up a handful of pointers at runtime.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Fri, 2008-07-25 at 11:10 +0200, Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 06:28:16 +0200
> Wolfgang Denk <wd@denx.de> wrote:
> 
> > In message <20080724185734.6d79c419@hskinnemo-gx745.norway.atmel.com> you wrote:
> > > We could build u-boot as a shared library I guess, but that feels a bit
> > > weird...
> > 
> > Shared? Shared by what?
> 
> An ELF shared library has the dynamic relocations we need. So if we
> build u-boot as an .so file, it should work in theory on most
> architectures.
> 
> Haavard
> 

well the elf binary of u-boot obviously has everything we need
regardless of what options it was compiled with. If we had a full linker
at runtime we could just do a relink to whatever address we wanted. 

It sounds a bit easier to just loop over a list of pointers and change
the values than to implement a complete linker but maybe that is just
me.

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Fri, 25 Jul 2008 13:55:58 +0200
kenneth johansson <kenneth@southpole.se> wrote:

> > An ELF shared library has the dynamic relocations we need. So if we
> > build u-boot as an .so file, it should work in theory on most
> > architectures.
> 
> well the elf binary of u-boot obviously has everything we need
> regardless of what options it was compiled with. If we had a full linker
> at runtime we could just do a relink to whatever address we wanted. 

No we couldn't if we don't have any relocation information. Just as you
can't relink just any ELF binary to run at a different location after
it's been through a final link, no matter how "full" the linker is.

And if the ELF file wasn't compiled as position-independent code to
begin with, there's just no way it will ever work if it's loaded at a
different address than it was linked at. You can't go through and fix
up all the compiler-generated address calculations when you don't even
have a clue where they are!

So whether or not the u-boot ELF file has everything we need depends a
_lot_ on what options were used when building it.

> It sounds a bit easier to just loop over a list of pointers and change
> the values than to implement a complete linker but maybe that is just
> me.

The question remains how should that list of pointers be generated? One
possible answer is to let the linker do it (as dynamic relocations).

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Joakim Tjernlund]

> -----Original Message-----
> From: u-boot-users-bounces at lists.sourceforge.net [mailto:u-boot-users-bounces at lists.sourceforge.net]
> On Behalf Of Haavard Skinnemoen
> Sent: den 25 juli 2008 14:19
> To: kenneth johansson
> Cc: vb; u-boot-users at lists.sourceforge.net; Wolfgang Denk
> Subject: Re: [U-Boot-Users] Changing u-boot relocation scheme
> 
> On Fri, 25 Jul 2008 13:55:58 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
> 
> > > An ELF shared library has the dynamic relocations we need. So if we
> > > build u-boot as an .so file, it should work in theory on most
> > > architectures.
> >
> > well the elf binary of u-boot obviously has everything we need
> > regardless of what options it was compiled with. If we had a full linker
> > at runtime we could just do a relink to whatever address we wanted.
> 
> No we couldn't if we don't have any relocation information. Just as you
> can't relink just any ELF binary to run at a different location after
> it's been through a final link, no matter how "full" the linker is.
> 
> And if the ELF file wasn't compiled as position-independent code to
> begin with, there's just no way it will ever work if it's loaded at a
> different address than it was linked at. You can't go through and fix
> up all the compiler-generated address calculations when you don't even
> have a clue where they are!
> 
> So whether or not the u-boot ELF file has everything we need depends a
> _lot_ on what options were used when building it.

Exactly.

> 
> > It sounds a bit easier to just loop over a list of pointers and change
> > the values than to implement a complete linker but maybe that is just
> > me.
> 
> The question remains how should that list of pointers be generated? One
> possible answer is to let the linker do it (as dynamic relocations).

I think the best bet is to make -mrelocatable work for PPC. Question is how
does the other arches do it? Do they include fixup ptrs by default or do
they lack this functionality?

  Jocke
> 
> Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Fri, 2008-07-25 at 14:19 +0200, Haavard Skinnemoen wrote: 
> On Fri, 25 Jul 2008 13:55:58 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
> 
> > > An ELF shared library has the dynamic relocations we need. So if we
> > > build u-boot as an .so file, it should work in theory on most
> > > architectures.
> > 
> > well the elf binary of u-boot obviously has everything we need
> > regardless of what options it was compiled with. If we had a full linker
> > at runtime we could just do a relink to whatever address we wanted. 
> 
> No we couldn't if we don't have any relocation information. Just as you
> can't relink just any ELF binary to run at a different location after
> it's been through a final link, no matter how "full" the linker is.

Take time and READ what people write. I wrote compiler option and you go
on about some final link that removes the relocation information. My
point was that it is irrelevant if you use pic shared whatever if you
are going to use the elf relocation information anyway granted you have
less work to do if the object was compiled as PIC. 


> So whether or not the u-boot ELF file has everything we need depends a
> _lot_ on what options were used when building it.

Not really only what info you stripped way in the last linker stage. But
this is going way off topic nobody is really arguing to include a linker
in u-boot so it does not matter what options is available if we did. 

> > It sounds a bit easier to just loop over a list of pointers and change
> > the values than to implement a complete linker but maybe that is just
> > me.
> 
> The question remains how should that list of pointers be generated? One
> possible answer is to let the linker do it (as dynamic relocations).

Since I have not done a linker I probably miss some information on what
makes the dynamic relocations so special ?? 

And could you outline how the last step in generating the binary image
would work. 

now it works as follows. One final static link with all the .a files and
a specified start address for TEXT. result is a elf file with al symbols
resolved adn it can be relinked to another address if one wants but we
use objcopy to make a binary.  

here is a patch to generate dynamic relocations in the elf file. What is
the next step? objcopy -j .rela.dyn -O binary  u-boot dyn_reloc_table ??

----------------
--- config.mk
+++ config.mk
@@ -215,7 +215,8 @@
 
 AFLAGS := $(AFLAGS_DEBUG) -D__ASSEMBLY__ $(CPPFLAGS)
 
-LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
+#LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
+LDFLAGS += -Bstatic -shared -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
 ifneq ($(TEXT_BASE),)
 LDFLAGS += -Ttext $(TEXT_BASE)
 endif

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Fri, Jul 25, 2008 at 7:33 AM, kenneth johansson <kenneth@southpole.se> wrote:

>
> here is a patch to generate dynamic relocations in the elf file. What is
> the next step? objcopy -j .rela.dyn -O binary  u-boot dyn_reloc_table ??
>
> ----------------
> --- config.mk
> +++ config.mk
> @@ -215,7 +215,8 @@
>
>  AFLAGS := $(AFLAGS_DEBUG) -D__ASSEMBLY__ $(CPPFLAGS)
>
> -LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
> +#LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
> +LDFLAGS += -Bstatic -shared -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
>  ifneq ($(TEXT_BASE),)
>  LDFLAGS += -Ttext $(TEXT_BASE)
>  endif
>

Interesting, but after this step we have a file which can't be
converted to a runnable binary without resolving the relocation
information first, right? This is not good then, as we need something
to run from reset.

I think this discussion covers much more ground than needed: certainly
the problem of running the same executable at different addresses has
been solved many times over, for different architectures and file
formats. The thing is that unless we want the loader be part of u-boot
we can not use those standard solutions. And I don't think we want,
because it would be an unnecessary complication.

I still believe that my original suggestion of determining the
addresses of pointers to relocate by comparing to stripped binaries is
the simplest way to do it across architectures. Apparently the same
could be achieved by using -mrelocatable on powerPc, this still
remains to  be seen.

cheers,
/vb

>
>

[U-Boot-Users] Changing u-boot relocation scheme

[Jerry Van Baren]

vb wrote:
> On Fri, Jul 25, 2008 at 7:33 AM, kenneth johansson <kenneth@southpole.se> wrote:
> 
>> here is a patch to generate dynamic relocations in the elf file. What is
>> the next step? objcopy -j .rela.dyn -O binary  u-boot dyn_reloc_table ??
>>
>> ----------------
>> --- config.mk
>> +++ config.mk
>> @@ -215,7 +215,8 @@
>>
>>  AFLAGS := $(AFLAGS_DEBUG) -D__ASSEMBLY__ $(CPPFLAGS)
>>
>> -LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
>> +#LDFLAGS += -Bstatic -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
>> +LDFLAGS += -Bstatic -shared -T $(LDSCRIPT) $(PLATFORM_LDFLAGS)
>>  ifneq ($(TEXT_BASE),)
>>  LDFLAGS += -Ttext $(TEXT_BASE)
>>  endif
>>
> 
> Interesting, but after this step we have a file which can't be
> converted to a runnable binary without resolving the relocation
> information first, right? This is not good then, as we need something
> to run from reset.
> 
> I think this discussion covers much more ground than needed: certainly
> the problem of running the same executable at different addresses has
> been solved many times over, for different architectures and file
> formats. The thing is that unless we want the loader be part of u-boot
> we can not use those standard solutions. And I don't think we want,
> because it would be an unnecessary complication.
> 
> I still believe that my original suggestion of determining the
> addresses of pointers to relocate by comparing to stripped binaries is
> the simplest way to do it across architectures. Apparently the same
> could be achieved by using -mrelocatable on powerPc, this still
> remains to  be seen.
> 
> cheers,
> /vb

Well, to circle back around, Grant Likely *DID* ELF-based relocation 
*SUCCESSFULLY* so it *CAN* be done the right way.  He just didn't have a 
full recipe that worked over all "popular" compiler versions, didn't 
have the time to identify /why/ it didn't work on certain compiler 
versions, and didn't try to run on architectures that he didn't have 
physical access to.

Doing the relocation the right way will be much more future-proof than 
doing it a different wrong way.

The relocation information is in the ELF file until and unless we remove 
it.  "Normal" ELF executables retain that relocation information... that 
is exactly what the "L" (it) is for.  The linux loader (elf loader) 
loads an executable into an arbitrary memory location and relocates it 
by fixing up addresses based on the relocation table.

To reiterate, Grant had a proof-of-concept (well beyond p-o-c, actually) 
that did everything you are asking for, but the right way and even in 
the same architecture that you are interested in.  Despite the chest 
beating that has gone on in this thread about supporting other 
architectures, I'm 100% positive that they are also supported with fixup 
tables, if only we knew enough to use them.

You can stand on the *shoulders* of giants or their toes.

Best regards,
gvb

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Fri, 25 Jul 2008 16:33:56 +0200
kenneth johansson <kenneth@southpole.se> wrote:

> On Fri, 2008-07-25 at 14:19 +0200, Haavard Skinnemoen wrote: 
> > On Fri, 25 Jul 2008 13:55:58 +0200
> > kenneth johansson <kenneth@southpole.se> wrote:
> > 
> > > > An ELF shared library has the dynamic relocations we need. So if we
> > > > build u-boot as an .so file, it should work in theory on most
> > > > architectures.
> > > 
> > > well the elf binary of u-boot obviously has everything we need
> > > regardless of what options it was compiled with. If we had a full linker
> > > at runtime we could just do a relink to whatever address we wanted. 
> > 
> > No we couldn't if we don't have any relocation information. Just as you
> > can't relink just any ELF binary to run at a different location after
> > it's been through a final link, no matter how "full" the linker is.
> 
> Take time and READ what people write. I wrote compiler option and you go
> on about some final link that removes the relocation information. My
> point was that it is irrelevant if you use pic shared whatever if you
> are going to use the elf relocation information anyway granted you have
> less work to do if the object was compiled as PIC. 

Oh, you're talking about some hypothetical u-boot binary that hasn't
been through the linker? How exactly do you generate it?

Besides, I talked about compiler options too (in the paragraph you cut
out). If you don't compile the code with -fPIC, most linkers won't be
able to make the result relocatable no matter what options you specify.

> > > It sounds a bit easier to just loop over a list of pointers and change
> > > the values than to implement a complete linker but maybe that is just
> > > me.
> > 
> > The question remains how should that list of pointers be generated? One
> > possible answer is to let the linker do it (as dynamic relocations).
> 
> Since I have not done a linker I probably miss some information on what
> makes the dynamic relocations so special ?? 

Yes, you probably do. Dynamic relocations are quite special as they are
intended for the _runtime_ system, not the linker. Therefore, they are
included in a loadable segment so that the program itself or the
dynamic loader can go through them and perform fixups at runtime.

Also, most platforms only allow a small handful of relocation types to
be used as dynamic relocations. This makes the job of the dynamic
loader much easier than that of the linker.

> And could you outline how the last step in generating the binary image
> would work. 

That's basically the question I've been trying to ask for some time. On
PowerPC, I assume -mrelocatable does the trick. On other platforms, I
just don't know.

> now it works as follows. One final static link with all the .a files and
> a specified start address for TEXT. result is a elf file with al symbols
> resolved adn it can be relinked to another address if one wants but we
> use objcopy to make a binary.  

Have you ever _tried_ to relink the final u-boot ELF file to another
address? How do you do that?

> here is a patch to generate dynamic relocations in the elf file. What is
> the next step? objcopy -j .rela.dyn -O binary  u-boot dyn_reloc_table ??

Might actually work, though we might need more linker options as well.
At least -Bsymbolic comes to mind. And I'm not sure if -Bstatic goes
well along with -shared...

In any case, there's no next step. The dynamic relocations are included
in a loadable segment, so they will end up in the .bin file after the
last objcopy step.

There will obviously be a fair amount of arch-specific code required to
make the actual relocation work though.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Fri, 2008-07-25 at 17:23 +0200, Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 16:33:56 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
> > now it works as follows. One final static link with all the .a files and
> > a specified start address for TEXT. result is a elf file with al symbols
> > resolved adn it can be relinked to another address if one wants but we
> > use objcopy to make a binary.  
> 
> Have you ever _tried_ to relink the final u-boot ELF file to another
> address? How do you do that?

I have before but I just did a quick test to make sure I'm not spreading
miss information. 

ld -Ttext 0x02000000 -Map u-boot_2.map -o u-boot_2 u-boot

at least creates a map file that looks ok for a u-boot compiled both
with -fpic and without. Did not actually run the result. 

> > here is a patch to generate dynamic relocations in the elf file. What is
> > the next step? objcopy -j .rela.dyn -O binary  u-boot dyn_reloc_table ??
> 
> Might actually work, though we might need more linker options as well.
> At least -Bsymbolic comes to mind. And I'm not sure if -Bstatic goes
> well along with -shared...
> 
> In any case, there's no next step. The dynamic relocations are included
> in a loadable segment, so they will end up in the .bin file after the
> last objcopy step.

hmm just checked what objcopy does and it do not work well with a object
file created with shared. the resulting binary do not have .text first
so it's not really workable at least not with that simple patch. 

cant find any option to make objcopy print out what section it actually
writes out in the out file. 

> There will obviously be a fair amount of arch-specific code required to
> make the actual relocation work though.

So the gain of using dynamic relocation is that we have fever relocation
types to implement and objcopy is not going to strip away the relocation
section as it do for non dynamic relocs. 

still the code to actually do the relocation is architecture dependent
and non existing. At least for powerpc we have a much easier option of
having gcc generate a fixup table. 

[U-Boot-Users] Changing u-boot relocation scheme

[J. William Campbell]

Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 16:33:56 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
>
>   
>> On Fri, 2008-07-25 at 14:19 +0200, Haavard Skinnemoen wrote: 
>>     
>>> On Fri, 25 Jul 2008 13:55:58 +0200
>>> kenneth johansson <kenneth@southpole.se> wrote:
>>>
>>>       
>>>>> An ELF shared library has the dynamic relocations we need. So if we
>>>>> build u-boot as an .so file, it should work in theory on most
>>>>> architectures.
>>>>>           
>>>> well the elf binary of u-boot obviously has everything we need
>>>> regardless of what options it was compiled with. If we had a full linker
>>>> at runtime we could just do a relink to whatever address we wanted. 
>>>>         
>>> No we couldn't if we don't have any relocation information. Just as you
>>> can't relink just any ELF binary to run at a different location after
>>> it's been through a final link, no matter how "full" the linker is.
>>>       
>> Take time and READ what people write. I wrote compiler option and you go
>> on about some final link that removes the relocation information. My
>> point was that it is irrelevant if you use pic shared whatever if you
>> are going to use the elf relocation information anyway granted you have
>> less work to do if the object was compiled as PIC. 
>>     
>
> Oh, you're talking about some hypothetical u-boot binary that hasn't
> been through the linker? How exactly do you generate it?
>   
No, he is talking about the u-boot ELF file before we strip off the 
relocation information in converting it to a .bin file. All ELF files 
contain enough relocation information to relocate the binary. If there 
are undefined symbols, it my not work, but it can be relocated.
> Besides, I talked about compiler options too (in the paragraph you cut
> out). If you don't compile the code with -fPIC, most linkers won't be
> able to make the result relocatable no matter what options you specify.
>   
The above statement is not correct. -fPIC is a compiler option 
requesting the compiler generate Position Independent Code. This 
involves adding code to relocate all references at execution time, not 
at load time. This makes the binary larger and slower in exchange for it 
running anywhere in the address space. The problem remaining with -fPIC 
on the PPC architecture is that certain operations still must be 
performed at load time. The GOT is part of the structure required to do 
this. It may be that we do not completely know all the things that the 
compiler expects to be done at load time, or it may be that with Grant's 
work and the correct compiler switches (-mrelocateable) we may know for 
the PPC. The C compiler produces relocatable code by default, but not 
PIC. It is true that the Linux loader can relocate ELF code "on the fly" 
as it loads into memory, PIC or not.
>   
>>>> It sounds a bit easier to just loop over a list of pointers and change
>>>> the values than to implement a complete linker but maybe that is just
>>>> me.
>>>>         
>>> The question remains how should that list of pointers be generated? One
>>> possible answer is to let the linker do it (as dynamic relocations).
>>>       
>> Since I have not done a linker I probably miss some information on what
>> makes the dynamic relocations so special ?? 
>>     
>
> Yes, you probably do. Dynamic relocations are quite special as they are
> intended for the _runtime_ system, not the linker. Therefore, they are
> included in a loadable segment so that the program itself or the
> dynamic loader can go through them and perform fixups at runtime.
>
> Also, most platforms only allow a small handful of relocation types to
> be used as dynamic relocations. This makes the job of the dynamic
> loader much easier than that of the linker.
>   
This is correct, the loader allows only "simple" relocations. Basically, 
the loader is doing the relocations done by the GOT if the code is PIC. 
The loader is not real complex. The problem is that for u-boot, the 
loader would need to run either at a fixed location or be itself 
relocated, which is a chicken and egg problem.

There is a problem with assuming that -fPIC is the way to go on all 
architectures. First, on some computers, such as IA32 for examples, it 
is just not practical. There are too few registers to dedicate one 
register to always containing a relocation value to be added to all 
references dynamically. There are other examples where the concept of 
PIC is equally problematic for various reasons. Second, since PIC is not 
required for "normal" Linux applications, the compiler is often "buggy" 
when PIC is specified. Further, PIC is often the last feature that is 
totally debugged on new architectures, so even if it will eventually 
work, it often won't do so to begin with.

Therefore, in some cases, the only approach that will work for a 
completely relocatable u-boot would be a small loader running in 
pre-assigned fixed memory locations that relocates u-boot to the desired 
address using the ELF relocation data. The GOT would not be available.
>   
>> And could you outline how the last step in generating the binary image
>> would work. 
>>     
>
> That's basically the question I've been trying to ask for some time. On
> PowerPC, I assume -mrelocatable does the trick. On other platforms, I
> just don't know.
>
>   
>> now it works as follows. One final static link with all the .a files and
>> a specified start address for TEXT. result is a elf file with al symbols
>> resolved adn it can be relinked to another address if one wants but we
>> use objcopy to make a binary.  
>>     
>
> Have you ever _tried_ to relink the final u-boot ELF file to another
> address? How do you do that?
>
>   
>> here is a patch to generate dynamic relocations in the elf file. What is
>> the next step? objcopy -j .rela.dyn -O binary  u-boot dyn_reloc_table ??
>>     
>
> Might actually work, though we might need more linker options as well.
> At least -Bsymbolic comes to mind. And I'm not sure if -Bstatic goes
> well along with -shared...
>
> In any case, there's no next step. The dynamic relocations are included
> in a loadable segment, so they will end up in the .bin file after the
> last objcopy step.
>
> There will obviously be a fair amount of arch-specific code required to
> make the actual relocation work though.
>   
Using the .so approach is lots harder (IMHO) than just a simple loader 
that moves u-boot and relocates it using ELF data. However, whether the 
ability to relocate u-boot to an arbitrary address is worth the effort 
is probably in the eye of the beholder. I think that there are more 
architectures that do not support relocation than actually do support 
relocation these days. Anybody that uses some kind of pre-loader, like 
Xilinx or Blackfin, probably has to know the memory size to begin with, 
so it's not an issue. YMMV

Best Regards,
Bill Campbell
> Haavard
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> U-Boot-Users mailing list
> U-Boot-Users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/u-boot-users
>
>
>   

[U-Boot-Users] Changing u-boot relocation scheme

[Jerry Van Baren]

kenneth johansson wrote:
> On Fri, 2008-07-25 at 17:23 +0200, Haavard Skinnemoen wrote:

[snip]

>> There will obviously be a fair amount of arch-specific code required to
>> make the actual relocation work though.
> 
> So the gain of using dynamic relocation is that we have fever relocation
> types to implement and objcopy is not going to strip away the relocation
> section as it do for non dynamic relocs. 
> 
> still the code to actually do the relocation is architecture dependent
> and non existing. At least for powerpc we have a much easier option of
> having gcc generate a fixup table. 

EVERYTHING *already* exists, we just aren't using it right.  That is 
what the elf loader does in a "normal" system.  Grant did it on the 
PowerPC architecture - look at the archives, he didn't write new code 
for this, it's already in there.

Here is the reversion back to the old way:
   <http://article.gmane.org/gmane.comp.boot-loaders.u-boot/33297>
If you un-revert it (re-revert it?), you will have what Grant did.  For 
some set of PowerPC platforms, it Just Works[tm].  For other 
architectures and platforms, start from this and make it work.

Note the list of changed files - no source file changes in it:
  board/cogent/u-boot.lds             |    1 +
  board/hymod/u-boot.lds              |    1 +
  board/mousse/u-boot.lds             |    1 +
  board/rsdproto/u-boot.lds           |    1 +
  cpu/mpc512x/config.mk               |    2 +-
  cpu/mpc5xx/config.mk                |    2 +-
  cpu/mpc5xx/u-boot.lds               |    1 +
  cpu/mpc5xxx/config.mk               |    2 +-
  cpu/mpc5xxx/u-boot-customlayout.lds |    1 +
  cpu/mpc5xxx/u-boot.lds              |    1 +
  cpu/mpc8220/config.mk               |    2 +-
  cpu/mpc8220/u-boot.lds              |    1 +
  cpu/mpc824x/config.mk               |    2 +-
  cpu/mpc824x/u-boot.lds              |    1 +
  cpu/mpc8260/config.mk               |    2 +-
  cpu/mpc8260/u-boot.lds              |    1 +
  cpu/mpc83xx/config.mk               |    2 +-
  cpu/mpc83xx/u-boot.lds              |    1 +
  include/common.h                    |    8 --------
  19 files changed, 18 insertions(+), 15 deletions(-)

Best regards,
gvb

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Fri, 2008-07-25 at 13:02 -0400, Jerry Van Baren wrote:
> kenneth johansson wrote:
> > On Fri, 2008-07-25 at 17:23 +0200, Haavard Skinnemoen wrote:
> 
> [snip]
> 
> >> There will obviously be a fair amount of arch-specific code required to
> >> make the actual relocation work though.
> > 
> > So the gain of using dynamic relocation is that we have fever relocation
> > types to implement and objcopy is not going to strip away the relocation
> > section as it do for non dynamic relocs. 
> > 
> > still the code to actually do the relocation is architecture dependent
> > and non existing. At least for powerpc we have a much easier option of
> > having gcc generate a fixup table. 
> 
> EVERYTHING *already* exists, we just aren't using it right.  That is 
> what the elf loader does in a "normal" system.  Grant did it on the 
> PowerPC architecture - look at the archives, he didn't write new code 
> for this, it's already in there.

I know exactly what Grant did. But he is not using the elf relocation
information as that is simply not included in the data that is the
u-boot.bin file. 

Also this discussion is more about doing it in a way that is not powerpc
specific and documented, we already have as you mentioned code for the
powerpc specific way but it's not using elf relocation information.

I was afraid that what was needed was more or less a complete linker but
it looks like if one generate the dynamic reloc table a much simpler
linker(relocation function) is needed. Still probably a lot more complex
than the GOT and fixup code that is just a loop over a table. We may
need a case also ;-) 

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Fri, 25 Jul 2008 19:28:48 +0200
kenneth johansson <kenneth@southpole.se> wrote:

> I was afraid that what was needed was more or less a complete linker but
> it looks like if one generate the dynamic reloc table a much simpler
> linker(relocation function) is needed. Still probably a lot more complex
> than the GOT and fixup code that is just a loop over a table. We may
> need a case also ;-) 

Hmm...looks like linking with -pie (or --pic-executable) does
something vaguely resembling the trick. But I don't know what vintage
of ld you need for this to be available and actually work...

hskinnemoen at siona:~/work/u-boot/master$ avr32-linux-readelf -d u-boot

Dynamic section at offset 0x1b8f0 contains 12 entries:
  Tag        Type                         Name/Value
 0x00000004 (HASH)                       0x182ac
 0x00000005 (STRTAB)                     0x18260
 0x00000006 (SYMTAB)                     0x18150
 0x0000000a (STRSZ)                      74 (bytes)
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000003 (PLTGOT)                     0x1ae70
 0x70000001 (Processor Specific: 70000001) 0xe6c
 0x00000015 (DEBUG)                      0x0
 0x00000007 (RELA)                       0x1833c
 0x00000008 (RELASZ)                     6996 (bytes)
 0x00000009 (RELAENT)                    12 (bytes)
 0x00000000 (NULL)                       0x0

The dynamic relocation section became surprisingly large. I wonder if
we actually manage to manually relocate all those buggers...I checked a
couple of them and in both cases it looked like something that needed
to be relocated, but I couldn't find any code that actually did it.

One example from common/cmd_itest.c:

op_tbl_t op_table [] = {
        { "-lt", LT },
        { "<"  , LT },
        { "-gt", GT },
        { ">"  , GT },
        { "-eq", EQ },
        { "==" , EQ },
        { "-ne", NE },
        { "!=" , NE },
        { "<>" , NE },
        { "-ge", GE },
        { ">=" , GE },
        { "-le", LE },
        { "<=" , LE },
};

Does this really work without any relocation? Or is it being relocated
and I'm just too blind to see it?

The symbol tables can probably be made smaller with some careful
ldscript tweaking.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Fri, 25 Jul 2008 11:21:12 -0400
Jerry Van Baren <gerald.vanbaren@ge.com> wrote:

> The relocation information is in the ELF file until and unless we remove 
> it.  "Normal" ELF executables retain that relocation information... that 
> is exactly what the "L" (it) is for.  The linux loader (elf loader) 
> loads an executable into an arbitrary memory location and relocates it 
> by fixing up addresses based on the relocation table.

You're mixing two different relocation tables. Statically linked
executables (like u-boot) normally don't have any relocation
information in them after they are fully linked. Dynamically linked
executables, position-independent executables (PIE) and shared
libraries have dynamic relocations which are not only in the ELF file,
but in a loadable segment so that they can be accessed at run time.

The relocation information from the .o files are not preserved in the
final executable unless you specify the --emit-relocs flags, which
exists purely for debugging purposes. The relocations used by the
dynamic loader come from an entirely different, simpler set of
relocation types which are normally not found in .o files.

Why do you think objdump has two different options for dumping normal
(-r) vs. dynamic (-R) relocations?

And no, Linux ELF executables can _not_ be loaded into an arbitrary
memory location unless they are PIE. Shared libraries, however, can be
loaded at arbitrary memory locations, though things like pre-linking
might make it more optimal to try to place them at the same address
everywhere.

Ok, I'll stop the chest-beating now. But please stop trying to tell
people that adding a powerpc-specific option (which nobody seems to
know how really works) to the command line will work on any other
architectures than powerpc.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Jerry Van Baren]

Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 11:21:12 -0400
> Jerry Van Baren <gerald.vanbaren@ge.com> wrote:
> 
>> The relocation information is in the ELF file until and unless we remove 
>> it.  "Normal" ELF executables retain that relocation information... that 
>> is exactly what the "L" (it) is for.  The linux loader (elf loader) 
>> loads an executable into an arbitrary memory location and relocates it 
>> by fixing up addresses based on the relocation table.
> 
> You're mixing two different relocation tables. Statically linked
> executables (like u-boot) normally don't have any relocation
> information in them after they are fully linked. Dynamically linked
> executables, position-independent executables (PIE) and shared
> libraries have dynamic relocations which are not only in the ELF file,
> but in a loadable segment so that they can be accessed at run time.
> 
> The relocation information from the .o files are not preserved in the
> final executable unless you specify the --emit-relocs flags, which
> exists purely for debugging purposes. The relocations used by the
> dynamic loader come from an entirely different, simpler set of
> relocation types which are normally not found in .o files.
> 
> Why do you think objdump has two different options for dumping normal
> (-r) vs. dynamic (-R) relocations?
> 
> And no, Linux ELF executables can _not_ be loaded into an arbitrary
> memory location unless they are PIE. Shared libraries, however, can be
> loaded at arbitrary memory locations, though things like pre-linking
> might make it more optimal to try to place them at the same address
> everywhere.

Thanks for the explanation.  I have not worked at this level with 
executable formats in a long time.  My experience harks back to Z80 
formats (I forgot which, but post-CPM which loaded into a fixed address) 
  and .COM and .EXE formats which have rudimentary relocation fixup 
information so that they could be run at arbitrary addresses.  Those are 
non-virtual (or didn't use the MMU) formats, equivalent to your PIE 
reference.

> Ok, I'll stop the chest-beating now. But please stop trying to tell
> people that adding a powerpc-specific option (which nobody seems to
> know how really works) to the command line will work on any other
> architectures than powerpc.

Fair enough.

> Haavard

Discovering my banner is labeled "obsolete",
gvb

[U-Boot-Users] Changing u-boot relocation scheme

[J. William Campbell]

Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 19:28:48 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
>
>   
>> I was afraid that what was needed was more or less a complete linker but
>> it looks like if one generate the dynamic reloc table a much simpler
>> linker(relocation function) is needed. Still probably a lot more complex
>> than the GOT and fixup code that is just a loop over a table. We may
>> need a case also ;-) 
>>     
>
>   
First of all, you are correct that "fully linked" executables contain no 
relocation information unless the user does something like 
--emit-relocs. I was wrong about that.
> Hmm...looks like linking with -pie (or --pic-executable) does
> something vaguely resembling the trick. But I don't know what vintage
> of ld you need for this to be available and actually work...
>
> hskinnemoen at siona:~/work/u-boot/master$ avr32-linux-readelf -d u-boot
>
> Dynamic section at offset 0x1b8f0 contains 12 entries:
>   Tag        Type                         Name/Value
>  0x00000004 (HASH)                       0x182ac
>  0x00000005 (STRTAB)                     0x18260
>  0x00000006 (SYMTAB)                     0x18150
>  0x0000000a (STRSZ)                      74 (bytes)
>  0x0000000b (SYMENT)                     16 (bytes)
>  0x00000003 (PLTGOT)                     0x1ae70
>  0x70000001 (Processor Specific: 70000001) 0xe6c
>  0x00000015 (DEBUG)                      0x0
>  0x00000007 (RELA)                       0x1833c
>  0x00000008 (RELASZ)                     6996 (bytes)
>  0x00000009 (RELAENT)                    12 (bytes)
>  0x00000000 (NULL)                       0x0
>
> The dynamic relocation section became surprisingly large. I wonder if
> we actually manage to manually relocate all those buggers...I checked a
> couple of them and in both cases it looked like something that needed
> to be relocated, but I couldn't find any code that actually did it.
>
> One example from common/cmd_itest.c:
>
> op_tbl_t op_table [] = {
>         { "-lt", LT },
>         { "<"  , LT },
>         { "-gt", GT },
>         { ">"  , GT },
>         { "-eq", EQ },
>         { "==" , EQ },
>         { "-ne", NE },
>         { "!=" , NE },
>         { "<>" , NE },
>         { "-ge", GE },
>         { ">=" , GE },
>         { "-le", LE },
>         { "<=" , LE },
> };
>
> Does this really work without any relocation? Or is it being relocated
> and I'm just too blind to see it?
>   
Is your u-boot built with -fPIC on all the C compiles? If so, op_table 
references may work without any actual relocation if the distance from 
the code segment to the segment containing op_table is not changed when 
u-boot is "relocated" to the new address. If you look at the assembly 
code for op_table being used, it may be done via an offset from the 
start of the code + a register pointing to the start of the code. The 
offset will not change when the code is moved, and the register will 
contain the correct start of code address, so it works. If it is not 
-fPIC, and it is not being executed at the address for which it was 
linked, I have no idea how it would work without a "loader" relocating 
all the references. There are "only" 583 relocation entries specified!

I think you are correct that the --pic-executable switch is the 
"generic" way to get the relocation addresses. I expect it is much 
better supported than the PPC specific compiler switches, because ld 
needs to have the relocation data on all platforms. The only additional 
requirement is for ld to actually output this data. It seems pretty 
likely that this is implemented on all platforms, because at least a 
subset of it is required for shared libraries and dynamically linked 
executables.

Best Regards,
Bill Campbell
> The symbol tables can probably be made smaller with some careful
> ldscript tweaking.
>
> Haavard
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> U-Boot-Users mailing list
> U-Boot-Users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/u-boot-users
>
>
>   

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Fri, 2008-07-25 at 20:35 +0200, Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 19:28:48 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
> 
> > I was afraid that what was needed was more or less a complete linker but
> > it looks like if one generate the dynamic reloc table a much simpler
> > linker(relocation function) is needed. Still probably a lot more complex
> > than the GOT and fixup code that is just a loop over a table. We may
> > need a case also ;-) 
> 
> Hmm...looks like linking with -pie (or --pic-executable) does
> something vaguely resembling the trick. But I don't know what vintage
> of ld you need for this to be available and actually work...

Can't see any reason for using this flag over -fPIC for a program like
u-boot.

> hskinnemoen at siona:~/work/u-boot/master$ avr32-linux-readelf -d u-boot
> 
> Dynamic section at offset 0x1b8f0 contains 12 entries:
>   Tag        Type                         Name/Value
>  0x00000004 (HASH)                       0x182ac
>  0x00000005 (STRTAB)                     0x18260
>  0x00000006 (SYMTAB)                     0x18150
>  0x0000000a (STRSZ)                      74 (bytes)
>  0x0000000b (SYMENT)                     16 (bytes)
>  0x00000003 (PLTGOT)                     0x1ae70
>  0x70000001 (Processor Specific: 70000001) 0xe6c
>  0x00000015 (DEBUG)                      0x0
>  0x00000007 (RELA)                       0x1833c
>  0x00000008 (RELASZ)                     6996 (bytes)
>  0x00000009 (RELAENT)                    12 (bytes)
>  0x00000000 (NULL)                       0x0
> 
> The dynamic relocation section became surprisingly large. I wonder if
> we actually manage to manually relocate all those buggers...I checked a
> couple of them and in both cases it looked like something that needed
> to be relocated, but I couldn't find any code that actually did it.

yes it's scary that it's working as well as it does. This is however
solved by reactivating Grant Likely's patch.

> One example from common/cmd_itest.c:
> 
> op_tbl_t op_table [] = {
>         { "-lt", LT },
>         { "<"  , LT },
>         { "-gt", GT },
>         { ">"  , GT },
>         { "-eq", EQ },
>         { "==" , EQ },
>         { "-ne", NE },
>         { "!=" , NE },
>         { "<>" , NE },
>         { "-ge", GE },
>         { ">=" , GE },
>         { "-le", LE },
>         { "<=" , LE },
> };
> 
> Does this really work without any relocation? Or is it being relocated
> and I'm just too blind to see it?

No you are right there is probably many places like this. the code just
use the old location and that works most of the time.

> The symbol tables can probably be made smaller with some careful
> ldscript tweaking.
> 
> Haavard
> 

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <f608b67d0807250751j137e70c0oe67affcf46399c8e@mail.gmail.com> you wrote:
> 
> I think this discussion covers much more ground than needed: certainly
> the problem of running the same executable at different addresses has
> been solved many times over, for different architectures and file
> formats. The thing is that unless we want the loader be part of u-boot
> we can not use those standard solutions. And I don't think we want,
> because it would be an unnecessary complication.

We do not need a "loader". All we need is "just" relocation working
completely and correctly. At the moment, it is now working completely.

> I still believe that my original suggestion of determining the
> addresses of pointers to relocate by comparing to stripped binaries is
> the simplest way to do it across architectures. Apparently the same

Maybe I should have been more explicit right from the beginning. I
repeat: forget it. This will *not* go into mainline. Zero chance.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
How many seconds are there in a year? If I tell you there are 3.155 x
10^7, you won't even try to remember it. On the other hand, who could
forget that, to within half a percent, pi seconds is  a  nanocentury.
                                               -- Tom Duff, Bell Labs

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <1217006928.7972.47.camel@duo> you wrote:
>
> I know exactly what Grant did. But he is not using the elf relocation
> information as that is simply not included in the data that is the
> u-boot.bin file. 

So what? Is this a problem? Why would we need to use XXX if we can
perform all we need to do by using YYY instead, especially if this is
simpler to implement?

> Also this discussion is more about doing it in a way that is not powerpc
> specific and documented, we already have as you mentioned code for the
> powerpc specific way but it's not using elf relocation information.

Who says that we MUST use the ELF reloc info?
 
Or why?

> I was afraid that what was needed was more or less a complete linker but
> it looks like if one generate the dynamic reloc table a much simpler
> linker(relocation function) is needed. Still probably a lot more complex
> than the GOT and fixup code that is just a loop over a table. We may
> need a case also ;-) 

Please re-read Grant's patch. There were NO changes to the existing
simple code!

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Murder is contrary to the laws of man and God.
	-- M-5 Computer, "The Ultimate Computer", stardate 4731.3

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <20080725205017.564dedff@siona.local> you wrote:
>
> Ok, I'll stop the chest-beating now. But please stop trying to tell
> people that adding a powerpc-specific option (which nobody seems to
> know how really works) to the command line will work on any other
> architectures than powerpc.

OK - then please you explain exactly which other architectures have
problems with relocation?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
I am more bored than you could ever possibly be.  Go back to work.

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <013a01c8ee5a$966d6c40$c34844c0$@Tjernlund@transmode.se> you wrote:
>
> I think the best bet is to make -mrelocatable work for PPC. Question is how
> does the other arches do it? Do they include fixup ptrs by default or do
> they lack this functionality?

The question that needs to answered first is if any other
architectures are affected at all, and if so, which ones.

For example, ARM and MIPS (and probably most others) do *NOT* perform
any relocation at all (this should be fixed, of course - being one of
the big probalems we inherited from the original ARMBoot port), but as
is the whole discussion is a non-isssue on these architectures.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Systems programmers are the high priests of a low cult.
                                                       -- R.S. Barton

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Thu, Jul 24, 2008 at 2:45 AM, vb <vb@vsbe.com> wrote:
> Wolfgang, thank you for your reply, let me try to explain myself a bit clearer:
>
> On Wed, Jul 23, 2008 at 8:18 PM, Wolfgang Denk <wd@denx.de> wrote:
>> In message <f608b67d0807231039i434e96dbvda86590776db2cb0@mail.gmail.com> you wrote:
>> If you invest time in solving such problems, than your time willbe
>> much better iinvested if you try to help solving the remaining issues
>> with  Grant's code.
>>
>> What Grant suggests is the way to go. I do not think your approach has
>> chances for mainline.
>>
>
> I am sorry to hear this, especially since what I suggest would be
> completely compiler agnostic and would allow to avoid some of the
> limitations one must follow today while adding stuff to u-boot.

The problem is that the u-boot C runtime setup is insane and must be
fixed.  Fix the C runtime and the problem goes away.  The approach
being suggested here would have us *preserve* the insane C runtime and
actually depend on the runtime to remain insane in order to work.

Not a good approach.

The C runtime problem is solvable, but I didn't have the time or
resources to properly dig into it and I got frustrated before it was
fully debugged.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Thu, Jul 24, 2008 at 2:09 PM, Kenneth Johansson <kenneth@southpole.se> wrote:
> On Thu, 2008-07-24 at 10:37 -0700, vb wrote:
>> But as you pointed out, this would work on ppc only (with a 'good'
>> compiler), and still remains to be proven, I will get to it a bit
>> later.
>
> I have run u-boot with everything compiled with -mrelocatable and normal
> relocation fixup code removed so I know it works fine. So consider it
> proven. But since it apparently is a problem in some situation the best
> is probably to make it a configuration option.

Can you test against various versions of gcc?  That is where I ran into trouble.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Fri, Jul 25, 2008 at 10:36 PM, Wolfgang Denk <wd@denx.de> wrote:
> In message <f608b67d0807250751j137e70c0oe67affcf46399c8e@mail.gmail.com> you wrote:
>
> Maybe I should have been more explicit right from the beginning. I
> repeat: forget it. This will *not* go into mainline. Zero chance.
>

well, it's your call, I'ill stick with -mrelocatable on 'non native'
modules then.

cheers,
/vb

> Best regards,
>
> Wolfgang Denk
>
> --
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
> How many seconds are there in a year? If I tell you there are 3.155 x
> 10^7, you won't even try to remember it. On the other hand, who could
> forget that, to within half a percent, pi seconds is  a  nanocentury.
>                                               -- Tom Duff, Bell Labs
>

[U-Boot-Users] Changing u-boot relocation scheme

[vb]

On Fri, Jul 25, 2008 at 10:43 PM, Grant Likely
<grant.likely@secretlab.ca> wrote:
>
> The problem is that the u-boot C runtime setup is insane and must be
> fixed.  Fix the C runtime and the problem goes away.  The approach
> being suggested here would have us *preserve* the insane C runtime and
> actually depend on the runtime to remain insane in order to work.
>

Grant, thank you for pitching in.

It would be great if you could elaborate what exactly is insane about
C runtime setup (especially since in u-boot its code comes from the
source tree and not from some obscure compiler dependent library) and
how its insanity is related here.

TIA,
/vb

> Not a good approach.
>
> The C runtime problem is solvable, but I didn't have the time or
> resources to properly dig into it and I got frustrated before it was
> fully debugged.
>
> g.
>
> --
> Grant Likely, B.Sc., P.Eng.
> Secret Lab Technologies Ltd.
>

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Fri, Jul 25, 2008 at 1:28 PM, kenneth johansson <kenneth@southpole.se> wrote:
> I was afraid that what was needed was more or less a complete linker but
> it looks like if one generate the dynamic reloc table a much simpler
> linker(relocation function) is needed. Still probably a lot more complex
> than the GOT and fixup code that is just a loop over a table. We may
> need a case also ;-)

The GOT relocation works.  It works really well.  As long as gcc and
ld put the correct GOT bits into the GOT table.  It seems not all
versions of gcc do it in the same way with the same flags.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Fri, Jul 25, 2008 at 2:50 PM, Haavard Skinnemoen
<haavard.skinnemoen@atmel.com> wrote:
> Ok, I'll stop the chest-beating now. But please stop trying to tell
> people that adding a powerpc-specific option (which nobody seems to
> know how really works) to the command line will work on any other
> architectures than powerpc.

You are completely right.  For the purposes of this discussion, this
is a powerpc-centric problem.  The relocation code is written in
assembly and is processor specific.  Let the ARM folks make sure ARM
relocation works (and I think it already does).  Let the MIPS folks do
the same.  Each arch has it's own set of CFLAGS and LDFLAGS needed to
produce a sane binary.  They should be picked off and fixed one arch
at a time.

BTW, searching for assignments to reloc_off in the source will tell
you which platforms are broken.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Sat, Jul 26, 2008 at 2:06 AM, Grant Likely <grant.likely@secretlab.ca> wrote:
> BTW, searching for assignments to reloc_off in the source will tell
> you which platforms are broken.

Oops, actually I'm insane on this statement.  Not setting reloc_off
could mean that relocation is simply not performed.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Sat, Jul 26, 2008 at 1:54 AM, vb <vb@vsbe.com> wrote:
> On Fri, Jul 25, 2008 at 10:43 PM, Grant Likely
> <grant.likely@secretlab.ca> wrote:
>>
>> The problem is that the u-boot C runtime setup is insane and must be
>> fixed.  Fix the C runtime and the problem goes away.  The approach
>> being suggested here would have us *preserve* the insane C runtime and
>> actually depend on the runtime to remain insane in order to work.
>>
>
> Grant, thank you for pitching in.
>
> It would be great if you could elaborate what exactly is insane about
> C runtime setup (especially since in u-boot its code comes from the
> source tree and not from some obscure compiler dependent library) and
> how its insanity is related here.

In a sane C runtime, pointers declared in a static variable are
usable.  Since u-boot's relocation does not fix up the pointers, that
means it is insane.

The mechanism exists to do relocation properly.  It is in the u-boot
code and it conforms to the PowerPC ABI.  However, we're not using GCC
correctly so we cannot reliably get the relocation data loaded into
the .got and .fixup sections.  This is all discussed in detail in the
u-boot archives (see the gmane search link that Jerry posted at the
beginning of this thread).

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Sat, 2008-07-26 at 07:36 +0200, Wolfgang Denk wrote:
> In message <1217006928.7972.47.camel@duo> you wrote:
> >
> > I know exactly what Grant did. But he is not using the elf relocation
> > information as that is simply not included in the data that is the
> > u-boot.bin file. 
> 
> So what? Is this a problem? Why would we need to use XXX if we can
> perform all we need to do by using YYY instead, especially if this is
> simpler to implement?

The only reason would be to make all arch as similar as possible. The
discussion would be totally retarded if we only though about powerpc
since the problem is already solved and implemented.

> > Also this discussion is more about doing it in a way that is not powerpc
> > specific and documented, we already have as you mentioned code for the
> > powerpc specific way but it's not using elf relocation information.
> 
> Who says that we MUST use the ELF reloc info?

The post I was replaying to indicated that the existing code already did
this so I simply corrected that statement.

> Or why?

If it turns out that is is not that complicated all architectures could
do it in the same way. 

[U-Boot-Users] Changing u-boot relocation scheme

[kenneth johansson]

On Sat, 2008-07-26 at 01:48 -0400, Grant Likely wrote:
> On Thu, Jul 24, 2008 at 2:09 PM, Kenneth Johansson <kenneth@southpole.se> wrote:
> > On Thu, 2008-07-24 at 10:37 -0700, vb wrote:
> >> But as you pointed out, this would work on ppc only (with a 'good'
> >> compiler), and still remains to be proven, I will get to it a bit
> >> later.
> >
> > I have run u-boot with everything compiled with -mrelocatable and normal
> > relocation fixup code removed so I know it works fine. So consider it
> > proven. But since it apparently is a problem in some situation the best
> > is probably to make it a configuration option.
> 
> Can you test against various versions of gcc?  That is where I ran into trouble.

Well If you have a specific version you know is not working I could test
that.

[U-Boot-Users] Changing u-boot relocation scheme

[Grant Likely]

On Sat, Jul 26, 2008 at 3:53 AM, kenneth johansson <kenneth@southpole.se> wrote:
> On Sat, 2008-07-26 at 01:48 -0400, Grant Likely wrote:
>> On Thu, Jul 24, 2008 at 2:09 PM, Kenneth Johansson <kenneth@southpole.se> wrote:
>> > On Thu, 2008-07-24 at 10:37 -0700, vb wrote:
>> >> But as you pointed out, this would work on ppc only (with a 'good'
>> >> compiler), and still remains to be proven, I will get to it a bit
>> >> later.
>> >
>> > I have run u-boot with everything compiled with -mrelocatable and normal
>> > relocation fixup code removed so I know it works fine. So consider it
>> > proven. But since it apparently is a problem in some situation the best
>> > is probably to make it a configuration option.
>>
>> Can you test against various versions of gcc?  That is where I ran into trouble.
>
> Well If you have a specific version you know is not working I could test
> that.

gcc 4.0 worked for me.  Anything newer or older seemed to behave
differently.  Looking at the size of the .got and .fixup sections
seems to be a good indication of if things are being placed correctly.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <fa686aa40807252306p14e0fe43ta7fba15b4b43b492@mail.gmail.com> you wrote:
> 
> You are completely right.  For the purposes of this discussion, this
> is a powerpc-centric problem.  The relocation code is written in
> assembly and is processor specific.  Let the ARM folks make sure ARM
> relocation works (and I think it already does).  Let the MIPS folks do
> the same.  Each arch has it's own set of CFLAGS and LDFLAGS needed to
> produce a sane binary.  They should be picked off and fixed one arch
> at a time.

Neither ARM nor MIPS do relocation. They link for a fixed address in
RAM (which for example means that they cannot deal in a sane way with
systems where the RAM size might change, not to mention issues with
things like support for pRAM, pre-initialized framebuffers or shared
log buffer which only work reasonably when you have relocation.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
This is now.  Later is later.

[U-Boot-Users] Changing u-boot relocation scheme

[Wolfgang Denk]

In message <1217058099.7972.75.camel@duo> you wrote:
>
> > Or why?
> 
> If it turns out that is is not that complicated all architectures could
> do it in the same way. 

That would require that all architectures implementrelocation in the
same way (or at all). Actually this is not the case.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
There you go man, Keep as cool as you can. It riles them  to  believe
that you perceive the web they weave. Keep on being free!

[U-Boot-Users] Changing u-boot relocation scheme

[Jean-Christophe PLAGNIOL-VILLARD]

On 01:57 Sat 26 Jul     , Grant Likely wrote:
> On Fri, Jul 25, 2008 at 1:28 PM, kenneth johansson <kenneth@southpole.se> wrote:
> > I was afraid that what was needed was more or less a complete linker but
> > it looks like if one generate the dynamic reloc table a much simpler
> > linker(relocation function) is needed. Still probably a lot more complex
> > than the GOT and fixup code that is just a loop over a table. We may
> > need a case also ;-)
> 
> The GOT relocation works.  It works really well.  As long as gcc and
> ld put the correct GOT bits into the GOT table.  It seems not all
> versions of gcc do it in the same way with the same flags.

Why not set the code depending on the GCC version?

Best Regards,
J.

[U-Boot-Users] Changing u-boot relocation scheme

[Joakim Tjernlund]

> -----Original Message-----
> From: u-boot-users-bounces at lists.sourceforge.net [mailto:u-boot-users-bounces at lists.sourceforge.net]
> On Behalf Of Jean-Christophe PLAGNIOL-VILLARD
> Sent: den 26 juli 2008 16:03
> To: Grant Likely
> Cc: u-boot-users at lists.sourceforge.net; Wolfgang Denk; vb; Jerry Van Baren; Haavard Skinnemoen
> Subject: Re: [U-Boot-Users] Changing u-boot relocation scheme
> 
> On 01:57 Sat 26 Jul     , Grant Likely wrote:
> > On Fri, Jul 25, 2008 at 1:28 PM, kenneth johansson <kenneth@southpole.se> wrote:
> > > I was afraid that what was needed was more or less a complete linker but
> > > it looks like if one generate the dynamic reloc table a much simpler
> > > linker(relocation function) is needed. Still probably a lot more complex
> > > than the GOT and fixup code that is just a loop over a table. We may
> > > need a case also ;-)
> >
> > The GOT relocation works.  It works really well.  As long as gcc and
> > ld put the correct GOT bits into the GOT table.  It seems not all
> > versions of gcc do it in the same way with the same flags.
> 
> Why not set the code depending on the GCC version?

For fun I had a look into eabi.asm code at 
 http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/config/rs6000/eabi.asm?rev=1.13&content-type=text/x-cvsweb-markup
and I noticed one difference:
The __eabi_uconvert() function skips NULL ptrs.

Perhaps this is the missing piece needed in start.S for PPC?

__eabi_convert pasted below for convenience.

    Jocke

FUNC_START(__eabi_convert)
        cmplw   1,3,4                                /* any pointers to convert? */
        subf    5,3,4                                /* calculate number of words to convert */
        bclr    4,4                                /* return if no pointers */

        srawi   5,5,2
        addi        3,3,-4                                /* start-4 for use with lwzu */
        mtctr   5

.Lcvt:
        lwzu        6,4(3)                                /* pointer to convert */
        cmpwi        0,6,0
        beq-        .Lcvt2                                /* if pointer is null, don't convert */

        add     6,6,12                                /* convert pointer */
        stw     6,0(3)
.Lcvt2:
        bdnz+   .Lcvt
        blr

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Fri, 25 Jul 2008 22:51:09 +0200
kenneth johansson <kenneth@southpole.se> wrote:

> > Hmm...looks like linking with -pie (or --pic-executable) does
> > something vaguely resembling the trick. But I don't know what vintage
> > of ld you need for this to be available and actually work...  
> 
> Can't see any reason for using this flag over -fPIC for a program like
> u-boot.

You need both. One is a compiler flag, the other is a linker flag. The
linker will probably barf if you try to create a PIC executable from
modules that were not compiled with -fPIC.

On the other hand, using -fPIC but not -pie takes us about 95% of the
way to a fully relocatable u-boot, and is what PowerPC, AVR32 and
others are doing today. The remaining cases -- static and global
initialized pointers -- are fixed up manually.

If we link with -pie and add a bit of code to handle the dynamic
relocations, we could get rid of the manual fixups. I don't know if the
result would be larger or smaller than what we have today, but it would
probably be a bit more robust.

> > Does this really work without any relocation? Or is it being relocated
> > and I'm just too blind to see it?  
> 
> No you are right there is probably many places like this. the code just
> use the old location and that works most of the time.

Ah, of course. The strings are probably read directly from flash.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Sat, 26 Jul 2008 07:36:35 +0200
Wolfgang Denk <wd@denx.de> wrote:

> n message <20080725205017.564dedff@siona.local> you wrote:
> >
> > Ok, I'll stop the chest-beating now. But please stop trying to tell
> > people that adding a powerpc-specific option (which nobody seems to
> > know how really works) to the command line will work on any other
> > architectures than powerpc.  
> 
> OK - then please you explain exactly which other architectures have
> problems with relocation?

Well, AVR32 doesn't exactly have a _problem_ with relocation except
that it relies on a handful of manual fixups for initialized pointers.
From a quick scan, Blackfin, m68k, MIPS, PPC and SPARC also rely on
this.

While it does work, it is somewhat fragile and requires people writing
new code to know about this particular quirk if they want to use
statically initialized pointers.

Also, as was shown elsewhere in this thread, quite a few strings are
being accessed directly from flash even after relocating to RAM. This
could at least in theory interfere with the flash driver due to cache
effects.

So I'm a bit concerned that if PowerPC introduces its own method of
relocating these data pointers, the code used on other platforms would
get less exposure and therefore be more susceptible to bugs. Ideally,
all platforms should do relocation the same way, and that necessarily
means using standard methods like ELF dynamic relocations.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[J. William Campbell]

Haavard Skinnemoen wrote:
> On Fri, 25 Jul 2008 22:51:09 +0200
> kenneth johansson <kenneth@southpole.se> wrote:
>
>   
>>> Hmm...looks like linking with -pie (or --pic-executable) does
>>> something vaguely resembling the trick. But I don't know what vintage
>>> of ld you need for this to be available and actually work...  
>>>       
>> Can't see any reason for using this flag over -fPIC for a program like
>> u-boot.
>>     
>
> You need both. One is a compiler flag, the other is a linker flag. The
> linker will probably barf if you try to create a PIC executable from
> modules that were not compiled with -fPIC.
>   
No, it won't. You just get a module with a lot more relocations to do. I 
have verified that all four possible combinations of the compiler -fPIC 
and linker -pie work and make sense. FWIW, -fPIC code on IA32 is about 
16% larger than non-PIC code, while on the Blackfin, -fPIC code is about 
2% larger than non-PIC code. This is an average over several large C++ 
applications.


> On the other hand, using -fPIC but not -pie takes us about 95% of the
> way to a fully relocatable u-boot, and is what PowerPC, AVR32 and
> others are doing today. The remaining cases -- static and global
> initialized pointers -- are fixed up manually.
>
> If we link with -pie and add a bit of code to handle the dynamic
> relocations, we could get rid of the manual fixups. I don't know if the
> result would be larger or smaller than what we have today, but it would
> probably be a bit more robust.
>   
I agree with this suggestion. This is the only way to ensure a "sane" 
environment, because it emulates what the compiler expects to happen. 
Looping over all the relocation entries and doing the "right thing" is 
architecture specific, but the process is general. The GOT entries can 
also be processed this way. Effort spent on this approach will tend to 
be more generic than the current PPC specific approach.
>   
>>> Does this really work without any relocation? Or is it being relocated
>>> and I'm just too blind to see it?  
>>>       
>> No you are right there is probably many places like this. the code just
>> use the old location and that works most of the time.
>>     
>
> Ah, of course. The strings are probably read directly from flash.
>   
Maybe not. I have been looking at assembly dumps of short examples on 
the IA32 built with -fPIC. It is clear that the method of addressing 
static variables and static constants is DIFFERENT from the method used 
for global variables. The relationship of the location of the text 
segment (executable code), the GOT data, and the static 
variables/constants must remain fixed. The location of the entire 
program can move, but it must move in one piece. If it does move as one 
piece, the lea (load effective address) instructions relative to the GOT 
pointer will be relocated to the new address correctly. These references 
are based totally on the offset from the point of reference. If the code 
is similar on your platform (which I bet it is), then the reference will 
not be to the flash but rather the "new" place where the data was 
moved.. Global variables, however are referenced indirectly via 32 bit 
address pointers in the GOT, and these addresses must be relocated by 
the "loader".
The "loader" also must relocate any initialized pointers, because the 
program itself does not. It would be interesting to know how this is 
accomplished, via what relocation codes, but it does happen.

Best Regards,
Bill Campbell
> Haavard
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> U-Boot-Users mailing list
> U-Boot-Users at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/u-boot-users
>
>
>   

[U-Boot-Users] Changing u-boot relocation scheme

[Haavard Skinnemoen]

On Sat, 26 Jul 2008 14:29:35 -0700
"J. William Campbell" <jwilliamcampbell@comcast.net> wrote:
> Haavard Skinnemoen wrote:
> > On Fri, 25 Jul 2008 22:51:09 +0200
> > kenneth johansson <kenneth@southpole.se> wrote:    
> >> Can't see any reason for using this flag over -fPIC for a program like
> >> u-boot.
> >>     
> >
> > You need both. One is a compiler flag, the other is a linker flag. The
> > linker will probably barf if you try to create a PIC executable from
> > modules that were not compiled with -fPIC.
> >   
> No, it won't.

On some platforms it will. Text relocations are nasty, so some
platforms (e.g. avr32) just refuse to deal with them. But that's not
really relevant -- each architecture should decide whether to compile
with -fPIC or not.

> You just get a module with a lot more relocations to do. I 
> have verified that all four possible combinations of the compiler -fPIC 
> and linker -pie work and make sense. FWIW, -fPIC code on IA32 is about 
> 16% larger than non-PIC code, while on the Blackfin, -fPIC code is about 
> 2% larger than non-PIC code. This is an average over several large C++ 
> applications.

Right...that's counting the whole loadable image or just the .text
section? Not suprising that a modern architecture like Blackfin likes
-fPIC a lot better than an old beast like i386 though.
 
> I agree with this suggestion. This is the only way to ensure a "sane" 
> environment, because it emulates what the compiler expects to happen. 
> Looping over all the relocation entries and doing the "right thing" is 
> architecture specific, but the process is general. The GOT entries can 
> also be processed this way. Effort spent on this approach will tend to 
> be more generic than the current PPC specific approach.

Right...I think the GOT entries already are processed this way, sort of.

> > Ah, of course. The strings are probably read directly from flash.
> >   
> Maybe not. I have been looking at assembly dumps of short examples on 
> the IA32 built with -fPIC. It is clear that the method of addressing 
> static variables and static constants is DIFFERENT from the method used 
> for global variables. The relationship of the location of the text 
> segment (executable code), the GOT data, and the static 
> variables/constants must remain fixed. The location of the entire 
> program can move, but it must move in one piece. If it does move as one 
> piece, the lea (load effective address) instructions relative to the GOT 
> pointer will be relocated to the new address correctly. These references 
> are based totally on the offset from the point of reference. If the code 
> is similar on your platform (which I bet it is), then the reference will 
> not be to the flash but rather the "new" place where the data was 
> moved..

Yes, address calculations in the code should be correct, as the whole
thing was compiled with -fPIC. Data references, however, are usually
not. The code being discussed here is an array of pointers to strings.
I'm pretty sure the pointers are still pointing to flash after
relocation.

> Global variables, however are referenced indirectly via 32 bit 
> address pointers in the GOT, and these addresses must be relocated by 
> the "loader".

The global variables themselves are accessed through the GOT, yes. But
the _value_ of a global variable is currently not relocated
automatically.

> The "loader" also must relocate any initialized pointers, because the 
> program itself does not. It would be interesting to know how this is 
> accomplished, via what relocation codes, but it does happen.

This is what's currently being done manually by adding a fixed offset
to all the pointers we "know" need to be relocated. When linking with
-pie, these initialized pointers will get a dynamic relocation entry
each so that we can replace all these manual fixups by simply iterating
over the relocations.

To summarize: Address calculations in executable code do not need to
change since we already compile with -fPIC. Initialized pointers,
however, are currently handled in a very suboptimal way, and linking
with -pie might be one piece of the solution to this.

Haavard

[U-Boot-Users] Changing u-boot relocation scheme

[J. William Campbell]

Haavard Skinnemoen wrote:
> On Sat, 26 Jul 2008 14:29:35 -0700
> "J. William Campbell" <jwilliamcampbell@comcast.net> wrote:
>   
>> Haavard Skinnemoen wrote:
>>     
>>> On Fri, 25 Jul 2008 22:51:09 +0200
>>> kenneth johansson <kenneth@southpole.se> wrote:    
>>>       
>>>> Can't see any reason for using this flag over -fPIC for a program like
>>>> u-boot.
>>>>     
>>>>         
>>> You need both. One is a compiler flag, the other is a linker flag. The
>>> linker will probably barf if you try to create a PIC executable from
>>> modules that were not compiled with -fPIC.
>>>   
>>>       
>> No, it won't.
>>     
>
> On some platforms it will. Text relocations are nasty, so some
> platforms (e.g. avr32) just refuse to deal with them. But that's not
> really relevant -- each architecture should decide whether to compile
> with -fPIC or not.
>
>   
>> You just get a module with a lot more relocations to do. I 
>> have verified that all four possible combinations of the compiler -fPIC 
>> and linker -pie work and make sense. FWIW, -fPIC code on IA32 is about 
>> 16% larger than non-PIC code, while on the Blackfin, -fPIC code is about 
>> 2% larger than non-PIC code. This is an average over several large C++ 
>> applications.
>>     
>
> Right...that's counting the whole loadable image or just the .text
> section? Not suprising that a modern architecture like Blackfin likes
> -fPIC a lot better than an old beast like i386 though.
>   
That is the entire loadable image. The percentage will therefore be 
slightly higher as a function of just the .text section. It is pretty 
representative of the effect though. On the "old beast", it is not so good!
>  
>   
>> I agree with this suggestion. This is the only way to ensure a "sane" 
>> environment, because it emulates what the compiler expects to happen. 
>> Looping over all the relocation entries and doing the "right thing" is 
>> architecture specific, but the process is general. The GOT entries can 
>> also be processed this way. Effort spent on this approach will tend to 
>> be more generic than the current PPC specific approach.
>>     
>
> Right...I think the GOT entries already are processed this way, sort of
>   
>>> Ah, of course. The strings are probably read directly from flash.
>>>   
>>>       
>> Maybe not. I have been looking at assembly dumps of short examples on 
>> the IA32 built with -fPIC. It is clear that the method of addressing 
>> static variables and static constants is DIFFERENT from the method used 
>> for global variables. The relationship of the location of the text 
>> segment (executable code), the GOT data, and the static 
>> variables/constants must remain fixed. The location of the entire 
>> program can move, but it must move in one piece. If it does move as one 
>> piece, the lea (load effective address) instructions relative to the GOT 
>> pointer will be relocated to the new address correctly. These references 
>> are based totally on the offset from the point of reference. If the code 
>> is similar on your platform (which I bet it is), then the reference will 
>> not be to the flash but rather the "new" place where the data was 
>> moved..
>>     
>
> Yes, address calculations in the code should be correct, as the whole
> thing was compiled with -fPIC. Data references, however, are usually
> not. The code being discussed here is an array of pointers to strings.
> I'm pretty sure the pointers are still pointing to flash after
> relocation.
>   
You are correct. The contents of initialized pointers are not relocated 
without using the relocation data provided by -pie on the ld script.
>   
>> Global variables, however are referenced indirectly via 32 bit 
>> address pointers in the GOT, and these addresses must be relocated by 
>> the "loader".
>>     
>
> The global variables themselves are accessed through the GOT, yes. But
> the _value_ of a global variable is currently not relocated
> automatically.
>
>   
>> The "loader" also must relocate any initialized pointers, because the 
>> program itself does not. It would be interesting to know how this is 
>> accomplished, via what relocation codes, but it does happen.
>>     
>
> This is what's currently being done manually by adding a fixed offset
> to all the pointers we "know" need to be relocated. When linking with
> -pie, these initialized pointers will get a dynamic relocation entry
> each so that we can replace all these manual fixups by simply iterating
> over the relocations.
>
> To summarize: Address calculations in executable code do not need to
> change since we already compile with -fPIC. Initialized pointers,
> however, are currently handled in a very suboptimal way, and linking
> with -pie might be one piece of the solution to this.
>   
I agree completely. Now we "just" need to add processing the relocation 
data. Also, the relocation vectors differ in contents from platform to 
platform, so a bit of "custom" processing will be needed for each 
architecture.

Bill Campbell
> Haavard
>
>
>   

[U-Boot] Changing u-boot relocation scheme

[vb]

On Wed, Jul 23, 2008 at 10:39 AM, vb <vb@vsbe.com> wrote:

>
> Here is the suggested alternative. I will talk about recent PPC
> platforms, but it could be done very similar for most of the other
> platforms I believe.
>
> In general the PPC u-boot image consists of three parts: the bulk of
> code and data, the startup page (the top 4k mapped out of reset), and
> the reset vector (the top four bytes in the address space). The bulk
> is usually aligned at a certain address matching the bootprom size,
> and there is a gap between the bulk and the startup page.
>
> Now, let's say we link and purge the bulk section twice, once locating
> it at offset 0 and once at the normal u-boot TEXT_BASE, building two
> binary images (not including the top 4k in both cases).
>
> Then we compare the two images and identify all locations which are
> different. All these locations are 4 byte aligned, and all differ by
> the same amount - equal TEXT_BASE. Now we can build a table of offsets
> which need to be changed when the program is moved into a different
> location, include this table in the resulting image between the bulk
> and the startup page, and use this table when relocating code.
>

so, I finally had some time to spend on this over the weekend, and
made it work on the canyonlands board (ppc460gt platform).

My original assessment was missing the fact that in some cases the
absolute address is split in two halves (when the 32 bit address value
is loaded into a registrer with two assembler instructions).  As a
result, there were two kinds of discrepancies between the images
located at different addresses: in the majority of cases the
difference was the same as the base address difference, and in a
couple of dozen cases the difernce between images was base address
difference shifted 16 bits to the right.

It turned out not a big deal, the code changes are minimal (100 or so
lines added and 200 or so deleted), not counting the script comparing
the images and generating the table and the resulting binary with the
table included in it.

If there is any interest, I can share the diffs with the community.

I also tried to make sense of the contents of the .fixup section
created by -mrelocatable, but I failed here - I expected it to contain
the offsets of the locatons to relocate, but it certainly is not it,
and I am not siure what it is.

The address fixup function is obviously completely isolated and can be
easily replaced to match the relocation information format.

To make sure I din not miss any place where gd->reloc_offset is used I
changed the name of the field temporarily and let the compiler find
all places where this field is used. Tesing was not all that thorough,
but inlcuded running cli commands, saving environment in flash,
downloading an image through bootp/tftp.

Let me know if anyone is interested,

cheers,
Vadim