[U-Boot] [PATCH v2 3/3] image: Allow images to indicate they're loadable at any address

Nicolas Pitre
2011-11-08T14:26:52Z

On Tue, 8 Nov 2011, Wolfgang Denk wrote:

> In message <alpine.LFD.2.02.1111071840080.3307@xanadu.home> you wrote:
> >
> > > I understand you are referring here to zImages only. Correct?
> >
> > Correct. Anything else is not relocatable.
> >
> > > Or will raw images (without the preloader) be fully relocatable, too?
> >
> > No.
>
> OK. So the situation on ARM is basicly not different from what we
> have on other architectures.
>
> The uImage format has been designed to handle this situation: we
> normally wrap the raw image (eventually after compressing it) into an
> uImage, and insert the load and entry point addresses that are well
> known at kernel build time. uImages are not directly executable.
>
> zImages on ARM include a preloader, that wraps the the raw image
> (eventually after compressing it) and packs it together with some
> code that internally accounts for the fixed load and entry point
> addresses of the kernel which implements the needed relocation and
> uncompression routines to form a new executable, position independent
> format.
>
> In both cases the _kernel_ image is not position independent. It must
> be loaded to a specific address and started at a specific entry point.
> The exact information where these are is known at built time, and
> somehow encoded in the images (here in the image header, there in the
> preloader code).
>
>
> Is this summary correct so far?

No. Your statement that says "The exact information where these are is
known at built time, and somehow encoded in the images (here in the
image header, there in the preloader code)" is false. None of that
information is known at build time nor encoded in the preloader as you
call it.

> > > Why is it so important to load it at specific (different) addresses
> > > when it can be started from any address?
> >
> > The kernel code can be started from any address. We want the _code_ to
> > be that way.
>
> This is not quite correct. The _kernel_ code has a fixed address.
> It is the preloader code which can be started from any address.

The preloader is part of the kernel, whether you like it or not, and it
is not going away any time soon. That would help the conversation if
you accepted it as such.

> > However a particular board may or may not load the kernel at any
> > address. This is a machine specific restriction, not a kernel
> > restriction.
>
> Correct. Stephen Warren also writes in <4EB87122.3050602@nvidia.com>:
> "Presumably whoever constructed the environment has the most detailed
> knowledge of the HW's and U-Boot environment's expected memory layout
> for that particular board." And in <4EB87375.1040100@nvidia.com>:
> "The only place that has full knowledge of the board's memory layout
> is the U-Boot environment for that board, and hence I assert that the
> U-Boot environment should define where to load the kernel (and initrd
> and FDT), and if U-Boot must copy them, where to copy them to. In
> particular, the creator of the uImage can't possibly pick these
> values and expect them to work everywhere."
>
> I mostly agree with this. That's the reason why U-Boot offers to
> handle this based on a better understanding of the low level hardware
> details. This seems to me to be an easier way that to handle this in
> the Linux kernel environment (in the form of the preloader).

You are mixing up issues. The bootloader has knowledge of where is the
best place to load kernel images on a per machine basis. This is why
the zImage code was made to accept and execute from any address in order
to give the bootloader any possible flexibility in that regard.
However it is the kernel's responsibility to deal with its own
environment and change it at will when we see fit, and therefore this is
best if the bootloader doesn't take any responsibility here.

> > > Maybe this is a key point. I simply fail to understand this.
> >
> > Let me repeat again. We want one single kernel image binary that ARM
> > distributions can use for all their target machines. It is therefore
> > necessary that uImage be free of any hardcoded load address (absolute or
> > relative). If a particular board require a particular load address for
> > the kernel, this must be encoded in its own u-Boot environment and not
> > in the distributed uImage. Failing that, uImage simply cannot be used
> > as a distribution format for the kernel because any address/offset
> > enforced by the uImage format is going to be incompatible with the needs
> > of a particular machine somewhere.
>
> uImage is _intented_ to carry a raw OS image and provide the boot
> loader the information where to load and start it.
>
> You do not want to do that, so you should not attempt to assault
> uImages to make it fit your completely different purposes. If you
> want to use relocatable zImages which have all the needed information
> encoded internally then just do so. But do not insist that uImage
> does things it was not designed for.

OK, fine. I don't really care about uImage if it cannot accommodate our
needs. I thought that the -1 extension was a pretty agile and
non-obstrusive way to make u-Boot more flexible and versatile, and not
only for Linux loading. Failing that, can we have support for loading
zImage directly then? There are precedents in the upstream u-Boot for
doing that on other architectures already.

> > No. The requirements on raw images are unchanged. you can use them if
> > you wish, but generic ARM distributions can't use that if they want to
> > target more than one SOC. Therefore raw images are not interesting by
> > the use case at hand.
>
> They are not interesting for you. But that does not mean that they
> are not interesting in general. Simon Glass brings up this point,
> too, in <CAPnjgZ1WajMLznSDNDLBys-C3Um2rVvZjOLk3BfSO2M=FRW2Dg@mail.gmail.com>:
>
> | Can I assume that we have (or can have) a 'make uImage' target or
> | similar in the kernel which can pack together:
> |
> | - a compressed kernel (not zImage, I mean something that U-Boot can
> | decompress), with a rel_offset of 32KB
> | - a DTB
> | - a ramdisk
> |
> | and that with Stephen's patch (committed to U-Boot) today, we can, in
> | U-Boot, with a script, load this uImage to somewhere and have U-Boot
> | decompress the kernel and set the bits out nicely in RAM, no matter
> | where that RAM is? The kernel will start at 32KB, and the other bits
> | will be somewhere above that. Then U-Boot can enter the kernel at 32KB
> | and all will be well, yes?
>
> Why are attempts blocked to provide such support for people who prefer
> this approach? A "one size fits all" approach like "we have always
> been using zImages, so it must be a perfect solution for everybody" is
> eventually _not_ the optimal solution for all.

I never intended to prevent raw images from being used with uImage.
This is however not the use case I'm looking after. That doesn't mean
that because I'm interested in a different usage model than yours that
I'm going to
actively prevent raw Image from being used. I see obvious limitations
with it, but if that is what you want then by all means just use that.
We cannot say as much from the u-Boot side for the use case I'm
interested in though.

> > What do you not understand in the fact that such a specific address
> > makes the resulting image not universal? I'm telling you that I need to
> > produce a kernel image that doesn't carry with it any machine specific
> > load address so that same image can be installed unmodified on any
> > machine.
>
> And what I'm telling you is that you could probably have that for free
> if you dropped the preloader.

But that is not what I want.

> > Instead, you insist on making that image less useful by attaching to it
> > some restrictions on its load address at build time rather than applying
> > those restrictions only at load time and only on machines where that
> > matters.
> >
> > For the last time, we don't want any address encoded in the kernel
> > image for the simple fact that we don't know at build time what machine
> > the kernel will be used on, and therefore what address to use. This is
>
> But instead of letting the boot loader who has the detailled knowledge
> about which memory is actually present on the target system you build
> a non-trivial preloader and insist that the ultimate way is to wrape
> the kernel with it.

Yes, because the kernel still requires to take total control on its
execution environment, and we are now doing that at run time instead of
compile time, which does include techniques like self modifying code
i.e. runtime patching of the kernel binary, etc.

> > why zImage was made totally relocatable and totally position
> > independent so it can figure out at run time what address to use.
> > Figuring out the address to use at "make uImage" time only works for a
> > kernel that will boot on a single specific machine.
>
> There is no actual reason this has to be that way. If you want then
> risk a view over the rim of your plate and look how it's working for
> other architectures. It's not really new problems we're dealing with
> here. Kernel images that run on a number of different processors and
> boards are not new, and device tree support has been available for
> years, too. It's not that there is no preexisting experience with
> such issues.
>
> On Power Architecture systems, we use uImage format and/or FIT image
> format to wrap raw kernel images (without preloader). If somebody
> wants, he can attach / include the DT blob to that image, and
> eventually also a ramdisk/initrd image. This is not mandatory, it is
> all optimal. And it just works.

I'm coming to the conclusion that your world view is different from mine
then. Maybe ARM used to be similar to PPC in the past, but ARM is
moving on and things are changing. The requirements are changing.
u-Boot is trying to hold us in the past and making itself more and more
an obstacle. The only thing you could do is to merge the provided
patches which doesn't affect in any way your own view, but would allow
for u-Boot not to be an obstacle anymore and everybody would be happy.
Given those patches are
rather simple and non intrusive, I'm failing to understand why you are
putting so much resistance in the name of some philosophical agenda or I
don't know what other reasons.

> > It is already the case, more or less. The raw image _must_ be loaded at
> > TEXT_OFFSET from start of RAM, regardless of where that RAM is. So if
> > all you care about is raw image kernels, then having a relative a load
> > address for uImage makes perfect sense.
>
> Thanks for this clear statement. So I will apply the patches to add
> such support to U-Boot.

Good. I've given you confirmation that you can use the kernel in the
way you want to use it. I consider it a valid although suboptimal use
case, but I'm not denying you that possibility.

> > This is a rigid restriction that a well intended bootloader would not
> > impose all the time if the user wishes to do otherwise. You might not
> > be concerned by use cases where we want images to be executed right
> > where they've been loaded irrespective of what that might be, but that
> > doesn't make that any less of a valid use case. And Stephen's patches
> > are providing just that.
>
> No, they aren't.
>
> I already NAKed these patches, and this discussion has made it clear
> to me that this was a correct decision. What you want is not uImages.

You are therefore denying me the ability to use the kernel according to
the use case I care about. Maybe I should reconsider my willingness to
let you use raw kernel image then? Because if you are not willing to
collaborate for the case I care about, why should I care about yours?

Nicolas