Building an uImage with appended DTB

Building an uImage with appended DTB

[Mason]

Hello,

Per Russell's advice (IIRC) I have a script to generate a uImage
of a kernel with an appended DTB:

export LOADADDR=0x80008000
MAKE="make -j2 ARCH=arm"
$MAKE dtbs zImage && \
cat arch/arm/boot/zImage arch/arm/boot/dts/tango4-vantage-1172.dtb >zImage.tmp && \
mv zImage.tmp arch/arm/boot/zImage && \
$MAKE uImage

Russell's version uses an explicit mkimage instead of "MAKE uImage"
perhaps in part to get around the following limitation:

if there is nothing to be done, cat+mv will update arch/arm/boot/zImage
thus the uImage will be rebuilt even when it's not necessary.

Doesn't the top-level Makefile support a target to build a uImage
with appended DTB?

(I suppose someone has submitted a patch for that. Was it perhaps
never merged? Or did I just miss the feature?)

Something like this:
https://patchwork.kernel.org/patch/1133461/

Regards.

Building an uImage with appended DTB

[Grant Likely]

On Wed, Nov 25, 2015 at 4:16 PM, Mason <slash.tmp@free.fr> wrote:
> Hello,
>
> Per Russell's advice (IIRC) I have a script to generate a uImage
> of a kernel with an appended DTB:
>
> export LOADADDR=0x80008000
> MAKE="make -j2 ARCH=arm"
> $MAKE dtbs zImage && \
> cat arch/arm/boot/zImage arch/arm/boot/dts/tango4-vantage-1172.dtb >zImage.tmp && \
> mv zImage.tmp arch/arm/boot/zImage && \
> $MAKE uImage
>
> Russell's version uses an explicit mkimage instead of "MAKE uImage"
> perhaps in part to get around the following limitation:
>
> if there is nothing to be done, cat+mv will update arch/arm/boot/zImage
> thus the uImage will be rebuilt even when it's not necessary.
>
> Doesn't the top-level Makefile support a target to build a uImage
> with appended DTB?
>
> (I suppose someone has submitted a patch for that. Was it perhaps
> never merged? Or did I just miss the feature?)

I don't remember what happened there. That was a long time ago. I
think it got nacked, but cannot remember the reason.

g.

Building an uImage with appended DTB

[Russell King - ARM Linux]

On Wed, Nov 25, 2015 at 04:38:35PM +0000, Grant Likely wrote:
> On Wed, Nov 25, 2015 at 4:16 PM, Mason <slash.tmp@free.fr> wrote:
> > Hello,
> >
> > Per Russell's advice (IIRC) I have a script to generate a uImage
> > of a kernel with an appended DTB:
> >
> > export LOADADDR=0x80008000
> > MAKE="make -j2 ARCH=arm"
> > $MAKE dtbs zImage && \
> > cat arch/arm/boot/zImage arch/arm/boot/dts/tango4-vantage-1172.dtb >zImage.tmp && \
> > mv zImage.tmp arch/arm/boot/zImage && \
> > $MAKE uImage
> >
> > Russell's version uses an explicit mkimage instead of "MAKE uImage"
> > perhaps in part to get around the following limitation:
> >
> > if there is nothing to be done, cat+mv will update arch/arm/boot/zImage
> > thus the uImage will be rebuilt even when it's not necessary.
> >
> > Doesn't the top-level Makefile support a target to build a uImage
> > with appended DTB?
> >
> > (I suppose someone has submitted a patch for that. Was it perhaps
> > never merged? Or did I just miss the feature?)
> 
> I don't remember what happened there. That was a long time ago. I
> think it got nacked, but cannot remember the reason.

It got nacked because we don't want these special boot loader specific
formats in the kernel, and it's getting beyond a joke.  It was a mistake
to merge the uImage target in the first place IMHO.

Previous to uboot, boot loaders were perfectly happy with zImage.  Today,
all boot loaders are perfectly happy with a zImage - it's only the old
legacy uboot versions that insist on uImage now.  There's now even less
of a reason to think about merging some uImage based DT munging than there
was several years ago.

Just do it in an external script, like I do here for platforms where it's
necessary.  Absolutely no need to hack around in the kernel with special
crap, like passing environment variables to make (which, you need a script
for anyway, because its laborious to have to keep on typing them.)

If you're writing a build script so you don't have to remember lots of
environment variables for the kernel, you might as well write the build
script to do the mkimage step for you.

So having it in the kernel is totally pointless and insane.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Building an uImage with appended DTB

[Mason]

On 25/11/2015 19:45, Russell King - ARM Linux wrote:
> On Wed, Nov 25, 2015 at 04:38:35PM +0000, Grant Likely wrote:
>> On Wed, Nov 25, 2015 at 4:16 PM, Mason wrote:
>>> Hello,
>>>
>>> Per Russell's advice (IIRC) I have a script to generate a uImage
>>> of a kernel with an appended DTB:
>>>
>>> export LOADADDR=0x80008000
>>> MAKE="make -j2 ARCH=arm"
>>> $MAKE dtbs zImage && \
>>> cat arch/arm/boot/zImage arch/arm/boot/dts/tango4-vantage-1172.dtb >zImage.tmp && \
>>> mv zImage.tmp arch/arm/boot/zImage && \
>>> $MAKE uImage
>>>
>>> Russell's version uses an explicit mkimage instead of "MAKE uImage"
>>> perhaps in part to get around the following limitation:
>>>
>>> if there is nothing to be done, cat+mv will update arch/arm/boot/zImage
>>> thus the uImage will be rebuilt even when it's not necessary.
>>>
>>> Doesn't the top-level Makefile support a target to build a uImage
>>> with appended DTB?
>>>
>>> (I suppose someone has submitted a patch for that. Was it perhaps
>>> never merged? Or did I just miss the feature?)
>>
>> I don't remember what happened there. That was a long time ago. I
>> think it got nacked, but cannot remember the reason.
> 
> It got nacked because we don't want these special boot loader specific
> formats in the kernel, and it's getting beyond a joke.  It was a mistake
> to merge the uImage target in the first place IMHO.
> 
> Previous to uboot, boot loaders were perfectly happy with zImage.  Today,
> all boot loaders are perfectly happy with a zImage - it's only the old
> legacy uboot versions that insist on uImage now.  There's now even less
> of a reason to think about merging some uImage based DT munging than there
> was several years ago.

Actually, I'm using a fairly recent version of U-Boot (2014.04)
IIUC, bootz should do the trick... But I'm not finding much doc
about it... <confused>

If I pass U-Boot a zImage, where should I load it, and how does
U-Boot know where to relocate it? (LOADADDR?)
And why is LOADADDR 0x80008000 and not 0x80000000 (the start
of the RAM)?

Misc links for my own reference:
https://patchwork.ozlabs.org/patch/146848/
https://stackoverflow.com/questions/22322304/image-vs-zimage-vs-uimage
https://unix.stackexchange.com/questions/122526/how-to-convert-a-zimage-into-uimage-for-booting-with-u-boot
http://lists.denx.de/pipermail/u-boot/2015-March/208551.html
http://www.denx.de/wiki/DULG/Manual

> Just do it in an external script, like I do here for platforms where it's
> necessary.  Absolutely no need to hack around in the kernel with special
> crap, like passing environment variables to make (which, you need a script
> for anyway, because its laborious to have to keep on typing them.)

How do you prevent your script from rebuilding the uImage every
time it is invoked?

> If you're writing a build script so you don't have to remember lots of
> environment variables for the kernel, you might as well write the build
> script to do the mkimage step for you.
> 
> So having it in the kernel is totally pointless and insane.

I understand.

Regards.

Building an uImage with appended DTB

[Russell King - ARM Linux]

On Wed, Nov 25, 2015 at 11:41:49PM +0100, Mason wrote:
> On 25/11/2015 19:45, Russell King - ARM Linux wrote:
> > It got nacked because we don't want these special boot loader specific
> > formats in the kernel, and it's getting beyond a joke.  It was a mistake
> > to merge the uImage target in the first place IMHO.
> > 
> > Previous to uboot, boot loaders were perfectly happy with zImage.  Today,
> > all boot loaders are perfectly happy with a zImage - it's only the old
> > legacy uboot versions that insist on uImage now.  There's now even less
> > of a reason to think about merging some uImage based DT munging than there
> > was several years ago.
> 
> Actually, I'm using a fairly recent version of U-Boot (2014.04)
> IIUC, bootz should do the trick... But I'm not finding much doc
> about it... <confused>
> 
> If I pass U-Boot a zImage, where should I load it, and how does
> U-Boot know where to relocate it? (LOADADDR?)
> And why is LOADADDR 0x80008000 and not 0x80000000 (the start
> of the RAM)?

I no longer remember all the details, but the reason we avoid the first
32K is because we need to have some place to put some page tables so we
can turn the caches on - and I decided it was all round easier to place
the page tables below the image, and these need to be aligned to 16K.
IIRC, in the distant past, I also needed to reserve space for other
things as well, so the base of the kernel became RAM start plus 32K.

> > Just do it in an external script, like I do here for platforms where it's
> > necessary.  Absolutely no need to hack around in the kernel with special
> > crap, like passing environment variables to make (which, you need a script
> > for anyway, because its laborious to have to keep on typing them.)
> 
> How do you prevent your script from rebuilding the uImage every
> time it is invoked?

I don't, but if you wanted to do that look at the tests you can do with
'test' aka '['.  You'll find the -nt test which will allow you to test
for one file being newer than the other.  You probably ought to also
test for the existence of the target file too.  So, something like:

if [ ! -e "$uimage" -o "$zimage" -nt "$uimage" ]; then
   mkimage...
fi

You only need the double-quotes around $uimage and $zimage if these
variables may be empty or contain spaces in their expansion.

> > If you're writing a build script so you don't have to remember lots of
> > environment variables for the kernel, you might as well write the build
> > script to do the mkimage step for you.
> > 
> > So having it in the kernel is totally pointless and insane.
> 
> I understand.

It gets even more fun when you've developed this idea to the extent
of building and packaging up a kernel with modules into a tarball,
transferring it to a number of platforms, and then having the platforms
reboot into the new kernel - with everything from the build right the
way through to the reboots all resulting from running one script with
no user interaction needed. :)

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Building an uImage with appended DTB

[Nicolas Pitre]

On Thu, 26 Nov 2015, Russell King - ARM Linux wrote:

> On Wed, Nov 25, 2015 at 11:41:49PM +0100, Mason wrote:
> > On 25/11/2015 19:45, Russell King - ARM Linux wrote:
> > > It got nacked because we don't want these special boot loader specific
> > > formats in the kernel, and it's getting beyond a joke.  It was a mistake
> > > to merge the uImage target in the first place IMHO.
> > > 
> > > Previous to uboot, boot loaders were perfectly happy with zImage.  Today,
> > > all boot loaders are perfectly happy with a zImage - it's only the old
> > > legacy uboot versions that insist on uImage now.  There's now even less
> > > of a reason to think about merging some uImage based DT munging than there
> > > was several years ago.
> > 
> > Actually, I'm using a fairly recent version of U-Boot (2014.04)
> > IIUC, bootz should do the trick... But I'm not finding much doc
> > about it... <confused>
> > 
> > If I pass U-Boot a zImage, where should I load it, and how does
> > U-Boot know where to relocate it? (LOADADDR?)
> > And why is LOADADDR 0x80008000 and not 0x80000000 (the start
> > of the RAM)?
> 
> I no longer remember all the details, but the reason we avoid the first
> 32K is because we need to have some place to put some page tables so we
> can turn the caches on - and I decided it was all round easier to place
> the page tables below the image, and these need to be aligned to 16K.
> IIRC, in the distant past, I also needed to reserve space for other
> things as well, so the base of the kernel became RAM start plus 32K.

These days you may load zImage anywhere and it will relocate itself out 
of the way from either the decompressed Image and the initial page 
table.  It is not smart enough to relocate the DTB and the initrd 
though.

U-Boot shouldn't have to relocate anything. I don't know how to convince 
it not to relocate things though.

As a rule of thumb, you can load all the pieces you need above the 32MB 
mark from start of RAM. And then you can start zImage regardless of its 
address in memory simply by branching to it, with the address of your 
DTB in r2.


Nicolas