[Buildroot] [Bug 5516] New: appended device tree blobs on uImage fails

[Buildroot] [Bug 5516] New: appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

           Summary: appended device tree blobs on uImage fails
           Product: buildroot
           Version: unspecified
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: minor
          Priority: P5
         Component: Other
        AssignedTo: unassigned at buildroot.uclibc.org
        ReportedBy: richard.genoud at gmail.com
                CC: buildroot at uclibc.org
   Estimated Hours: 0.0


The version impacted is 2012.08 (it's not in the list)
the commit 978928e4acb0ee4e335eaf4bf8feca895f372418 adds the ability to append
device tree blobs.
This works great with zImage.
BUT, with uImage there's a problem:
in linux.mk, the line :
cat $(KERNEL_ARCH_PATH)/boot/$(KERNEL_DTS_NAME).dtb >>
$(KERNEL_ARCH_PATH)/boot/zImage
is all right, but after that, the line :
$(TARGET_MAKE_ENV) $(MAKE) $(LINUX_MAKE_FLAGS) -C $(@D) uImage
erases the zImage that has been just done.

that's because in the kernel, the uImage rule is :
$(obj)/uImage:  $(obj)/zImage FORCE
        @$(check_for_multiple_loadaddr)
        $(call if_changed,uimage)
        @echo '  Image $@ is ready'

=> so if we do a make uImage, zImage is overwritten.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #1 from Maxime Ripard <maxime.ripard@free-electrons.com> 2012-10-03 11:40:55 UTC ---
I can't reproduce the problem here, doing :

$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- make -j8 zImage
$ cat arch/arm/boot/<random>.dtb >> arch/arm/boot/zImage
$ ls -l arch/arm/boot/zImage
-rwxrwxr-x 1 max max 2577673 oct.   3 13:36 arch/arm/boot/zImage
$ md5sum arch/arm/boot/zImage
d88b9ddc04847e06b34f80841a973ce5  arch/arm/boot/zImage
$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- make -j8 uImage
$ ls -l arch/arm/boot/zImage
-rwxrwxr-x 1 max max 2577673 oct.   3 13:36 arch/arm/boot/zImage
$ ls -l arch/arm/boot/uImage
-rw-rw-r-- 1 max max 2577737 oct.   3 13:37 arch/arm/boot/uImage
$ md5sum arch/arm/boot/zImage
d88b9ddc04847e06b34f80841a973ce5  arch/arm/boot/zImage

So it definitely looks to me like the zImage is left untouched.

What are you doing to see that behaviour? What is your configuration file? What
kernel version are you using?

(In reply to comment #0)
> that's because in the kernel, the uImage rule is :
> $(obj)/uImage:  $(obj)/zImage FORCE
>         @$(check_for_multiple_loadaddr)
>         $(call if_changed,uimage)
>         @echo '  Image $@ is ready'

I don't get by looking at that code why the zImage should be overwritten?

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #2 from richard.genoud at gmail.com 2012-10-03 13:25:24 UTC ---
(In reply to comment #1)
hum, I can't reproduce your zImage not beeing overwritten.
Ok, let's start from that:

Kernel v3.6 (and I tried with 3.0, 2.6.33) :

ARCH=arm
CROSS_COMPILE=/[blablabla]/buildroot/output/host/usr/bin/arm-unknown-linux-uclibcgnueabi-
make -j 8 zImage
  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: ? include/generated/mach-types.h ? est ? jour.
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  CHK     kernel/config_data.h
  Kernel: arch/arm/boot/Image is ready
  LZMA    arch/arm/boot/compressed/piggy.lzma
  AS      arch/arm/boot/compressed/piggy.lzma.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready

linux$ ((v3.6)) ARCH=arm
CROSS_COMPILE=/[blablabla]/buildroot/output/host/usr/bin/arm-unknown-linux-uclibcgnueabi-
make -j 8 uImage
  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: ? include/generated/mach-types.h ? est ? jour.
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  CHK     kernel/config_data.h
  Kernel: arch/arm/boot/Image is ready
  LZMA    arch/arm/boot/compressed/piggy.lzma
  AS      arch/arm/boot/compressed/piggy.lzma.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage                          <= zImage overwritten
  Kernel: arch/arm/boot/zImage is ready
  UIMAGE  arch/arm/boot/uImage
Image Name:   Linux-3.6.0
Created:      Wed Oct  3 14:14:37 2012
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    1906552 Bytes = 1861.87 kB = 1.82 MB
Load Address: 20008000
Entry Point:  20008000
  Image arch/arm/boot/uImage is ready

Even without touching to zImage, the objcopy is done again

NB: I tried that with a random config : at91sam9263_defconfig

with 2.6.33:
ARCH=arm
CROSS_COMPILE=/[blablalba]/buildroot/output/host/usr/bin/arm-unknown-linux-uclibcgnueabi-
make -j 8 zImage
  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: ? include/generated/mach-types.h ? est ? jour.
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  Kernel: arch/arm/boot/Image is ready
  SHIPPED arch/arm/boot/compressed/lib1funcs.S
  AS      arch/arm/boot/compressed/lib1funcs.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
linux$ ((v2.6.33)) ls --full-time arch/arm/boot/zImage
-rwxr-xr-x 1 rgenoud rgenoud 1473900 2012-10-03 14:53:07.839044793
+0200 arch/arm/boot/zImage
linux$ ((v2.6.33)) ARCH=arm
CROSS_COMPILE=/[blablabla]/buildroot/output/host/usr/bin/arm-unknown-linux-uclibcgnueabi-
make -j 8 uImage
  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: ? include/generated/mach-types.h ? est ? jour.
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  Kernel: arch/arm/boot/Image is ready
  SHIPPED arch/arm/boot/compressed/lib1funcs.S
  AS      arch/arm/boot/compressed/lib1funcs.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
  UIMAGE  arch/arm/boot/uImage
Image Name:   Linux-2.6.33
Created:      Wed Oct  3 14:53:17 2012
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    1473900 Bytes = 1439.36 kB = 1.41 MB
Load Address: 20008000
Entry Point:  20008000
  Image arch/arm/boot/uImage is ready
linux$ ((v2.6.33)) ls --full-time arch/arm/boot/zImage
-rwxr-xr-x 1 rgenoud rgenoud 1473900 2012-10-03 14:53:17.599033136
+0200 arch/arm/boot/zImage
So, dates are not the same.

The buildroot toolchain is compiled with:
BR2_UCLIBC_VERSION_STRING="0.9.33.2"
BR2_BINUTILS_VERSION="2.21.1"
BR2_GCC_VERSION="4.5.4"
(buildroot 2012.08)

2012/10/3  <bugzilla@busybox.net>:
> (In reply to comment #0)
>> that's because in the kernel, the uImage rule is :
>> $(obj)/uImage:  $(obj)/zImage FORCE
>>         @$(check_for_multiple_loadaddr)
>>         $(call if_changed,uimage)
>>         @echo '  Image $@ is ready'
>
> I don't get by looking at that code why the zImage should be overwritten?
The FORCE rule forces uImage creation. and those lines:
$(obj)/zImage:  $(obj)/compressed/vmlinux FORCE
        $(call if_changed,objcopy)
        @echo '  Kernel: $@ is ready'
are forcing zImage creation.
since uImage depends on zImage, zImage is re-evaluated, and recreated
because of FORCE.
At least, that's what I understand.
(file arch/arm/boot/Makefile )

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #3 from richard.genoud at gmail.com 2012-10-03 14:23:40 UTC ---
ok, this time to really be sure that I don't mess with anything, I made this
script:
cat ../test_zImage.sh 
#!/bin/sh

export
CROSS_COMPILE=PATH_TO_BUILDROOT/buildroot/output/host/usr/bin/arm-unknown-linux-uclibcgnueabi- 
export ARCH=arm

git checkout v3.0
git clean -fdx
make at91sam9263_defconfig 
make -j8 zImage 1>/dev/null 2>&1
ls --full-time arch/arm/boot/zImage
make -j8 uImage
ls --full-time arch/arm/boot/zImage


Be careful with it, there's the git clean -fdx that wipes out your kernel repo,
and the script itself by the way.

./test_zImage.sh
[...]
-rwxr-xr-x 1 rgenoud rgenoud 1790036 2012-10-03 15:52:55.074666703 +0200
arch/arm/boot/zImage
  CHK     include/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: ? include/generated/mach-types.h ? est ? jour.
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  Kernel: arch/arm/boot/Image is ready
  LZMA    arch/arm/boot/compressed/piggy.lzma
  SHIPPED arch/arm/boot/compressed/lib1funcs.S
  AS      arch/arm/boot/compressed/lib1funcs.o
  AS      arch/arm/boot/compressed/piggy.lzma.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
  UIMAGE  arch/arm/boot/uImage
Image Name:   Linux-3.0.0
Created:      Wed Oct  3 15:52:58 2012
Image Type:   ARM Linux Kernel Image (uncompressed)
Data Size:    1790036 Bytes = 1748.08 kB = 1.71 MB
Load Address: 20008000
Entry Point:  20008000
  Image arch/arm/boot/uImage is ready
-rwxr-xr-x 1 rgenoud rgenoud 1790036 2012-10-03 15:52:58.518707579 +0200
arch/arm/boot/zImage

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #4 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2012-10-03 14:32:34 UTC ---
Maxime is testing with a more recent kernel version (3.6). Maybe things have
changed since 2.6.33/3.0 that you are testing?

Maxime, would be good if you could test with older kernel versions as well.
They are not old enough to not support them in Buildroot (even though testing
those DT related features with 2.6.33 is kind of useless since there was no DT
on ARM at 2.6.33).

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #5 from richard.genoud at gmail.com 2012-10-03 14:52:33 UTC ---
(In reply to comment #4)
> Maxime is testing with a more recent kernel version (3.6). Maybe things have
> changed since 2.6.33/3.0 that you are testing?
> 
> Maxime, would be good if you could test with older kernel versions as well.
> They are not old enough to not support them in Buildroot (even though testing
> those DT related features with 2.6.33 is kind of useless since there was no DT
> on ARM at 2.6.33).

The 1st kernel I tested was 3.6-rc4.
I used the script against vanilla 3.6, and I still get zImage reconstructed.

besides, I'm not a Makefile guru, but the rule uImage needs zImage and FORCE
And the rule zImage needs compressed/vmlinux and FORCE
so it gets to be rebuilt. 
(by the way, compressed/vmlinux depends also on FORCE, like Image)

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #6 from Maxime Ripard <maxime.ripard@free-electrons.com> 2012-10-04 20:47:36 UTC ---
Created attachment 4574
  --> https://bugs.busybox.net/attachment.cgi?id=4574
Patch to use mkimage directly instead of make uImage

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #7 from Maxime Ripard <maxime.ripard@free-electrons.com> 2012-10-04 20:48:08 UTC ---
Could you test the attached patch? It should solve your problem, and will apply
on top of master.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #8 from richard.genoud at gmail.com 2012-10-05 06:46:06 UTC ---
ok, I can't test that right now, but I will on tuesday.
Thanks !

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #9 from richard.genoud at gmail.com 2012-10-09 08:12:16 UTC ---
(In reply to comment #7)
> Could you test the attached patch? It should solve your problem, and will apply
> on top of master.

Tested ok, works as expected.

Thanks !

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #10 from Shawn J. Goff <shawn7400@gmail.com> 2013-05-21 23:16:40 UTC ---
This patch was never committed. The issue is affecting me now on 2013.05_rc2
with a 3.9 kernel. The linux.mk file has changed enough that this won't apply.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #11 from Shawn J. Goff <shawn7400@gmail.com> 2013-05-21 23:52:40 UTC ---
Created attachment 4910
  --> https://bugs.busybox.net/attachment.cgi?id=4910
Modified Maxime's patch to apply to 2013.05_rc2

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #12 from richard.genoud at gmail.com 2013-05-22 09:39:21 UTC ---
As I didn't see this bug happening for quite some time (I even forgot about
it), I did some investigations:
With v3.7+, zImage is not overwritten when we are doing a "make uImage", so DTB
is appended correctly.
But versions before v3.7 overwrite the zImage.

bisecting:
cd linux
git bisect start
git bisect good v3.6
git bisect bad v3.7
git bisect run ../test_zImage_overwrite.sh

(script provided)

here is the bisect result :
c353acba28fb3fa1fd05fd6b85a9fc7938330f9c
kbuild: make: fix if_changed when command contains backslashes

=> that means that appending DTB to uImage only works for kernel >=
v3.6-6674-gc353acb in buildroot.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #13 from richard.genoud at gmail.com 2013-05-22 09:41:14 UTC ---
Created attachment 4916
  --> https://bugs.busybox.net/attachment.cgi?id=4916
bisect script to dectect if zImage is overwritten when make uImage is called

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #14 from Shawn J. Goff <shawn7400@gmail.com> 2013-05-22 10:36:36 UTC ---
No, as I said, it's happening to me now on 3.9. What's strange is that I had
been going along fine, then I changed some kernel options that I would not
expect to have anything to do with this, and now `make uImage` is clobbering my
zImage, so it's not as simple as that it always/never happens on specific
kernel versions.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #15 from richard.genoud at gmail.com 2013-05-22 13:33:35 UTC ---
(In reply to comment #14)
> No, as I said, it's happening to me now on 3.9. What's strange is that I had
> been going along fine, then I changed some kernel options that I would not
> expect to have anything to do with this, and now `make uImage` is clobbering my
> zImage, so it's not as simple as that it always/never happens on specific
> kernel versions.
Could you try to isolate the config option that change the behavior ? (or give
a .config that overwrite zImage, and another that doesn't)

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #16 from Shawn J. Goff <shawn7400@gmail.com> 2013-05-23 16:18:13 UTC ---
Created attachment 4922
  --> https://bugs.busybox.net/attachment.cgi?id=4922
config where uImage clobbers zImage

I was hoping to spend some time to get a minimal example, but I haven't gotten
to it yet, but I wanted to get something up in case anyone else is looking into
this at the same time. This config does have the problem.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

--- Comment #17 from Maxime Ripard <maxime.ripard@free-electrons.com> 2013-05-27 15:13:10 UTC ---
(In reply to comment #16)
> I was hoping to spend some time to get a minimal example, but I haven't gotten
> to it yet, but I wanted to get something up in case anyone else is looking into
> this at the same time. This config does have the problem.

I couldn't reproduce the problem here with 3.9 and the config you were pointing
to. Anyway, since 3.7 is quite recent anyway, we can always merge the patch, it
does no harm, and fixes things for some users.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.

[Buildroot] [Bug 5516] appended device tree blobs on uImage fails

[bugzilla at busybox.net]

https://bugs.busybox.net/show_bug.cgi?id=5516

Thomas Petazzoni <thomas.petazzoni@free-electrons.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #18 from Thomas Petazzoni <thomas.petazzoni@free-electrons.com> 2013-08-01 16:02:26 UTC ---
Fixed as of ae86a3c5331f47c4a5093a5ea8f7e53408a6ea98 which will be part of
2013.08.

-- 
Configure bugmail: https://bugs.busybox.net/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.