From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
To: linux-sh@vger.kernel.org
Subject: Re: Armadillo boot failures
Date: Tue, 16 Dec 2014 22:18:27 +0000 [thread overview]
Message-ID: <1684550.1vdjI6RaDt@avalon> (raw)
In-Reply-To: <CAMuHMdV-vMxObnaDXv84vq=qNf1NForN0LL1SkrzLxoxiMDSGQ@mail.gmail.com>
Hi Geert,
On Tuesday 16 December 2014 16:12:31 Geert Uytterhoeven wrote:
> Sometimes, Armadillo fails to boot.
>
> So far I've seen 4 failure modes (with DEBUG_LL):
> - It hangs after "Doing init=/init" (printed by the boot loader)
> - It hangs with
>
> Uncompressing Linux... done, booting the kernel.
> Error: unrecognized/unsupported processor variant (0x412fc093).
>
> - It starts the kernel, but prints a few times "unflatten: error -11
> processing FDT",
> and doesn't boot correctly as some devices seem to be missing.
>
> As commit 08d53aa58cb162e6 ("of/fdt: export fdt blob as /sys/firmware/fdt")
> calculates a CRC32 on the DTB, I added some extra code to investigate
> failure mode 3, and dump the DTB if the CRC32 doesn't match the expected
> value.
>
> Below are the results when comparing the corruption in 4 failures with the
> pristine DTB:
>
> --- /tmp/armadillo.dtb.good 2014-12-16 15:23:02.075461856 +0100
> +++ /tmp/armadillo.dtb.bad1 2014-12-16 16:03:43.027456464 +0100
> @@ -336,8 +336,8 @@
> 000014f0 00 00 00 03 00 00 00 04 00 00 00 88 00 03 d0 90
> |................| 00001500 00 00 00 03 00 00 00 04 00 00 00 95 00 03 d0
> 90 |................| 00001510 00 00 00 03 00 00 00 09 00 00 01 d5 64 69
> 73 61 |............disa| -00001520 62 6c 65 64 00 00 00 00 00 00 00 02
> 00 00 00 01 |bled............| -00001530 73 65 72 69 61 6c 40 65 36 63
> 38 30 30 30 30 00 |serial@e6c80000.| +00001520 e8 a5 54 c0 b0 65 66 06
> 0d c0 a0 e1 f0 d8 2d e9 |..T..ef.......-.| +00001530 04 b0 4c e2 a6 6f 80
> e2 00 40 a0 e1 06 00 a0 e1 |..L..o...@......| 00001540 00 00 00 03 00 00
> 00 24 00 00 00 1b 72 65 6e 65 |.......$....rene| 00001550 73 61 73 2c 73
> 63 69 66 61 2d 72 38 61 37 37 34 |sas,scifa-r8a774| 00001560 30 00 72 65
> 6e 65 73 61 73 2c 73 63 69 66 61 00 |0.renesas,scifa.| @@ -914,8 +914,8
> @@
> 00003910 00 00 00 03 00 00 00 0c 00 00 03 d7 00 00 00 0f
> |................| 00003920 00 00 00 4b 00 00 00 00 00 00 00 03 00 00 00
> 00 |...K............| 00003930 00 00 03 dc 00 00 00 03 00 00 00 04 00 00
> 01 58 |...............X| -00003940 00 00 00 18 00 00 00 03 00 00 00 04
> 00 00 01 5e |...............^| -00003950 00 00 00 18 00 00 00 02 00 00
> 00 01 72 65 67 75 |............regu| +00003940 50 f0 ff eb 07 00 a0 e1
> 24 d0 4b e2 f0 ab 9d e8 |P.......$.K.....| +00003950 2f 30 4c c0 59 17 b7
> d1 e8 a5 54 c0 b0 65 66 06 |/0L.Y.....T..ef.| 00003960 6c 61 74 6f 72 40
> 32 00 00 00 00 03 00 00 00 0f |lator@2.........| 00003970 00 00 00 1b 72
> 65 67 75 6c 61 74 6f 72 2d 67 70 |....regulator-gp| 00003980 69 6f 00 00
> 00 00 00 03 00 00 00 0b 00 00 03 72 |io.............r| ---
> /tmp/armadillo.dtb.good 2014-12-16 15:23:02.075461856 +0100
> +++ /tmp/armadillo.dtb.bad2 2014-12-16 13:29:41.583476878 +0100
> @@ -330,9 +330,9 @@
> 00001490 00 00 00 08 00 00 00 66 e6 c7 00 00 00 00 01 00
> |.......f........| 000014a0 00 00 00 03 00 00 00 0c 00 00 01 66 00 00 00
> 00 |...........f....| 000014b0 00 00 00 67 00 00 00 04 00 00 00 03 00 00
> 00 08 |...g............| -000014c0 00 00 01 ac 00 00 00 08 00 00 00 01
> 00 00 00 03 |................| -000014d0 00 00 00 08 00 00 01 b3 73 63
> 69 5f 69 63 6b 00 |........sci_ick.| -000014e0 00 00 00 03 00 00 00 04
> 00 00 00 7a 00 00 00 10 |...........z....| +000014c0 00 00 01 ac 00 00 00
> 08 e8 a5 54 c0 b0 65 66 06 |..........T..ef.| +000014d0 0d c0 a0 e1 f0
> d8 2d e9 04 b0 4c e2 a6 6f 80 e2 |......-...L..o..| +000014e0 00 40 a0
> e1 06 00 a0 e1 00 00 00 7a 00 00 00 10 |.@.........z....| 000014f0 00 00
> 00 03 00 00 00 04 00 00 00 88 00 03 d0 90 |................| 00001500 00
> 00 00 03 00 00 00 04 00 00 00 95 00 03 d0 90 |................| 00001510
> 00 00 00 03 00 00 00 09 00 00 01 d5 64 69 73 61 |............disa| @@
> -908,9 +908,9 @@
> 000038b0 72 65 67 75 6c 61 74 6f 72 40 31 00 00 00 00 03
> |regulator@1.....| 000038c0 00 00 00 10 00 00 00 1b 72 65 67 75 6c 61 74
> 6f |........regulato| 000038d0 72 2d 66 69 78 65 64 00 00 00 00 03 00 00
> 00 0a |r-fixed.........| -000038e0 00 00 03 72 53 44 48 49 30 20 56 63
> 63 00 00 00 |...rSDHI0 Vcc...| -000038f0 00 00 00 03 00 00 00 04 00 00
> 03 81 00 32 5a a0 |.............2Z.| -00003900 00 00 00 03 00 00 00 04
> 00 00 03 99 00 32 5a a0 |.............2Z.| +000038e0 00 00 03 72 53 44 48
> 49 50 f0 ff eb 07 00 a0 e1 |...rSDHIP.......| +000038f0 24 d0 4b e2 f0
> ab 9d e8 2f 30 4c c0 59 17 b7 d1 |$.K...../0L.Y...| +00003900 e8 a5 54
> c0 b0 65 66 06 00 00 03 99 00 32 5a a0 |..T..ef......2Z.| 00003910 00 00
> 00 03 00 00 00 0c 00 00 03 d7 00 00 00 0f |................| 00003920 00
> 00 00 4b 00 00 00 00 00 00 00 03 00 00 00 00 |...K............| 00003930
> 00 00 03 dc 00 00 00 03 00 00 00 04 00 00 01 58 |...............X| ---
> /tmp/armadillo.dtb.good 2014-12-16 15:23:02.075461856 +0100
> +++ /tmp/armadillo.dtb.bad3 2014-12-16 13:37:14.623475878 +0100
> @@ -209,8 +209,8 @@
> 00000d00 34 30 00 00 00 00 00 03 00 00 00 10 00 00 00 66
> |40.............f| 00000d10 e9 a0 00 00 00 00 08 00 e9 a0 18 00 00 00 08
> 00 |................| 00000d20 00 00 00 03 00 00 00 0c 00 00 01 66 00 00
> 00 00 |...........f....| -00000d30 00 00 00 6e 00 00 00 04 00 00 00 03
> 00 00 00 08 |...n............| -00000d40 00 00 01 ac 00 00 00 06 00 00
> 00 09 00 00 00 03 |................| +00000d30 00 d8 2d e9 04 b0 4c e2
> 00 00 50 e3 04 30 9f 15 |..-...L...P..0..| +00000d40 00 00 93 15 00 a8 9d
> e8 d4 f7 d3 c0 0d c0 a0 e1 |................| 00000d50 00 00 00 04 00 00
> 00 7a 00 00 00 04 00 00 00 03 |.......z........| 00000d60 00 00 00 04 00
> 00 00 88 00 03 d0 90 00 00 00 03 |................| 00000d70 00 00 00 04
> 00 00 00 95 00 03 d0 90 00 00 00 03 |................| @@ -523,8 +523,8
> @@
> 000020a0 00 00 00 03 00 00 00 04 00 00 02 86 00 00 00 08
> |................| 000020b0 00 00 00 03 00 00 00 00 00 00 02 90 00 00 00
> 02 |................| 000020c0 00 00 00 01 73 64 40 65 36 38 35 30 30 30
> 30 00 |....sd@e6850000.| -000020d0 00 00 00 03 00 00 00 15 00 00 00 1b
> 72 65 6e 65 |............rene| -000020e0 73 61 73 2c 73 64 68 69 2d 72
> 38 61 37 37 34 30 |sas,sdhi-r8a7740| +000020d0 00 c0 81 15 00 00 91 e5
> 28 d0 4b e2 f0 af 9d e8 |........(.K.....| +000020e0 4c 74 55 c0 80 96 98
> 00 9f 86 01 00 8c 74 55 c0 |LtU..........tU.| 000020f0 00 00 00 00 00 00
> 00 03 00 00 00 08 00 00 00 66 |...............f| 00002100 e6 85 00 00 00
> 00 01 00 00 00 00 03 00 00 00 24 |...............$| 00002110 00 00 01 66
> 00 00 00 00 00 00 00 75 00 00 00 04 |...f.......u....| @@ -629,8 +629,8
> @@
> 00002740 00 00 00 04 00 00 00 0f 00 00 00 01 00 00 00 03
> |................| 00002750 00 00 00 00 00 00 02 e4 00 00 00 01 65 78 74
> 61 |............exta| 00002760 6c 72 5f 63 6c 6b 00 00 00 00 00 03 00 00
> 00 0c |lr_clk..........| -00002770 00 00 00 1b 66 69 78 65 64 2d 63 6c
> 6f 63 6b 00 |....fixed-clock.| -00002780 00 00 00 03 00 00 00 04 00 00
> 02 eb 00 00 00 00 |................| +00002770 03 10 a0 e3 01 20 a0 e3
> 05 30 a0 e1 10 85 f6 eb |..... ...0......| +00002780 78 a8 9d e8 5c 83 4a
> c0 0d c0 a0 e1 00 d8 2d e9 |x...\.J.......-.| 00002790 00 00 00 03 00 00
> 00 04 00 00 00 6a 00 00 80 00 |...........j....| 000027a0 00 00 00 03 00
> 00 00 07 00 00 02 f8 65 78 74 61 |............exta| 000027b0 6c 72 00 00
> 00 00 00 03 00 00 00 04 00 00 01 58 |lr.............X| ---
> /tmp/armadillo.dtb.good 2014-12-16 15:23:02.075461856 +0100
> +++ /tmp/armadillo.dtb.bad4 2014-12-16 15:17:14.339462624 +0100
> @@ -323,8 +323,8 @@
> 00001420 00 00 00 04 00 00 00 95 00 03 d0 90 00 00 00 03
> |................| 00001430 00 00 00 09 00 00 01 d5 64 69 73 61 62 6c 65
> 64 |........disabled| 00001440 00 00 00 00 00 00 00 02 00 00 00 01 73 65
> 72 69 |............seri| -00001450 61 6c 40 65 36 63 37 30 30 30 30 00
> 00 00 00 03 |al@e6c70000.....| -00001460 00 00 00 24 00 00 00 1b 72 65
> 6e 65 73 61 73 2c |...$....renesas,| +00001450 e8 a5 54 c0 b0 65 66 06
> 0d c0 a0 e1 f0 d8 2d e9 |..T..ef.......-.| +00001460 04 b0 4c e2 a6 6f 80
> e2 00 40 a0 e1 06 00 a0 e1 |..L..o...@......| 00001470 73 63 69 66 61 2d
> 72 38 61 37 37 34 30 00 72 65 |scifa-r8a7740.re| 00001480 6e 65 73 61 73
> 2c 73 63 69 66 61 00 00 00 00 03 |nesas,scifa.....| 00001490 00 00 00 08
> 00 00 00 66 e6 c7 00 00 00 00 01 00 |.......f........| @@ -901,8 +901,8
> @@
> 00003840 00 00 03 72 66 69 78 65 64 2d 33 2e 33 56 00 00
> |...rfixed-3.3V..| 00003850 00 00 00 03 00 00 00 04 00 00 03 81 00 32 5a
> a0 |.............2Z.| 00003860 00 00 00 03 00 00 00 04 00 00 03 99 00 32
> 5a a0 |.............2Z.| -00003870 00 00 00 03 00 00 00 00 00 00 03 b1
> 00 00 00 03 |................| -00003880 00 00 00 00 00 00 03 c5 00 00
> 00 03 00 00 00 04 |................| +00003870 50 f0 ff eb 07 00 a0 e1
> 24 d0 4b e2 f0 ab 9d e8 |P.......$.K.....| +00003880 b3 30 4c c0 59 17 b7
> d1 e8 a5 54 c0 b0 65 66 06 |.0L.Y.....T..ef.| 00003890 00 00 01 58 00 00
> 00 16 00 00 00 03 00 00 00 04 |...X............| 000038a0 00 00 01 5e 00
> 00 00 16 00 00 00 02 00 00 00 01 |...^............| 000038b0 72 65 67 75
> 6c 61 74 6f 72 40 31 00 00 00 00 03 |regulator@1.....|
>
> Findings:
> - Corruption always happens in multiples of 8 bytes,
> - The same patterns appear multiple times:
> - "e8 a5 54 c0 b0 65 66 06" is seen 6 times
> - "00 40 a0 e1 06 00 a0 e1" is seen 3 times
> - "04 b0 4c e2 a6 6f 80 e2" is seen 3 times
> - "0d c0 a0 e1 f0 d8 2d e9" is seen 3 times
> - "24 d0 4b e2 f0 ab 9d e8" is seen 3 times
> - "50 f0 ff eb 07 00 a0 e1" is seen 3 times
> - The 32-byte sequence "e8 a5 54 c0 b0 65 66 06 0d c0 a0 e1 f0 d8 2d
> e9 04 b0 4c e2 a6 6f 80 e2 00 40 a0 e1 06 00 a0 e1" is seen twice.
>
> I tried matching these patterns with the kernel image. Some patterns are
> rare, others are seen very frequently in the kernel.
>
> Most interesting is this 32-byte match:
> 0029db80 00 d8 2d e9 04 b0 4c e2 00 00 50 e3 04 30 9f 15
> |..-...L...P..0..| 0029db90 00 00 93 15 00 a8 9d e8 d4 f7 d3 c0 0d c0 a0
> e1 |................|
>
> This is a piece of code from the function cpuidle_get_cpu_driver().
> But apart from that, I didn't find something interesting.
>
> Anyone with a clue? Is this just a bug in hermit's L2 cache setup?
I've noticed that hermit corrupts the last byte of the DT blob in some (not
fully understood) circumstances that depend on the blob size. My kernel
compilation scripts append an extra byte to the .dtb for that reason. Could it
help in your case ?
--
Regards,
Laurent Pinchart
next prev parent reply other threads:[~2014-12-16 22:18 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-12-16 15:12 Armadillo boot failures Geert Uytterhoeven
2014-12-16 22:18 ` Laurent Pinchart [this message]
2014-12-17 7:57 ` Geert Uytterhoeven
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1684550.1vdjI6RaDt@avalon \
--to=laurent.pinchart@ideasonboard.com \
--cc=linux-sh@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.