[U-Boot] U-Boot overlaps BSS and initrd on arm64

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Dmitry Vyukov]

Hello,

We've got a boot problem on arm64 devices. Here is boot log:
https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/

Here is some debugging that Andrey and Mark did:

On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> So we have
> Dtb address  0x81f00000
> Load address 0x80000000
> Which gives us 31Mb for kernel.
>
> It says that Kernel image is 24.62 MiB, but that's without BSS.
> If bss is big enough it might overwrite dtb.
> And indeed, build details -
> https://kernelci.org/build/id/581e850959b514e564f03bdc/
> shows that bss is 8.5 Mb which is enough to overlap with dtb.

On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> FWIW, since v3.17 we've had an image_size field in the arm64 Image
> header which describes the "real" size of the Image, BSS included. See
> [1,2].
> It should be possible to modify U-Boot to use that to automatically
> place the DTB and initrd at non-clashing locations (or at least to
> expose the value somehow).
> I had assumed U-Boot already did that, but it doesn't seem to be the
> case.
> [1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=a2c1d73b94ed49f5fac12e95052d7b140783f800
> [2] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/arm64/booting.txt?h=v4.9-rc4


Thanks

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Tom Rini]

On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> Hello,
> 
> We've got a boot problem on arm64 devices. Here is boot log:
> https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> 
> Here is some debugging that Andrey and Mark did:
> 
> On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > So we have
> > Dtb address  0x81f00000
> > Load address 0x80000000
> > Which gives us 31Mb for kernel.
> >
> > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > If bss is big enough it might overwrite dtb.
> > And indeed, build details -
> > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> 
> On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > header which describes the "real" size of the Image, BSS included. See
> > [1,2].
> > It should be possible to modify U-Boot to use that to automatically
> > place the DTB and initrd at non-clashing locations (or at least to
> > expose the value somehow).
> > I had assumed U-Boot already did that, but it doesn't seem to be the
> > case.

Yes, we've supported the image_size field since v2016.07 and that board
is running v2016.01.  Unfortunately the booting.txt changes that added
the image_size field weren't publicized widely so we didn't see it until
someone else ran into the problem you're describing.

FWIW, it sounds like the Juno board would be a good candidate to be a
board to add testing modern U-Boot on, in kernelci.org.  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20161108/edc91979/attachment.sig>

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Will Deacon]

On Tue, Nov 08, 2016 at 12:51:44PM -0500, Tom Rini wrote:
> On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> > Hello,
> > 
> > We've got a boot problem on arm64 devices. Here is boot log:
> > https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> > https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > 
> > Here is some debugging that Andrey and Mark did:
> > 
> > On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > So we have
> > > Dtb address  0x81f00000
> > > Load address 0x80000000
> > > Which gives us 31Mb for kernel.
> > >
> > > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > > If bss is big enough it might overwrite dtb.
> > > And indeed, build details -
> > > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> > 
> > On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > > header which describes the "real" size of the Image, BSS included. See
> > > [1,2].
> > > It should be possible to modify U-Boot to use that to automatically
> > > place the DTB and initrd at non-clashing locations (or at least to
> > > expose the value somehow).
> > > I had assumed U-Boot already did that, but it doesn't seem to be the
> > > case.
> 
> Yes, we've supported the image_size field since v2016.07 and that board
> is running v2016.01.  Unfortunately the booting.txt changes that added
> the image_size field weren't publicized widely so we didn't see it until
> someone else ran into the problem you're describing.

Hmm, that's a good point. If you like, I could add something to that file
asking for all changes to CC the u-boot list? We should probably do the
same for edk2.

Will

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Tom Rini]

On Tue, Nov 08, 2016 at 06:01:42PM +0000, Will Deacon wrote:
> On Tue, Nov 08, 2016 at 12:51:44PM -0500, Tom Rini wrote:
> > On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> > > Hello,
> > > 
> > > We've got a boot problem on arm64 devices. Here is boot log:
> > > https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> > > https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > 
> > > Here is some debugging that Andrey and Mark did:
> > > 
> > > On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > > > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > So we have
> > > > Dtb address  0x81f00000
> > > > Load address 0x80000000
> > > > Which gives us 31Mb for kernel.
> > > >
> > > > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > > > If bss is big enough it might overwrite dtb.
> > > > And indeed, build details -
> > > > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > > > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> > > 
> > > On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > > > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > > > header which describes the "real" size of the Image, BSS included. See
> > > > [1,2].
> > > > It should be possible to modify U-Boot to use that to automatically
> > > > place the DTB and initrd at non-clashing locations (or at least to
> > > > expose the value somehow).
> > > > I had assumed U-Boot already did that, but it doesn't seem to be the
> > > > case.
> > 
> > Yes, we've supported the image_size field since v2016.07 and that board
> > is running v2016.01.  Unfortunately the booting.txt changes that added
> > the image_size field weren't publicized widely so we didn't see it until
> > someone else ran into the problem you're describing.
> 
> Hmm, that's a good point. If you like, I could add something to that file
> asking for all changes to CC the u-boot list? We should probably do the
> same for edk2.

That would be nice, sure.  I was even thinking that something like the
cross-distro or boot-architecture lists that linaro runs would have had
this info go by as well.  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20161108/42d74654/attachment.sig>

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Will Deacon]

On Tue, Nov 08, 2016 at 01:04:40PM -0500, Tom Rini wrote:
> On Tue, Nov 08, 2016 at 06:01:42PM +0000, Will Deacon wrote:
> > On Tue, Nov 08, 2016 at 12:51:44PM -0500, Tom Rini wrote:
> > > On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> > > > Hello,
> > > > 
> > > > We've got a boot problem on arm64 devices. Here is boot log:
> > > > https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> > > > https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > 
> > > > Here is some debugging that Andrey and Mark did:
> > > > 
> > > > On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > > > > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > So we have
> > > > > Dtb address  0x81f00000
> > > > > Load address 0x80000000
> > > > > Which gives us 31Mb for kernel.
> > > > >
> > > > > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > > > > If bss is big enough it might overwrite dtb.
> > > > > And indeed, build details -
> > > > > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > > > > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> > > > 
> > > > On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > > > > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > > > > header which describes the "real" size of the Image, BSS included. See
> > > > > [1,2].
> > > > > It should be possible to modify U-Boot to use that to automatically
> > > > > place the DTB and initrd at non-clashing locations (or at least to
> > > > > expose the value somehow).
> > > > > I had assumed U-Boot already did that, but it doesn't seem to be the
> > > > > case.
> > > 
> > > Yes, we've supported the image_size field since v2016.07 and that board
> > > is running v2016.01.  Unfortunately the booting.txt changes that added
> > > the image_size field weren't publicized widely so we didn't see it until
> > > someone else ran into the problem you're describing.
> > 
> > Hmm, that's a good point. If you like, I could add something to that file
> > asking for all changes to CC the u-boot list? We should probably do the
> > same for edk2.
> 
> That would be nice, sure.  I was even thinking that something like the
> cross-distro or boot-architecture lists that linaro runs would have had
> this info go by as well.  Thanks!

D'oh, the u-boot list seems to be subscriber-only, so I'm not going to make
a whole lot of friends if I ask kernel devs to add it to cc.

Also, you apparently Acked the image_size change:

http://git.kernel.org/linus/a2c1d73b94ed

so it was at least partially publicized ;)

Will

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Tom Rini]

On Tue, Nov 08, 2016 at 07:34:34PM +0000, Will Deacon wrote:
> On Tue, Nov 08, 2016 at 01:04:40PM -0500, Tom Rini wrote:
> > On Tue, Nov 08, 2016 at 06:01:42PM +0000, Will Deacon wrote:
> > > On Tue, Nov 08, 2016 at 12:51:44PM -0500, Tom Rini wrote:
> > > > On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> > > > > Hello,
> > > > > 
> > > > > We've got a boot problem on arm64 devices. Here is boot log:
> > > > > https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> > > > > https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > 
> > > > > Here is some debugging that Andrey and Mark did:
> > > > > 
> > > > > On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > > > > > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > > So we have
> > > > > > Dtb address  0x81f00000
> > > > > > Load address 0x80000000
> > > > > > Which gives us 31Mb for kernel.
> > > > > >
> > > > > > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > > > > > If bss is big enough it might overwrite dtb.
> > > > > > And indeed, build details -
> > > > > > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > > > > > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> > > > > 
> > > > > On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > > > > > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > > > > > header which describes the "real" size of the Image, BSS included. See
> > > > > > [1,2].
> > > > > > It should be possible to modify U-Boot to use that to automatically
> > > > > > place the DTB and initrd at non-clashing locations (or at least to
> > > > > > expose the value somehow).
> > > > > > I had assumed U-Boot already did that, but it doesn't seem to be the
> > > > > > case.
> > > > 
> > > > Yes, we've supported the image_size field since v2016.07 and that board
> > > > is running v2016.01.  Unfortunately the booting.txt changes that added
> > > > the image_size field weren't publicized widely so we didn't see it until
> > > > someone else ran into the problem you're describing.
> > > 
> > > Hmm, that's a good point. If you like, I could add something to that file
> > > asking for all changes to CC the u-boot list? We should probably do the
> > > same for edk2.
> > 
> > That would be nice, sure.  I was even thinking that something like the
> > cross-distro or boot-architecture lists that linaro runs would have had
> > this info go by as well.  Thanks!
> 
> D'oh, the u-boot list seems to be subscriber-only, so I'm not going to make
> a whole lot of friends if I ask kernel devs to add it to cc.

Ah yes.  So boot-architecture is probably best.

> Also, you apparently Acked the image_size change:
> 
> http://git.kernel.org/linus/a2c1d73b94ed
> 
> so it was at least partially publicized ;)

Ah yes, now I recall :)  In my defense, I ack'd them when it was going
around, but I never saw when it went in.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20161108/ec6a2d22/attachment.sig>

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Will Deacon]

On Tue, Nov 08, 2016 at 02:47:30PM -0500, Tom Rini wrote:
> On Tue, Nov 08, 2016 at 07:34:34PM +0000, Will Deacon wrote:
> > On Tue, Nov 08, 2016 at 01:04:40PM -0500, Tom Rini wrote:
> > > On Tue, Nov 08, 2016 at 06:01:42PM +0000, Will Deacon wrote:
> > > > On Tue, Nov 08, 2016 at 12:51:44PM -0500, Tom Rini wrote:
> > > > > On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> > > > > > Hello,
> > > > > > 
> > > > > > We've got a boot problem on arm64 devices. Here is boot log:
> > > > > > https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> > > > > > https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > > 
> > > > > > Here is some debugging that Andrey and Mark did:
> > > > > > 
> > > > > > On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > > > > > > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > > > So we have
> > > > > > > Dtb address  0x81f00000
> > > > > > > Load address 0x80000000
> > > > > > > Which gives us 31Mb for kernel.
> > > > > > >
> > > > > > > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > > > > > > If bss is big enough it might overwrite dtb.
> > > > > > > And indeed, build details -
> > > > > > > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > > > > > > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> > > > > > 
> > > > > > On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > > > > > > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > > > > > > header which describes the "real" size of the Image, BSS included. See
> > > > > > > [1,2].
> > > > > > > It should be possible to modify U-Boot to use that to automatically
> > > > > > > place the DTB and initrd at non-clashing locations (or at least to
> > > > > > > expose the value somehow).
> > > > > > > I had assumed U-Boot already did that, but it doesn't seem to be the
> > > > > > > case.
> > > > > 
> > > > > Yes, we've supported the image_size field since v2016.07 and that board
> > > > > is running v2016.01.  Unfortunately the booting.txt changes that added
> > > > > the image_size field weren't publicized widely so we didn't see it until
> > > > > someone else ran into the problem you're describing.
> > > > 
> > > > Hmm, that's a good point. If you like, I could add something to that file
> > > > asking for all changes to CC the u-boot list? We should probably do the
> > > > same for edk2.
> > > 
> > > That would be nice, sure.  I was even thinking that something like the
> > > cross-distro or boot-architecture lists that linaro runs would have had
> > > this info go by as well.  Thanks!
> > 
> > D'oh, the u-boot list seems to be subscriber-only, so I'm not going to make
> > a whole lot of friends if I ask kernel devs to add it to cc.
> 
> Ah yes.  So boot-architecture is probably best.

This list: https://lists.linaro.org/pipermail/boot-architecture/ ?

I was about to point out that there have only been 11 posts there in the last
four months, but then I saw the "2015" part :/

Will

[U-Boot] U-Boot overlaps BSS and initrd on arm64

[Tom Rini]

On Tue, Nov 08, 2016 at 11:50:19PM +0000, Will Deacon wrote:
> On Tue, Nov 08, 2016 at 02:47:30PM -0500, Tom Rini wrote:
> > On Tue, Nov 08, 2016 at 07:34:34PM +0000, Will Deacon wrote:
> > > On Tue, Nov 08, 2016 at 01:04:40PM -0500, Tom Rini wrote:
> > > > On Tue, Nov 08, 2016 at 06:01:42PM +0000, Will Deacon wrote:
> > > > > On Tue, Nov 08, 2016 at 12:51:44PM -0500, Tom Rini wrote:
> > > > > > On Tue, Nov 08, 2016 at 09:41:13AM -0800, Dmitry Vyukov wrote:
> > > > > > > Hello,
> > > > > > > 
> > > > > > > We've got a boot problem on arm64 devices. Here is boot log:
> > > > > > > https://storage.kernelci.org/mainline/v4.9-rc4/arm64-defconfig+CONFIG_KASAN=y/lab-baylibre-seattle/boot-juno-r2.txt
> > > > > > > https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > > > 
> > > > > > > Here is some debugging that Andrey and Mark did:
> > > > > > > 
> > > > > > > On Tue, Nov 8, 2016 at 2:00 AM, Andrey Ryabinin wrote:
> > > > > > > > I've looked at juno-r2: https://kernelci.org/boot/id/581ece5a59b514e448f03bd7/
> > > > > > > > So we have
> > > > > > > > Dtb address  0x81f00000
> > > > > > > > Load address 0x80000000
> > > > > > > > Which gives us 31Mb for kernel.
> > > > > > > >
> > > > > > > > It says that Kernel image is 24.62 MiB, but that's without BSS.
> > > > > > > > If bss is big enough it might overwrite dtb.
> > > > > > > > And indeed, build details -
> > > > > > > > https://kernelci.org/build/id/581e850959b514e564f03bdc/
> > > > > > > > shows that bss is 8.5 Mb which is enough to overlap with dtb.
> > > > > > > 
> > > > > > > On Tue, Nov 8, 2016 at 3:21 AM, Mark Rutland wrote:
> > > > > > > > FWIW, since v3.17 we've had an image_size field in the arm64 Image
> > > > > > > > header which describes the "real" size of the Image, BSS included. See
> > > > > > > > [1,2].
> > > > > > > > It should be possible to modify U-Boot to use that to automatically
> > > > > > > > place the DTB and initrd at non-clashing locations (or at least to
> > > > > > > > expose the value somehow).
> > > > > > > > I had assumed U-Boot already did that, but it doesn't seem to be the
> > > > > > > > case.
> > > > > > 
> > > > > > Yes, we've supported the image_size field since v2016.07 and that board
> > > > > > is running v2016.01.  Unfortunately the booting.txt changes that added
> > > > > > the image_size field weren't publicized widely so we didn't see it until
> > > > > > someone else ran into the problem you're describing.
> > > > > 
> > > > > Hmm, that's a good point. If you like, I could add something to that file
> > > > > asking for all changes to CC the u-boot list? We should probably do the
> > > > > same for edk2.
> > > > 
> > > > That would be nice, sure.  I was even thinking that something like the
> > > > cross-distro or boot-architecture lists that linaro runs would have had
> > > > this info go by as well.  Thanks!
> > > 
> > > D'oh, the u-boot list seems to be subscriber-only, so I'm not going to make
> > > a whole lot of friends if I ask kernel devs to add it to cc.
> > 
> > Ah yes.  So boot-architecture is probably best.
> 
> This list: https://lists.linaro.org/pipermail/boot-architecture/ ?
> 
> I was about to point out that there have only been 11 posts there in the last
> four months, but then I saw the "2015" part :/

Yeah, it's not a highly active list.  It feels like it's the appropriate
list for such things, but, eh.  We did almost right last time, I just
didn't follow up and looking at when changes went mainline.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20161111/3387134f/attachment.sig>