[U-Boot] How to test new bootloaders on Jetson TX1?

[U-Boot] How to test new bootloaders on Jetson TX1?

[Andreas Färber]

Hello,

I would like to test the latest version of U-Boot on the Jetson TX1.

Unfortunately U-Boot is lacking a README that would explain how to do that:

http://git.denx.de/?p=u-boot.git;a=tree;f=board/nvidia/p2371-2180;h=097fb9aaec8abd522b7b8382e5c4169f5ea8f691;hb=HEAD


I understand that the Jetson TK1's Python based
https://github.com/NVIDIA/tegra-uboot-flasher-scripts
have never been updated for Tegra X1.

That leaves L4T Jetson TX1 Driver Package, latest version being R28.1.0.

Here's what I have tried:

$ sudo ./flash.sh p2371-2180-devkit mmcblk0p1

On openSUSE Leap 42.3 this keeps failing the first time:

copying
bctfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/BCT/P2180_A00_LP4_DSC_204Mhz.cfg)...
done.
copying
bootloader(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cboot.bin)...
done.
	populating kernel to rootfs... done.
	populating initrd to rootfs... done.
	populating extlinux.conf.emmc to rootfs... done.
	populating
/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
to rootfs... done.
done.
Making Boot image... done.
copying
bcffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cfg/board_config_p2597-devkit.xml)...
done.
Existing
sosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/nvtboot_recovery.bin)
reused.
copying
tegraboot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/nvtboot.bin)...
done.
Existing
bpffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/bpmp.bin)
reused.
copying
wb0boot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/warmboot.bin)...
done.
Existing
tosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/tos.img)
reused.
Existing
eksfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/eks.img)
reused.
copying
dtbfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb)...
done.
Making system.img...
/dev/loop0 is not block device. Terminating..

I have managed to work around that by running sudo losetup -f once.

That gets me to a:

U-Boot 2016.07-g0ce7ca2 (Jul 20 2017 - 00:37:03 -0700)

(where our efiboot command doesn't quite work yet)


https://elinux.org/Jetson/TX1_Upstream_Kernel

claims: "The distributed tarball contains a pre-built U-Boot and kernel,
but these may be replaced with any images you wish to flash, by
replacing the distributed images in the directory tree."
and "To flash only the primary bootloader and U-Boot: ./flash.sh -k EBT
p2371-2180 mmcblk0p1"

Apparently that's not true anymore (R23.1.1 armhf vs. R28.1.0 aarch64):
Operation succeeds, but U-Boot is still the old one.

Possible cause: The EBT partition gets a cboot.bin, not u-boot-dtb.bin.

If I repeat without -k EBT and with the openSUSE OBS builds of U-Boot
v2018.01 then again flashing appears to succeed, but boot gets stuck at:

[0003.260] Starting Bpmp FW
[0003.263] BPMP-FW Carveout: Base = 0xff2c0000 and Size = 0x40000

(short of showing the U-Boot banner)


Also, ATF shows up as:

NOTICE:  BL31: v1.2(release):cc5fd7c
NOTICE:  BL31: Built : 00:37:02, Jul 20 2017

(which is most certainly lacking Jan 2018 Spectre mitigations for CA57)

Security Bulletin
http://nvidia.custhelp.com/app/answers/detail/a_id/4616
announces an R28.2 for later this month, but with t210 in mainline ATF I
would really like to use my own patched ATF build already.

https://github.com/ARM-software/arm-trusted-firmware/blob/6f62574767546b11199142b1b577a86571051c40/docs/plat/nvidia-tegra.rst
doesn't explain how to get the t210 bl31.bin onto a board either.


https://elinux.org/Jetson_TX1 links to multiple App Notes on a
non-reachable Nvidia FTP server. I found that replacing ftp with http,
they can still be reached (dated 2015), but
http://download.nvidia.com/tegra-public-appnotes/t210-nvtboot-flow.html
isn't too helpful either, other than giving a rough overview.


Colleagues have succeeded in loading a U-Boot via RCM into RAM and
executing from there, but I figure that won't quite work for ATF.
And it's no permanent solution anyway.


So could someone please comment on how to perform a minimally-invasive
update of the individual Open Source firmware components for Jetson TX1?

Thanks,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1?

[Mikko Perttunen]

Hi Andreas,

In the cboot + U-Boot combination, cboot loads U-Boot from the usual 
kernel partition (LNX or kernel depending on system), so flashing U-Boot 
there should do the trick. I believe this did indeed change in some L4T 
version, so the wiki page needs to be updated. Tom should know more 
about this.

I expect Varun to know the details about ATF, but I'll check if I can 
find some answer myself.

Thanks,
Mikko

On 02/15/2018 03:51 AM, Andreas Färber wrote:
> Hello,
> 
> I would like to test the latest version of U-Boot on the Jetson TX1.
> 
> Unfortunately U-Boot is lacking a README that would explain how to do that:
> 
> http://git.denx.de/?p=u-boot.git;a=tree;f=board/nvidia/p2371-2180;h=097fb9aaec8abd522b7b8382e5c4169f5ea8f691;hb=HEAD
> 
> 
> I understand that the Jetson TK1's Python based
> https://github.com/NVIDIA/tegra-uboot-flasher-scripts
> have never been updated for Tegra X1.
> 
> That leaves L4T Jetson TX1 Driver Package, latest version being R28.1.0.
> 
> Here's what I have tried:
> 
> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
> 
> On openSUSE Leap 42.3 this keeps failing the first time:
> 
> copying
> bctfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/BCT/P2180_A00_LP4_DSC_204Mhz.cfg)...
> done.
> copying
> bootloader(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cboot.bin)...
> done.
> 	populating kernel to rootfs... done.
> 	populating initrd to rootfs... done.
> 	populating extlinux.conf.emmc to rootfs... done.
> 	populating
> /home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
> to rootfs... done.
> done.
> Making Boot image... done.
> copying
> bcffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/cfg/board_config_p2597-devkit.xml)...
> done.
> Existing
> sosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/nvtboot_recovery.bin)
> reused.
> copying
> tegraboot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/nvtboot.bin)...
> done.
> Existing
> bpffile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/bpmp.bin)
> reused.
> copying
> wb0boot(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/t210ref/warmboot.bin)...
> done.
> Existing
> tosfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/tos.img)
> reused.
> Existing
> eksfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/bootloader/eks.img)
> reused.
> copying
> dtbfile(/home/andreas/MCU/ARM/JetsonTX1/Linux_for_Tegra/kernel/dtb/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb)...
> done.
> Making system.img...
> /dev/loop0 is not block device. Terminating..
> 
> I have managed to work around that by running sudo losetup -f once.
> 
> That gets me to a:
> 
> U-Boot 2016.07-g0ce7ca2 (Jul 20 2017 - 00:37:03 -0700)
> 
> (where our efiboot command doesn't quite work yet)
> 
> 
> https://elinux.org/Jetson/TX1_Upstream_Kernel
> 
> claims: "The distributed tarball contains a pre-built U-Boot and kernel,
> but these may be replaced with any images you wish to flash, by
> replacing the distributed images in the directory tree."
> and "To flash only the primary bootloader and U-Boot: ./flash.sh -k EBT
> p2371-2180 mmcblk0p1"
> 
> Apparently that's not true anymore (R23.1.1 armhf vs. R28.1.0 aarch64):
> Operation succeeds, but U-Boot is still the old one.
> 
> Possible cause: The EBT partition gets a cboot.bin, not u-boot-dtb.bin.
> 
> If I repeat without -k EBT and with the openSUSE OBS builds of U-Boot
> v2018.01 then again flashing appears to succeed, but boot gets stuck at:
> 
> [0003.260] Starting Bpmp FW
> [0003.263] BPMP-FW Carveout: Base = 0xff2c0000 and Size = 0x40000
> 
> (short of showing the U-Boot banner)
> 
> 
> Also, ATF shows up as:
> 
> NOTICE:  BL31: v1.2(release):cc5fd7c
> NOTICE:  BL31: Built : 00:37:02, Jul 20 2017
> 
> (which is most certainly lacking Jan 2018 Spectre mitigations for CA57)
> 
> Security Bulletin
> http://nvidia.custhelp.com/app/answers/detail/a_id/4616
> announces an R28.2 for later this month, but with t210 in mainline ATF I
> would really like to use my own patched ATF build already.
> 
> https://github.com/ARM-software/arm-trusted-firmware/blob/6f62574767546b11199142b1b577a86571051c40/docs/plat/nvidia-tegra.rst
> doesn't explain how to get the t210 bl31.bin onto a board either.
> 
> 
> https://elinux.org/Jetson_TX1 links to multiple App Notes on a
> non-reachable Nvidia FTP server. I found that replacing ftp with http,
> they can still be reached (dated 2015), but
> http://download.nvidia.com/tegra-public-appnotes/t210-nvtboot-flow.html
> isn't too helpful either, other than giving a rough overview.
> 
> 
> Colleagues have succeeded in loading a U-Boot via RCM into RAM and
> executing from there, but I figure that won't quite work for ATF.
> And it's no permanent solution anyway.
> 
> 
> So could someone please comment on how to perform a minimally-invasive
> update of the individual Open Source firmware components for Jetson TX1?
> 
> Thanks,
> Andreas
> 

[U-Boot] How to test new bootloaders on Jetson TX1?

[Andreas Färber]

Hi Mikko,

Am 15.02.2018 um 08:56 schrieb Mikko Perttunen:
> In the cboot + U-Boot combination, cboot loads U-Boot from the usual
> kernel partition (LNX or kernel depending on system), so flashing U-Boot
> there should do the trick. I believe this did indeed change in some L4T
> version, so the wiki page needs to be updated. Tom should know more
> about this.

Thanks for explaining.

The LNX partition is getting a boot.img - what is the relation to the
four U-Boot files? flash.sh source is not really helping here.

Might any changes to RP1 and/or DTB partitions be necessary to match my
newer U-Boot, or does U-Boot use an internal .dtb these days?

Is there any more efficient way to flash just U-Boot? -k LNX possibly?

I had played with the -L option before (which mentions u-boot-dtb.bin),
but recall it ran into some form of checksum error on boot when passing
it my file...

> I expect Varun to know the details about ATF, but I'll check if I can
> find some answer myself.

Thanks again for your efforts.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1?

[Mikko Perttunen]

On 15.02.2018 15:25, Andreas Färber wrote:
> Hi Mikko,
>
> Am 15.02.2018 um 08:56 schrieb Mikko Perttunen:
>> In the cboot + U-Boot combination, cboot loads U-Boot from the usual
>> kernel partition (LNX or kernel depending on system), so flashing U-Boot
>> there should do the trick. I believe this did indeed change in some L4T
>> version, so the wiki page needs to be updated. Tom should know more
>> about this.
>
> Thanks for explaining.
>
> The LNX partition is getting a boot.img - what is the relation to the
> four U-Boot files? flash.sh source is not really helping here.

cboot is only capable of booting Android .img's so the U-Boot binary 
(u-boot-dtb.bin according ton Jon) is packaged as one using mkbootimg or 
similar - I expect flash.sh to do this automatically.

>
> Might any changes to RP1 and/or DTB partitions be necessary to match my
> newer U-Boot, or does U-Boot use an internal .dtb these days?

My understanding is that it uses an internal .dtb.

>
> Is there any more efficient way to flash just U-Boot? -k LNX possibly?

That might work. Jon should know the details.

>
> I had played with the -L option before (which mentions u-boot-dtb.bin),
> but recall it ran into some form of checksum error on boot when passing
> it my file...
>
>> I expect Varun to know the details about ATF, but I'll check if I can
>> find some answer myself.
>
> Thanks again for your efforts.

I found out that ATF and optionally a secure OS (Trusty usually) are 
contained in the file tos.img. The image contains a padded header, 
followed by bl31.bin and the secure OS binary concatenated.

The header format should be pretty straightforward to reverse-engineer 
(need to change some fields specifying lengths), but I'll ask around if 
we can get a script to generate it released.

Cheers,
Mikko

>
> Regards,
> Andreas
>

[U-Boot] How to test new bootloaders on Jetson TX1?

[Jon Hunter]

On 15/02/18 01:51, Andreas Färber wrote:
> Hello,
> 
> I would like to test the latest version of U-Boot on the Jetson TX1.
> 
> Unfortunately U-Boot is lacking a README that would explain how to do that:
> 
> http://git.denx.de/?p=u-boot.git;a=tree;f=board/nvidia/p2371-2180;h=097fb9aaec8abd522b7b8382e5c4169f5ea8f691;hb=HEAD
> 
> 
> I understand that the Jetson TK1's Python based
> https://github.com/NVIDIA/tegra-uboot-flasher-scripts
> have never been updated for Tegra X1.
> 
> That leaves L4T Jetson TX1 Driver Package, latest version being R28.1.0.
> 
> Here's what I have tried:
> 
> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
This should work. Which u-boot binary are you copying and where are you
copying it?

I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory
Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.

Cheers
Jon

-- 
nvpublic

[U-Boot] How to test new bootloaders on Jetson TX1?

[Andreas Färber]

Am 15.02.2018 um 10:22 schrieb Jon Hunter:
> On 15/02/18 01:51, Andreas Färber wrote:
>> I would like to test the latest version of U-Boot on the Jetson TX1.
[...]
>> Here's what I have tried:
>>
>> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
> This should work. Which u-boot binary are you copying and where are you
> copying it?
> 
> I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory
> Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.

I downloaded the .rpm from
https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180
extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to
bootloader/t210ref/p2371-2180/ - and as described it makes a difference
in that it then ceases to boot to a U-Boot prompt.

I then have to use jumper J9 to enter RCM again.

So something is getting flashed, it's just not working right.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1?

[Jon Hunter]

On 15/02/18 12:32, Andreas Färber wrote:
> Am 15.02.2018 um 10:22 schrieb Jon Hunter:
>> On 15/02/18 01:51, Andreas Färber wrote:
>>> I would like to test the latest version of U-Boot on the Jetson TX1.
> [...]
>>> Here's what I have tried:
>>>
>>> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
>> This should work. Which u-boot binary are you copying and where are you
>> copying it?
>>
>> I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory
>> Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
> 
> I downloaded the .rpm from
> https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180
> extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to
> bootloader/t210ref/p2371-2180/ - and as described it makes a difference
> in that it then ceases to boot to a U-Boot prompt.

Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However,
I believe you are still booting the wrong u-boot file.

If you look at the p2371-2180-devkit.conf you will see that it has ...

 DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin

For using the upstream u-boot, we need to use the u-boot-dtb.bin and not
the u-boot.bin. So you can either ...

1. Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that you
copied.
2. Move the u-boot-dtb.bin from your rpm to
bootloader/t210ref/p2371-2180/u-boot.bin.

Cheers
Jon

-- 
nvpublic

[U-Boot] How to test new bootloaders on Jetson TX1?

[Andreas Färber]

Hi Jon,

Am 15.02.2018 um 15:01 schrieb Jon Hunter:
> On 15/02/18 12:32, Andreas Färber wrote:
>> Am 15.02.2018 um 10:22 schrieb Jon Hunter:
>>> On 15/02/18 01:51, Andreas Färber wrote:
>>>> I would like to test the latest version of U-Boot on the Jetson TX1.
>> [...]
>>>> Here's what I have tried:
>>>>
>>>> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
>>> This should work. Which u-boot binary are you copying and where are you
>>> copying it?
>>>
>>> I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory
>>> Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
>>
>> I downloaded the .rpm from
>> https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180
>> extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to
>> bootloader/t210ref/p2371-2180/ - and as described it makes a difference
>> in that it then ceases to boot to a U-Boot prompt.
> 
> Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However,
> I believe you are still booting the wrong u-boot file.
> 
> If you look at the p2371-2180-devkit.conf you will see that it has ...
> 
>  DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
> 
> For using the upstream u-boot, we need to use the u-boot-dtb.bin and not
> the u-boot.bin. So you can either ...
> 
> 1. Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that you
> copied.
> 2. Move the u-boot-dtb.bin from your rpm to
> bootloader/t210ref/p2371-2180/u-boot.bin.

Both in the Nvidia tarball and in our upstream based package builds,
u-boot.bin and u-boot-dtb.bin are identical. Since some time
u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.

Should this work with the vanilla upstream files, or do they need any
headers or signatures or something?

Do any of you have that upstream version booting successfully, or might
it be an upstream U-Boot regression? I was so far assuming it's a user
error, so haven't tried bisecting yet.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1?

[Jon Hunter]

On 15/02/18 15:12, Andreas Färber wrote:
> Hi Jon,
> 
> Am 15.02.2018 um 15:01 schrieb Jon Hunter:
>> On 15/02/18 12:32, Andreas Färber wrote:
>>> Am 15.02.2018 um 10:22 schrieb Jon Hunter:
>>>> On 15/02/18 01:51, Andreas Färber wrote:
>>>>> I would like to test the latest version of U-Boot on the Jetson TX1.
>>> [...]
>>>>> Here's what I have tried:
>>>>>
>>>>> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
>>>> This should work. Which u-boot binary are you copying and where are you
>>>> copying it?
>>>>
>>>> I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory
>>>> Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
>>>
>>> I downloaded the .rpm from
>>> https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2180
>>> extracting all of u-boot, u-boot.bin, u-boot.dtb and u-boot-dtb.bin to
>>> bootloader/t210ref/p2371-2180/ - and as described it makes a difference
>>> in that it then ceases to boot to a U-Boot prompt.
>>
>> Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. However,
>> I believe you are still booting the wrong u-boot file.
>>
>> If you look at the p2371-2180-devkit.conf you will see that it has ...
>>
>>  DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
>>
>> For using the upstream u-boot, we need to use the u-boot-dtb.bin and not
>> the u-boot.bin. So you can either ...
>>
>> 1. Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that you
>> copied.
>> 2. Move the u-boot-dtb.bin from your rpm to
>> bootloader/t210ref/p2371-2180/u-boot.bin.
> 
> Both in the Nvidia tarball and in our upstream based package builds,
> u-boot.bin and u-boot-dtb.bin are identical. Since some time
> u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.

Good to know!

> Should this work with the vanilla upstream files, or do they need any
> headers or signatures or something?

Yes should work fine with vanilla upstream.

> Do any of you have that upstream version booting successfully, or might
> it be an upstream U-Boot regression? I was so far assuming it's a user
> error, so haven't tried bisecting yet.

I am using "U-Boot 2017.03". However, I realise that I am currently
based upon L4T rel24.2.1 (as rel28.1 was not available when I started
using L4T for testing upstream). I hope it would not matter, but you
never know. I plan to move to rel28.2 once it is released. If we think
it is a problem with rel28.1 I should be able to test my side.

Cheers
Jon

-- 
nvpublic

[U-Boot] How to test new bootloaders on Jetson TX1?

[Tom Warren]

-----Original Message-----
From: Jonathan Hunter 
Sent: Thursday, February 15, 2018 8:39 AM
To: Andreas F채rber <afaerber@suse.de>
Cc: linux-tegra at vger.kernel.org; U-Boot <u-boot@lists.denx.de>; Alexander Graf <agraf@suse.de>; Mian Yousaf Kaukab <yousaf.kaukab@suse.com>; Tom Warren <TWarren@nvidia.com>; Varun Wadekar <vwadekar@nvidia.com>
Subject: Re: How to test new bootloaders on Jetson TX1?


On 15/02/18 15:12, Andreas F채rber wrote:
> Hi Jon,
> 
> Am 15.02.2018 um 15:01 schrieb Jon Hunter:
>> On 15/02/18 12:32, Andreas F채rber wrote:
>>> Am 15.02.2018 um 10:22 schrieb Jon Hunter:
>>>> On 15/02/18 01:51, Andreas F채rber wrote:
>>>>> I would like to test the latest version of U-Boot on the Jetson TX1.
>>> [...]
>>>>> Here's what I have tried:
>>>>>
>>>>> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
>>>> This should work. Which u-boot binary are you copying and where are 
>>>> you copying it?
>>>>
>>>> I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory 
>>>> Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
>>>
>>> I downloaded the .rpm from
>>> https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2
>>> 180 extracting all of u-boot, u-boot.bin, u-boot.dtb and 
>>> u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described 
>>> it makes a difference in that it then ceases to boot to a U-Boot 
>>> prompt.
>>
>> Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. 
>> However, I believe you are still booting the wrong u-boot file.
>>
>> If you look at the p2371-2180-devkit.conf you will see that it has ...
>>
>>  DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
>>
>> For using the upstream u-boot, we need to use the u-boot-dtb.bin and 
>> not the u-boot.bin. So you can either ...
>>
>> 1. Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that 
>> you copied.
>> 2. Move the u-boot-dtb.bin from your rpm to 
>> bootloader/t210ref/p2371-2180/u-boot.bin.
> 
> Both in the Nvidia tarball and in our upstream based package builds, 
> u-boot.bin and u-boot-dtb.bin are identical. Since some time 
> u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.

Good to know!

> Should this work with the vanilla upstream files, or do they need any 
> headers or signatures or something?

Yes should work fine with vanilla upstream.

> Do any of you have that upstream version booting successfully, or 
> might it be an upstream U-Boot regression? I was so far assuming it's 
> a user error, so haven't tried bisecting yet.

I am using "U-Boot 2017.03". However, I realise that I am currently based upon L4T rel24.2.1 (as rel28.1 was not available when I started using L4T for testing upstream). I hope it would not matter, but you never know. I plan to move to rel28.2 once it is released. If we think it is a problem with rel28.1 I should be able to test my side.

[Tom] Current (R28.x) U-Boot requires CBoot to load - we no longer use NVTBoot as we did in R24.x.  Hence, upstream U-Boot needs some modifications to be able to be loaded by CBoot instead of NVTBoot.  I haven't done that work yet (patches to convert upstream Denx U-Boot source for T210/T186 to use CBoot as the loader - don't know how I'll structure it so old (NVTBoot) bootflow will still work as well as the new (CBoot) bootflow).  I've got it on my plate, but other priorities have prevented me from working on it. I'll try to get it into the queue.  Until then, you won't be able to load/run upstream U-Boot using the boot flow in R28.x. Sorry.


Cheers
Jon

--
nvpublic

[U-Boot] How to test new bootloaders on Jetson TX1?

[Varun Wadekar]

Andreas, can you try the TOS packaging script available in our public repo?

http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=tools/gen_tos_part_img.py;h=47828f7028e56a6cffb9b773502b13dc431a015e;hb=HEAD

Please let me know if this does not work for you.

For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.

-----Original Message-----
From: Tom Warren 
Sent: Thursday, February 15, 2018 7:44 AM
To: Jonathan Hunter <jonathanh@nvidia.com>; Andreas Färber <afaerber@suse.de>
Cc: linux-tegra at vger.kernel.org; U-Boot <u-boot@lists.denx.de>; Alexander Graf <agraf@suse.de>; Mian Yousaf Kaukab <yousaf.kaukab@suse.com>; Varun Wadekar <vwadekar@nvidia.com>
Subject: RE: How to test new bootloaders on Jetson TX1?



-----Original Message-----
From: Jonathan Hunter 
Sent: Thursday, February 15, 2018 8:39 AM
To: Andreas Färber <afaerber@suse.de>
Cc: linux-tegra at vger.kernel.org; U-Boot <u-boot@lists.denx.de>; Alexander Graf <agraf@suse.de>; Mian Yousaf Kaukab <yousaf.kaukab@suse.com>; Tom Warren <TWarren@nvidia.com>; Varun Wadekar <vwadekar@nvidia.com>
Subject: Re: How to test new bootloaders on Jetson TX1?


On 15/02/18 15:12, Andreas Färber wrote:
> Hi Jon,
> 
> Am 15.02.2018 um 15:01 schrieb Jon Hunter:
>> On 15/02/18 12:32, Andreas Färber wrote:
>>> Am 15.02.2018 um 10:22 schrieb Jon Hunter:
>>>> On 15/02/18 01:51, Andreas Färber wrote:
>>>>> I would like to test the latest version of U-Boot on the Jetson TX1.
>>> [...]
>>>>> Here's what I have tried:
>>>>>
>>>>> $ sudo ./flash.sh p2371-2180-devkit mmcblk0p1
>>>> This should work. Which u-boot binary are you copying and where are 
>>>> you copying it?
>>>>
>>>> I copy the u-boot-dtb.bin over u-boot.bin in the L4T directory 
>>>> Linux_for_Tegra/bootloader/t186ref/p2771-0000/500/u-boot.bin.
>>>
>>> I downloaded the .rpm from
>>> https://build.opensuse.org/package/show/hardware:boot/u-boot-p2371-2
>>> 180 extracting all of u-boot, u-boot.bin, u-boot.dtb and 
>>> u-boot-dtb.bin to bootloader/t210ref/p2371-2180/ - and as described 
>>> it makes a difference in that it then ceases to boot to a U-Boot 
>>> prompt.
>>
>> Sorry, looks like I mixed TX1 and TX2. I see you are using TX1. 
>> However, I believe you are still booting the wrong u-boot file.
>>
>> If you look at the p2371-2180-devkit.conf you will see that it has ...
>>
>>  DFLT_KERNEL_IMAGE="bootloader/${target_board}/p2371-2180/u-boot.bin
>>
>> For using the upstream u-boot, we need to use the u-boot-dtb.bin and 
>> not the u-boot.bin. So you can either ...
>>
>> 1. Update the p2371-2180-devkit.conf to use the u-boot-dtb.bin that 
>> you copied.
>> 2. Move the u-boot-dtb.bin from your rpm to 
>> bootloader/t210ref/p2371-2180/u-boot.bin.
> 
> Both in the Nvidia tarball and in our upstream based package builds, 
> u-boot.bin and u-boot-dtb.bin are identical. Since some time 
> u-boot-dtb.bin gets copied over as u-boot.bin for consistency upstream.

Good to know!

> Should this work with the vanilla upstream files, or do they need any 
> headers or signatures or something?

Yes should work fine with vanilla upstream.

> Do any of you have that upstream version booting successfully, or 
> might it be an upstream U-Boot regression? I was so far assuming it's 
> a user error, so haven't tried bisecting yet.

I am using "U-Boot 2017.03". However, I realise that I am currently based upon L4T rel24.2.1 (as rel28.1 was not available when I started using L4T for testing upstream). I hope it would not matter, but you never know. I plan to move to rel28.2 once it is released. If we think it is a problem with rel28.1 I should be able to test my side.

[Tom] Current (R28.x) U-Boot requires CBoot to load - we no longer use NVTBoot as we did in R24.x.  Hence, upstream U-Boot needs some modifications to be able to be loaded by CBoot instead of NVTBoot.  I haven't done that work yet (patches to convert upstream Denx U-Boot source for T210/T186 to use CBoot as the loader - don't know how I'll structure it so old (NVTBoot) bootflow will still work as well as the new (CBoot) bootflow).  I've got it on my plate, but other priorities have prevented me from working on it. I'll try to get it into the queue.  Until then, you won't be able to load/run upstream U-Boot using the boot flow in R28.x. Sorry.


Cheers
Jon

--
nvpublic

[U-Boot] How to test new bootloaders on Jetson TX1? - ATF

[Andreas Färber]

Hi Varun,

Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
> Andreas, can you try the TOS packaging script available in our public repo?
> 
> http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=tools/gen_tos_part_img.py;h=47828f7028e56a6cffb9b773502b13dc431a015e;hb=HEAD

Great, that script does work. It is lacking usage output, but looking at
the code, its arguments were self-documenting.

> Please let me know if this does not work for you.
> 
> For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.

I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports,
without SPD. BL31 appears to initialize okay, but later something runs
into an unhandled SMC 0x82000015 - things then go south and it doesn't
reach the Nvidia U-Boot. Serial log attached.

According to
https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-sip-service.rst
that SMC function ID falls into the SiP range, so indeed something
Nvidia-specific missing in v1.4?

Regards,
Andreas

https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-tegra210

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: boot.mine.tos.log
Type: text/x-log
Size: 18998 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180216/acd3cf4f/attachment.bin>

[U-Boot] How to test new bootloaders on Jetson TX1? - ATF

[Varun Wadekar]

Yes. That's a custom SMC we have for some non-L4T uses. It has not been upstreamed yet.

You can add dummy handling in tegra_sip_calls.c to move forward.

________________________________
From: Andreas Färber <afaerber@suse.de>
Sent: Thursday, February 15, 2018 7:04 PM
To: Varun Wadekar
Cc: Tom Warren; Jonathan Hunter; linux-tegra at vger.kernel.org; U-Boot; Alexander Graf; Mian Yousaf Kaukab
Subject: Re: How to test new bootloaders on Jetson TX1? - ATF

Hi Varun,

Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
> Andreas, can you try the TOS packaging script available in our public repo?
>
> http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=tools/gen_tos_part_img.py;h=47828f7028e56a6cffb9b773502b13dc431a015e;hb=HEAD

Great, that script does work. It is lacking usage output, but looking at
the code, its arguments were self-documenting.

> Please let me know if this does not work for you.
>
> For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.

I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports,
without SPD. BL31 appears to initialize okay, but later something runs
into an unhandled SMC 0x82000015 - things then go south and it doesn't
reach the Nvidia U-Boot. Serial log attached.

According to
https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-sip-service.rst
that SMC function ID falls into the SiP range, so indeed something
Nvidia-specific missing in v1.4?

Regards,
Andreas

https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-tegra210

--
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1? - ATF

[Andreas Färber]

Am 16.02.2018 um 07:18 schrieb Varun Wadekar:
> Yes. That's a custom SMC we have for some non-L4T uses. It has not been
> upstreamed yet.
> 
> You can add dummy handling in tegra_sip_calls.c to move forward.

Thanks, on my 1.4 based branch I succeeded to boot into Nvidia U-Boot
with the following change:

--- a/plat/nvidia/tegra/common/tegra_sip_calls.c
+++ b/plat/nvidia/tegra/common/tegra_sip_calls.c
@@ -162,6 +162,11 @@ uint64_t tegra_sip_handler(uint32_t smc_fid,
                 */
                break;

+       case 0x82000015:
+               INFO("%s: SMC 0x%x\n", __func__, smc_fid);
+               SMC_RET1(handle, 0);
+               break;
+
        default:
                ERROR("%s: unhandled SMC (0x%x)\n", __func__, smc_fid);
                break;

The INFO output shows up exactly once.

On master branch I've further needed to fix a regression:
https://github.com/ARM-software/arm-trusted-firmware/pull/1271

Regards,
Andreas

> 
> ------------------------------------------------------------------------
> *From:* Andreas Färber <afaerber@suse.de>
> *Sent:* Thursday, February 15, 2018 7:04 PM
> *To:* Varun Wadekar
> *Cc:* Tom Warren; Jonathan Hunter; linux-tegra at vger.kernel.org; U-Boot;
> Alexander Graf; Mian Yousaf Kaukab
> *Subject:* Re: How to test new bootloaders on Jetson TX1? - ATF
> 
> Hi Varun,
> 
> Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
>> Andreas, can you try the TOS packaging script available in our public repo?
>> 
>> http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=tools/gen_tos_part_img.py;h=47828f7028e56a6cffb9b773502b13dc431a015e;hb=HEAD
> 
> Great, that script does work. It is lacking usage output, but looking at
> the code, its arguments were self-documenting.
> 
>> Please let me know if this does not work for you.
>> 
>> For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
> 
> I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports,
> without SPD. BL31 appears to initialize okay, but later something runs
> into an unhandled SMC 0x82000015 - things then go south and it doesn't
> reach the Nvidia U-Boot. Serial log attached.
> 
> According to
> https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-sip-service.rst
> that SMC function ID falls into the SiP range, so indeed something
> Nvidia-specific missing in v1.4?
> 
> Regards,
> Andreas
> 
> https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-tegra210

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1? - ATF

[Varun Wadekar]

Sounds good. I reviewed the ATF change upstream. Looks good.

________________________________
From: Andreas Färber <afaerber@suse.de>
Sent: Friday, February 16, 2018 7:34 PM
To: Varun Wadekar
Cc: Tom Warren; Jonathan Hunter; linux-tegra at vger.kernel.org; U-Boot; Alexander Graf; Mian Yousaf Kaukab
Subject: Re: How to test new bootloaders on Jetson TX1? - ATF

Am 16.02.2018 um 07:18 schrieb Varun Wadekar:
> Yes. That's a custom SMC we have for some non-L4T uses. It has not been
> upstreamed yet.
>
> You can add dummy handling in tegra_sip_calls.c to move forward.

Thanks, on my 1.4 based branch I succeeded to boot into Nvidia U-Boot
with the following change:

--- a/plat/nvidia/tegra/common/tegra_sip_calls.c
+++ b/plat/nvidia/tegra/common/tegra_sip_calls.c
@@ -162,6 +162,11 @@ uint64_t tegra_sip_handler(uint32_t smc_fid,
                 */
                break;

+       case 0x82000015:
+               INFO("%s: SMC 0x%x\n", __func__, smc_fid);
+               SMC_RET1(handle, 0);
+               break;
+
        default:
                ERROR("%s: unhandled SMC (0x%x)\n", __func__, smc_fid);
                break;

The INFO output shows up exactly once.

On master branch I've further needed to fix a regression:
https://github.com/ARM-software/arm-trusted-firmware/pull/1271

Regards,
Andreas

>
> ------------------------------------------------------------------------
> *From:* Andreas Färber <afaerber@suse.de>
> *Sent:* Thursday, February 15, 2018 7:04 PM
> *To:* Varun Wadekar
> *Cc:* Tom Warren; Jonathan Hunter; linux-tegra at vger.kernel.org; U-Boot;
> Alexander Graf; Mian Yousaf Kaukab
> *Subject:* Re: How to test new bootloaders on Jetson TX1? - ATF
>
> Hi Varun,
>
> Am 15.02.2018 um 17:57 schrieb Varun Wadekar:
>> Andreas, can you try the TOS packaging script available in our public repo?
>>
>> http://nv-tegra.nvidia.com/gitweb/?p=3rdparty/ote_partner/tlk.git;a=blob;f=tools/gen_tos_part_img.py;h=47828f7028e56a6cffb9b773502b13dc431a015e;hb=HEAD
>
> Great, that script does work. It is lacking usage output, but looking at
> the code, its arguments were self-documenting.
>
>> Please let me know if this does not work for you.
>>
>> For the upstream ATF code, our downstream has not caught up with upstream yet, so I am not sure if upstream would directly work for TX1. But its definitely worth a try.
>
> I tried R28.1 flash.sh -k TOS with my ATF v1.4 with Spectre backports,
> without SPD. BL31 appears to initialize okay, but later something runs
> into an unhandled SMC 0x82000015 - things then go south and it doesn't
> reach the Nvidia U-Boot. Serial log attached.
>
> According to
> https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/arm-sip-service.rst
> that SMC function ID falls into the SiP range, so indeed something
> Nvidia-specific missing in v1.4?
>
> Regards,
> Andreas
>
> https://build.opensuse.org/package/show/hardware:boot/arm-trusted-firmware-tegra210

--
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

[U-Boot] How to test new bootloaders on Jetson TX1?

[Stephen Warren]

On 02/14/2018 06:51 PM, Andreas Färber wrote:
> Hello,
> 
> I would like to test the latest version of U-Boot on the Jetson TX1.
> 
> Unfortunately U-Boot is lacking a README that would explain how to do that:
 > ...

Here is some consolidated background on U-Boot on TX1:

In all cases, U-Boot uses a built-in DTB for its own driver 
initialization and other operation.

In T124 and earlier, it was possible to have a completely OSS boot 
process. So, U-Boot could be the only bootloader. In this case, part of 
U-Boot (the SPL) would run on the ARM7/AVP/COP, and part on the main CPU 
(CCPLEX). Details of all that are at:

https://http.download.nvidia.com/tegra-public-appnotes/index.html

With T210, the boot logic that runs on the ARM7/AVP/BPMP-Lite became 
more complex, so we elected to drop the U-Boot SPL and re-use the NVIDIA 
binary bootloader for the AVP/BPMP-Lite portion to avoid re-implementing 
it, leaving U-Boot to run solely on the CCPLEX, and dropping usage of 
U-Boot SPL.

In L4T r24 and earlier, U-Boot was 99.9% of the boot code that ran at 
EL2 or lower on the CCPLEX (ATF - ARM Trusted Firmware - would run at 
EL3). In this case, the other 0.1% of the code running at EL2 runs 
before U-Boot and passes to U-Boot some parameter block with details 
such as memory size, carveouts, etc. The L4T r24 T210 port of U-Boot 
parses this parameter block to determine RAM size, carveout size, etc., 
so is closely tied to this boot model. The upstream T210 port (as of 
today at least) ignores the parameter block, determines RAM size from HW 
registers, and hard-codes some overly-large carveout size to avoid 
trampling any carveout. In this r24 boot model, U-Boot (L4T downstream 
or upstream) loads a DTB from disk, does a little manipulation of it 
(e.g. fills in RAM size, kernel command-line), and passes it a kernel 
that it loads from disk.

With L4T r24, you can use the script bootloader/exec-uboot.sh to load 
and run a new U-Boot over the USB port's recovery protocol, without 
writing anything to flash. This is what my automated upstream U-Boot 
tester does for usptream branches for T210.

I won't discuss L4T r25/r26 since they weren't broadly distributed and 
had slightly different boot models, and discussing them would just be 
confusing.

With L4T r28, we aligned the T210 and T186 boot models. Now, cboot (an 
NVIDIA binary bootloader) always runs on the CCPLEX at EL2 (ATF still 
runs EL3) and performs most general system setup. cboot loads the 
"kernel" from a dedicated partition (which contains no filesystem). This 
kernel could actually be a Linux kernel, but L4T typically places U-Boot 
into this partition and hence chain-loads it. In this model, U-Boot 
receives a DTB from cboot, which is parsed by U-Boot to determine RAM 
size, carveout size, etc. For L4T, U-Boot passes this same DTB on to 
whatever kernel it loads and boots (with a couple minor modifications 
such as over-writing RAM size and kernel command-line). The advantage of 
passing on the DTB is that any DTB edits performed by cboot don't need 
to be re-implemnted by U-Boot. For upstream, people typically (always?) 
have U-Boot load a different DTB to pass to the kernel, since the DTB 
that cboot uses-and-passes-on must use some downstream-specific DT 
bindings/schema that are not appropriate to pass to an upstream kernel 
that uses upstream DT bindings. Only downstream L4T r28 U-Boot supports 
this boot model; upstream U-Boot is not written to accept and parse a 
DTB at runtime. You can control whether U-Boot passes on the DTB or 
loads a new one by (a) extlinux.conf: include a DTB statement or not, 
(b) in other scripts, by either loading a DTB in the script and passing 
that address to booti or passing the address of the cboot-supplied DTB 
to booti; U-Boot sets a variable to point to the cboot-supplied DTB 
making this easy. When running L4T r28, if you want update U-Boot, 
you'll need to flash the kernel partition (T210: LNX, T186: kernel) 
since that's where U-Boot is stored. Obviously L4T's r28 U-Boot release 
works with this model. I'd actually expect upstream to do so too, 
although that combination isn't actually tested; my automated upstream 
U-Boot tester still uses L4T r24 since that boot process is what the 
upstream T210 port was originally developed against. This may change if 
we switch upstream U-Boot to the new boot style.

With L4T r28, there is no bootloader/exec-uboot.sh script, so you must 
write any updated U-Boot to flash in order to test it.

For T186, this new boot model was all we ever supported; cboot is always 
used, U-Boot if uses is stored in the kernel partition, and upstream 
T186 U-Boot only supports the new L4T r28 boot model.

ATF and optionally a secure OS are part of "tos.img" in the L4T release, 
which gets flashed to the TOS partition on T210 and secure-os partition 
on T186. As Varun mentioned, there's a header format for this partition 
and you can use the script he linked in order to generate the header. I 
don't know of anyone using upstream ATF on T210 or T186.

Hopefully that addresses all the questions in this thread; if not let me 
know!