[U-Boot] scripts: RPi 2: only 1 out of 4 CPUs brought up

[U-Boot] scripts: RPi 2: only 1 out of 4 CPUs brought up

[Jonas Jensen]

Hello,

I have found the following issue with RPi 2:

Only 1 CPU is brought up when the kernel is started from script (see [1]).

All 4 CPUs are brought up if started "manually" typing in environment
variables from said script (see [2]).

U-Boot: 2015.04-rc5-gd6d345b
kernel: d9fa11803a0d2e3b1eaf4041ddd0ca5651300d96 branch rpi-4.1.y from
https://github.com/raspberrypi/linux

[1] Automatic boot: http://paste.debian.net/269052/
[2] Manual boot: http://paste.debian.net/269053/
[3] boot.scr.uimg: http://paste.debian.net/269066/
[4] environment variables: http://paste.debian.net/269067/

Example 1:

load mmc 0:1 ${scriptaddr} boot.scr.uimg; source ${scriptaddr}

.. result in linux bringing up only the first CPU

Example 2:

U-Boot> setenv fdtfile bcm2709-rpi-2-b.dtb
U-Boot> setenv bootargs earlyprintk console=tty0 console=ttyAMA0
root=/dev/mmcblk0p2 rootwait rw
U-Boot> fatload mmc 0:1 ${kernel_addr_r} zImage
U-Boot> fatload mmc 0:1 ${fdt_addr_r} ${fdtfile}
U-Boot> bootz ${kernel_addr_r} - ${fdt_addr_r}

.. result in linux bringing up all four CPUs

#u-boot Marex_ suggested adding "setenv fdt_high 0xffffffff" and
adding "sleep 1" between script commands none of which helped fix the
problem.

Is there a known fix or workaround?

I prefer something like boot.scr.uimg that can be added as a single
step enabling the correct environment.


Regards,
   Jonas

[U-Boot] scripts: RPi 2: only 1 out of 4 CPUs brought up

[Stephen Warren]

On 06/30/2015 05:56 AM, Jonas Jensen wrote:
> Hello,
> 
> I have found the following issue with RPi 2:
> 
> Only 1 CPU is brought up when the kernel is started from script (see [1]).
> 
> All 4 CPUs are brought up if started "manually" typing in environment
> variables from said script (see [2]).

I suspect it's a fluke that it works when you run things manually/slowly.

I /think/ that before the binary FW starts U-Boot (or whatever it
boots), it releases all 4 CPUs from reset, and CPU1..n all end up
running a tiny piece of code ("SMP pen") that just loops doing nothing
until some flag is set. All code that runs on CPU0 must be careful not
to corrupt that code or the flag it uses. However, U-Boot (and I expect
the upstream kernel) don't yet know about this, and hence
sometimes/often corrupt that SMP pen, resulting in strange behaviour.
I'm pretty sure I've seen all 4 CPUs start booting the Linux kernel in
parallel before.

In summary, I know there are issues in this area when using U-Boot. I
don't know of any fix at present. Either U-Boot must hard-code some
reserved memory regions, or perhaps the binary FW has some way of
informing SW where the SMP pen region is, and U-Boot needs to learn how
to find/interpret that information (and pass it to the kernel via
/memreserve/ or similar in DT).

Eric, did you find out any more details on the SMP pen mechanism since I
last asked you about it?

[U-Boot] scripts: RPi 2: only 1 out of 4 CPUs brought up

[Eric Anholt]

Stephen Warren <swarren@wwwdotorg.org> writes:

> On 06/30/2015 05:56 AM, Jonas Jensen wrote:
>> Hello,
>> 
>> I have found the following issue with RPi 2:
>> 
>> Only 1 CPU is brought up when the kernel is started from script (see [1]).
>> 
>> All 4 CPUs are brought up if started "manually" typing in environment
>> variables from said script (see [2]).
>
> I suspect it's a fluke that it works when you run things manually/slowly.
>
> I /think/ that before the binary FW starts U-Boot (or whatever it
> boots), it releases all 4 CPUs from reset, and CPU1..n all end up
> running a tiny piece of code ("SMP pen") that just loops doing nothing
> until some flag is set. All code that runs on CPU0 must be careful not
> to corrupt that code or the flag it uses. However, U-Boot (and I expect
> the upstream kernel) don't yet know about this, and hence
> sometimes/often corrupt that SMP pen, resulting in strange behaviour.
> I'm pretty sure I've seen all 4 CPUs start booting the Linux kernel in
> parallel before.
>
> In summary, I know there are issues in this area when using U-Boot. I
> don't know of any fix at present. Either U-Boot must hard-code some
> reserved memory regions, or perhaps the binary FW has some way of
> informing SW where the SMP pen region is, and U-Boot needs to learn how
> to find/interpret that information (and pass it to the kernel via
> /memreserve/ or similar in DT).
>
> Eric, did you find out any more details on the SMP pen mechanism since I
> last asked you about it?

Oh, I thought we were settled on this back in May -- the CPUs are
spinning in the low 8kb.  They are in that state by the time the
firmware has handed off to us.  I think it's modifying a /memreserve/
node for you to know where the pen is.

Hey, do you know what's going on with merging code?  I feel really stuck
here -- I've got a graphics driver, and a Pi2 implementation, and Andrea
has cpufreq, and we're still stuck on getting the firmware driver
merged.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20150710/79b28754/attachment.sig>

[U-Boot] scripts: RPi 2: only 1 out of 4 CPUs brought up

[Stephen Warren]

On 07/11/2015 12:16 AM, Eric Anholt wrote:
> Stephen Warren <swarren@wwwdotorg.org> writes:
...
>> Eric, did you find out any more details on the SMP pen mechanism 
>> since I last asked you about it?
> 
> Oh, I thought we were settled on this back in May -- the CPUs are 
> spinning in the low 8kb.  They are in that state by the time the 
> firmware has handed off to us.  I think it's modifying a 
> /memreserve/ node for you to know where the pen is.

Ah OK. I'll have to look into U-Boot and see if something is
accidentally using page 0 for something, or perhaps the kernel is just
using that as the first free page...

> Hey, do you know what's going on with merging code?  I feel really 
> stuck here -- I've got a graphics driver, and a Pi2
> implementation, and Andrea has cpufreq, and we're still stuck on
> getting the firmware driver merged.

Sorry, no. Lee would merge any SoC core patches (i.e.
arch/arm/mach-bcm2835), and relevant maintainers own the graphics and
cpufreq drivers.