[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[fenghua at phytium.com.cn]

From: David Feng <fenghua@phytium.com.cn>

CPU_RELEASE_ADDR should be defined when CONFIG_ARMV8_MULTIENTRY is used.

Signed-off-by: David Feng <fenghua@phytium.com.cn>
---
 board/armltd/vexpress64/vexpress64.c |    8 ++++++++
 include/configs/vexpress_aemv8a.h    |    9 +++++++++
 2 files changed, 17 insertions(+)

diff --git a/board/armltd/vexpress64/vexpress64.c b/board/armltd/vexpress64/vexpress64.c
index de62864..bb6b5f6 100644
--- a/board/armltd/vexpress64/vexpress64.c
+++ b/board/armltd/vexpress64/vexpress64.c
@@ -22,6 +22,14 @@ int board_init(void)
 
 int dram_init(void)
 {
+#ifdef CONFIG_ARMV8_MULTIENTRY
+	/*
+	 * Clear spin table so that secondary processors
+	 * observe the correct value after waken up from wfe.
+	 */
+	*(unsigned long *)CPU_RELEASE_ADDR = 0;
+#endif
+
 	gd->ram_size = PHYS_SDRAM_1_SIZE;
 	return 0;
 }
diff --git a/include/configs/vexpress_aemv8a.h b/include/configs/vexpress_aemv8a.h
index 810eef1..6fa651a 100644
--- a/include/configs/vexpress_aemv8a.h
+++ b/include/configs/vexpress_aemv8a.h
@@ -54,6 +54,15 @@
 /* Flat Device Tree Definitions */
 #define CONFIG_OF_LIBFDT
 
+#ifdef CONFIG_ARMV8_MULTIENTRY
+/* SMP Spin Table Definitions */
+#ifdef CONFIG_BASE_FVP
+#define CPU_RELEASE_ADDR		(CONFIG_SYS_SDRAM_BASE + 0x03f00000)
+#else
+#define CPU_RELEASE_ADDR		(CONFIG_SYS_SDRAM_BASE + 0x7fff0)
+#endif
+#endif
+
 /* CS register bases for the original memory map. */
 #define V2M_PA_CS0			0x00000000
 #define V2M_PA_CS1			0x14000000
-- 
1.7.9.5

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[Linus Walleij]

On Tue, Mar 10, 2015 at 3:08 AM,  <fenghua@phytium.com.cn> wrote:

> From: David Feng <fenghua@phytium.com.cn>
>
> CPU_RELEASE_ADDR should be defined when CONFIG_ARMV8_MULTIENTRY is used.
>
> Signed-off-by: David Feng <fenghua@phytium.com.cn>

As asked earlier: how can I test this with FVP or the base model?

I'm very interested in doing this as I guess it involves starting U-Boot
at EL3 on bare metal and I really want to try this.

> +/* SMP Spin Table Definitions */
> +#ifdef CONFIG_BASE_FVP
> +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x03f00000)
> +#else
> +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x7fff0)
> +#endif

Where are these address defines coming from?

Do these spin tables exist in a standard ARM FVP or base model?

I get the impression that a secondary operating system is being booted
on the secondary CPU at one of these addresses, but why is it running
at these addresses specifically, and is that something coming with these
simulators, or is it some image that is loaded on the side, that the
community does not have access to?

Yours,
Linus Walleij

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[FengHua]

> -----Original Messages-----
> From: "Linus Walleij" <linus.walleij@linaro.org>
> Sent Time: 2015-03-10 18:08:03 (Tuesday)
> To: "David Feng" <fenghua@phytium.com.cn>
> Cc: "U-Boot Mailing List" <u-boot@lists.denx.de>, "Tom Rini" <trini@ti.com>, "Albert ARIBAUD" <albert.u.boot@aribaud.net>
> Subject: Re: [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined
> 
> On Tue, Mar 10, 2015 at 3:08 AM,  <fenghua@phytium.com.cn> wrote:
> 
> > From: David Feng <fenghua@phytium.com.cn>
> >
> > CPU_RELEASE_ADDR should be defined when CONFIG_ARMV8_MULTIENTRY is used.
> >
> > Signed-off-by: David Feng <fenghua@phytium.com.cn>
> 
> As asked earlier: how can I test this with FVP or the base model?
> 
I usually use Foundation Model.

> I'm very interested in doing this as I guess it involves starting U-Boot
> at EL3 on bare metal and I really want to try this.
> 
> > +/* SMP Spin Table Definitions */
> > +#ifdef CONFIG_BASE_FVP
> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x03f00000)
> > +#else
> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x7fff0)
> > +#endif
> 
> Where are these address defines coming from?
It's just hard coded and should be the same value with that in DTS.

> 
> Do these spin tables exist in a standard ARM FVP or base model?
> 
> I get the impression that a secondary operating system is being booted
> on the secondary CPU at one of these addresses, but why is it running
> at these addresses specifically, and is that something coming with these
> simulators, or is it some image that is loaded on the side, that the
> community does not have access to?
> 
PSCI is not implemented in uboot-armv8. If booting u-boot on bare-metal
only spin table can be used. All we do is describing booting method(spin table) and cpu release
address in DTS. Linux kernel get cpu release address from DTS also.

Yours,
David.

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[bhupesh.sharma at freescale.com]

> -----Original Message-----
> From: U-Boot [mailto:u-boot-bounces at lists.denx.de] On Behalf Of FengHua
> Sent: Wednesday, March 11, 2015 6:38 PM
> To: Linus Walleij
> Cc: U-Boot Mailing List
> Subject: Re: [U-Boot] [PATCH] Vexpress64: Fix the compiling error when
> CONFIG_ARMV8_MULTIENTRY defined
> 
> 
> 
> 
> > -----Original Messages-----
> > From: "Linus Walleij" <linus.walleij@linaro.org> Sent Time: 2015-03-10
> > 18:08:03 (Tuesday)
> > To: "David Feng" <fenghua@phytium.com.cn>
> > Cc: "U-Boot Mailing List" <u-boot@lists.denx.de>, "Tom Rini"
> > <trini@ti.com>, "Albert ARIBAUD" <albert.u.boot@aribaud.net>
> > Subject: Re: [PATCH] Vexpress64: Fix the compiling error when
> > CONFIG_ARMV8_MULTIENTRY defined
> >
> > On Tue, Mar 10, 2015 at 3:08 AM,  <fenghua@phytium.com.cn> wrote:
> >
> > > From: David Feng <fenghua@phytium.com.cn>
> > >
> > > CPU_RELEASE_ADDR should be defined when CONFIG_ARMV8_MULTIENTRY is
> used.
> > >
> > > Signed-off-by: David Feng <fenghua@phytium.com.cn>
> >
> > As asked earlier: how can I test this with FVP or the base model?
> >
> I usually use Foundation Model.
> 
> > I'm very interested in doing this as I guess it involves starting
> > U-Boot at EL3 on bare metal and I really want to try this.
> >
> > > +/* SMP Spin Table Definitions */
> > > +#ifdef CONFIG_BASE_FVP
> > > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE +
> 0x03f00000)
> > > +#else
> > > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE +
> 0x7fff0)
> > > +#endif
> >
> > Where are these address defines coming from?
> It's just hard coded and should be the same value with that in DTS.
> 
> >
> > Do these spin tables exist in a standard ARM FVP or base model?
> >
> > I get the impression that a secondary operating system is being booted
> > on the secondary CPU at one of these addresses, but why is it running
> > at these addresses specifically, and is that something coming with
> > these simulators, or is it some image that is loaded on the side, that
> > the community does not have access to?
> >
> PSCI is not implemented in uboot-armv8. If booting u-boot on bare-metal
> only spin table can be used. All we do is describing booting method(spin
> table) and cpu release address in DTS. Linux kernel get cpu release
> address from DTS also.
> 

Arnab's patches enable PSCI for amrv8 on u-boot:
http://comments.gmane.org/gmane.comp.boot-loaders.u-boot/207882

We have tested the same on ARMv8 foundation model as well.

Regards,
Bhupesh

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[Linus Walleij]

On Wed, Mar 11, 2015 at 2:08 PM, FengHua <fenghua@phytium.com.cn> wrote:

(...)
>> As asked earlier: how can I test this with FVP or the base model?
>>
> I usually use Foundation Model.

OK... That is the same as the FVP I'm using I guess:

$ Foundation_v8pkg/models/Linux64_GCC-4.1/Foundation_v8 --version
ARM V8 Foundation Model r0p0 (model build 9.0.24)
Copyright 2013 ARM Limited.
All Rights Reserved.

Correct?

>> I'm very interested in doing this as I guess it involves starting U-Boot
>> at EL3 on bare metal and I really want to try this.
>>
>> > +/* SMP Spin Table Definitions */
>> > +#ifdef CONFIG_BASE_FVP
>> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x03f00000)
>> > +#else
>> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x7fff0)
>> > +#endif
>>
>> Where are these address defines coming from?
>
> It's just hard coded and should be the same value with that in DTS.

I look in the DTS from the Linux kernel:

arch/arm64/boot/dts/arm/foundation-v8.dts:

                cpu at 0 {
                        device_type = "cpu";
                        compatible = "arm,armv8";
                        reg = <0x0 0x0>;
                        enable-method = "spin-table";
                        cpu-release-addr = <0x0 0x8000fff8>;
                        next-level-cache = <&L2_0>;
                };
                cpu at 1 {
                        device_type = "cpu";
                        compatible = "arm,armv8";
                        reg = <0x0 0x1>;
                        enable-method = "spin-table";
                        cpu-release-addr = <0x0 0x8000fff8>;
                        next-level-cache = <&L2_0>;
                };
(...)

It's not the same addres for what I can tell,

CONFIG_SYS_SDRAM_BASE + 0x03f00000 = 0x83f00000

but the DTS cpu-release-addr is 0x8000fff8...

Curiously we also have an ontology problem here: the DTS in
the Linux kernel does use spin tables, but there is another set of
DTS files in the ARM Trusted Firmware distribution, for the same
simulator, stating PSCI as CPU release mechanism. These are
the only ones that work properly when using ARM TF.

(Well obviously you don't use these...)

>> Do these spin tables exist in a standard ARM FVP or base model?
>>
>> I get the impression that a secondary operating system is being booted
>> on the secondary CPU at one of these addresses, but why is it running
>> at these addresses specifically, and is that something coming with these
>> simulators, or is it some image that is loaded on the side, that the
>> community does not have access to?
>>
> PSCI is not implemented in uboot-armv8.

Nope. But it is implemented in ARM Trusted Firmware for ARMv8.
ARM TF install the PSCI handlers before U-Boot is executed.

> If booting u-boot on bare-metal
> only spin table can be used. All we do is describing booting
> method(spin table) and cpu release
> address in DTS. Linux kernel get cpu release address from DTS also.

Yep, I want to try this method...

I just cannot even get U-Boot to run on the foundation model.

I alter CONFIG_SYS_TEXT_BASE to 0x0:

#define CONFIG_SYS_TEXT_BASE        0x00000000

Then I run the simulator like so:

Foundation_v8pkg/models/Linux64_GCC-4.1/Foundation_v8 \
--cores=4                                 \
--no-secure-memory                        \
--visualization                           \
--gicv3                                   \
--data="u-boot-fvp-semi.bin"@0x00000000

Do you do this as well? Or how do you kick your simulator to
run U-Boot on bare metal?

I must be missing something.

Yours,
Linus Walleij

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[FengHua]

hi Linus,

> -----Original Messages-----
> From: "Linus Walleij" <linus.walleij@linaro.org>
> Sent Time: 2015-03-20 17:39:48 (Friday)
> To: FengHua <fenghua@phytium.com.cn>
> Cc: "U-Boot Mailing List" <u-boot@lists.denx.de>, "albert.u.boot" <albert.u.boot@aribaud.net>
> Subject: Re: Re: [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined
> 
> On Wed, Mar 11, 2015 at 2:08 PM, FengHua <fenghua@phytium.com.cn> wrote:
> 
> (...)
> >> As asked earlier: how can I test this with FVP or the base model?
> >>
> > I usually use Foundation Model.
> 
> OK... That is the same as the FVP I'm using I guess:
> 
> $ Foundation_v8pkg/models/Linux64_GCC-4.1/Foundation_v8 --version
> ARM V8 Foundation Model r0p0 (model build 9.0.24)
> Copyright 2013 ARM Limited.
> All Rights Reserved.
> 
> Correct?
Yes

> 
> >> I'm very interested in doing this as I guess it involves starting U-Boot
> >> at EL3 on bare metal and I really want to try this.
> >>
> >> > +/* SMP Spin Table Definitions */
> >> > +#ifdef CONFIG_BASE_FVP
> >> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x03f00000)
> >> > +#else
> >> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x7fff0)
> >> > +#endif
> >>
> >> Where are these address defines coming from?
> >
> > It's just hard coded and should be the same value with that in DTS.
> 
> I look in the DTS from the Linux kernel:
> 
> arch/arm64/boot/dts/arm/foundation-v8.dts:
> 
>                 cpu at 0 {
>                         device_type = "cpu";
>                         compatible = "arm,armv8";
>                         reg = <0x0 0x0>;
>                         enable-method = "spin-table";
>                         cpu-release-addr = <0x0 0x8000fff8>;
>                         next-level-cache = <&L2_0>;
>                 };
>                 cpu at 1 {
>                         device_type = "cpu";
>                         compatible = "arm,armv8";
>                         reg = <0x0 0x1>;
>                         enable-method = "spin-table";
>                         cpu-release-addr = <0x0 0x8000fff8>;
>                         next-level-cache = <&L2_0>;
>                 };
> (...)
> 
> It's not the same addres for what I can tell,
> 
> CONFIG_SYS_SDRAM_BASE + 0x03f00000 = 0x83f00000
> 
> but the DTS cpu-release-addr is 0x8000fff8...
> 
> Curiously we also have an ontology problem here: the DTS in
> the Linux kernel does use spin tables, but there is another set of
> DTS files in the ARM Trusted Firmware distribution, for the same
> simulator, stating PSCI as CPU release mechanism. These are
> the only ones that work properly when using ARM TF.
> 
> (Well obviously you don't use these...)

PSCI is prefered by Linux.

> 
> >> Do these spin tables exist in a standard ARM FVP or base model?
> >>
> >> I get the impression that a secondary operating system is being booted
> >> on the secondary CPU at one of these addresses, but why is it running
> >> at these addresses specifically, and is that something coming with these
> >> simulators, or is it some image that is loaded on the side, that the
> >> community does not have access to?
> >>
> > PSCI is not implemented in uboot-armv8.
> 
> Nope. But it is implemented in ARM Trusted Firmware for ARMv8.
> ARM TF install the PSCI handlers before U-Boot is executed.
> 
> > If booting u-boot on bare-metal
> > only spin table can be used. All we do is describing booting
> > method(spin table) and cpu release
> > address in DTS. Linux kernel get cpu release address from DTS also.
> 
> Yep, I want to try this method...
> 
> I just cannot even get U-Boot to run on the foundation model.
> 
> I alter CONFIG_SYS_TEXT_BASE to 0x0:
> 
> #define CONFIG_SYS_TEXT_BASE        0x00000000
> 
> Then I run the simulator like so:
> 
> Foundation_v8pkg/models/Linux64_GCC-4.1/Foundation_v8 \
> --cores=4                                 \
> --no-secure-memory                        \
> --visualization                           \
> --gicv3                                   \
> --data="u-boot-fvp-semi.bin"@0x00000000
> 
> Do you do this as well? Or how do you kick your simulator to
> run U-Boot on bare metal?
> 
CONFIG_SYS_TEXT_BASE should be defined as 0x80000000.
The reset PC value is 0x80000000 on Foundation Model.
and I use "--image=u-boot.bin" instead of "--data=...".

Yours,
David.

[U-Boot] [PATCH] Vexpress64: Fix the compiling error when CONFIG_ARMV8_MULTIENTRY defined

[Mark Rutland]

> >> > +/* SMP Spin Table Definitions */
> >> > +#ifdef CONFIG_BASE_FVP
> >> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x03f00000)
> >> > +#else
> >> > +#define CPU_RELEASE_ADDR               (CONFIG_SYS_SDRAM_BASE + 0x7fff0)
> >> > +#endif
> >>
> >> Where are these address defines coming from?
> >
> > It's just hard coded and should be the same value with that in DTS.
> 
> I look in the DTS from the Linux kernel:
> 
> arch/arm64/boot/dts/arm/foundation-v8.dts:
> 
>                 cpu at 0 {
>                         device_type = "cpu";
>                         compatible = "arm,armv8";
>                         reg = <0x0 0x0>;
>                         enable-method = "spin-table";
>                         cpu-release-addr = <0x0 0x8000fff8>;
>                         next-level-cache = <&L2_0>;
>                 };
>                 cpu at 1 {
>                         device_type = "cpu";
>                         compatible = "arm,armv8";
>                         reg = <0x0 0x1>;
>                         enable-method = "spin-table";
>                         cpu-release-addr = <0x0 0x8000fff8>;
>                         next-level-cache = <&L2_0>;
>                 };
> (...)
> 
> It's not the same addres for what I can tell,
> 
> CONFIG_SYS_SDRAM_BASE + 0x03f00000 = 0x83f00000
> 
> but the DTS cpu-release-addr is 0x8000fff8...
> 
> Curiously we also have an ontology problem here: the DTS in
> the Linux kernel does use spin tables, but there is another set of
> DTS files in the ARM Trusted Firmware distribution, for the same
> simulator, stating PSCI as CPU release mechanism. These are
> the only ones that work properly when using ARM TF.

FWIW in the bootwrapper we inject the relevant PSCI properties into the
DTB if the bootwrapper is configured to use PSCI, and we should really
do the same for spin-table.

Given the enable-method is entirely dependent on the FW, it would be
better for FW to fill in an appropriate value (where possible), leaving
those out of the dts.

Mark.