Standard/common method for booting secondary cores

Standard/common method for booting secondary cores

[Mason]

Hello,

With the current push for CONFIG_ARCH_MULTIPLATFORM,
I'm wondering if things like starting secondary cores
have been "standardized"/factorized to the point where
I can just use some default implementation?

I think the implementation I'm using as a starting
point uses SMC calls. I think we are using "TrustZone"
where Linux runs in NS (non secure) mode, and a tiny
proprietary OS runs in secure mode.

IIUC (which is probably NOT the case), when Linux runs
in NS mode, some operations that are typically carried
out at boot/init are not allowed, such as

- starting secondary cores
- configuring the L2 cache controller

and this must be done by the secure OS via SMC?

I'd be happy to be given pointers to internet references
and do my own reading. I've bookmarked a few on related
subjects:

http://events.linuxfoundation.org/sites/events/files/slides/clement-smp-bring-up-on-arm-soc.pdf
http://www.linux-arm.org/pub/LinuxPlatform/RealViewLink/Booting_ARM_Linux_SMP_on_MPCore.doc
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0388g/Beihjjgb.html
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.prd29-genc-009492c/index.html

Are the arch/arm/common/mcpm_* files relevant?

Regards.

Standard/common method for booting secondary cores

[Dave Martin]

On Tue, Apr 21, 2015 at 12:43:03AM +0200, Mason wrote:
> Hello,
> 
> With the current push for CONFIG_ARCH_MULTIPLATFORM,
> I'm wondering if things like starting secondary cores
> have been "standardized"/factorized to the point where
> I can just use some default implementation?
> 
> I think the implementation I'm using as a starting
> point uses SMC calls. I think we are using "TrustZone"
> where Linux runs in NS (non secure) mode, and a tiny
> proprietary OS runs in secure mode.

_If_ you're implementing something new and have a say in what gets
implemented in the firmware, please consider PSCI.

Without this, little standardisation exists.  Low level CPU
power management must typically involve SMCs, but there's
no standardisation of their exact semantics, which implies that
some platform-specific glue will likely be needed.  The best you can
do is look for one or more backends in the kernel that are
close to what you need (if any), and if so propose some refactoring
to extend support to your platform.

> IIUC (which is probably NOT the case), when Linux runs
> in NS mode, some operations that are typically carried
> out at boot/init are not allowed, such as
> 
> - starting secondary cores
> - configuring the L2 cache controller
> 
> and this must be done by the secure OS via SMC?

Generally, yes.

> I'd be happy to be given pointers to internet references
> and do my own reading. I've bookmarked a few on related
> subjects:
> 
> http://events.linuxfoundation.org/sites/events/files/slides/clement-smp-bring-up-on-arm-soc.pdf
> http://www.linux-arm.org/pub/LinuxPlatform/RealViewLink/Booting_ARM_Linux_SMP_on_MPCore.doc
> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0388g/Beihjjgb.html
> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.prd29-genc-009492c/index.html

For PSCI, you can look at http://infocenter.arm.com/help/topic/com.arm.doc.den0022c/index.html.

> 
> Are the arch/arm/common/mcpm_* files relevant?

MCPM is infrastructure for handling _cluster_ powerup/powerdown, in
kernels where Linux runs in Secure (or otherwise has a high level of
access to power controls).

If unsure, say N.

[...]

Cheers
---Dave

Standard/common method for booting secondary cores

[Mark Rutland]

Hi,

> With the current push for CONFIG_ARCH_MULTIPLATFORM,
> I'm wondering if things like starting secondary cores
> have been "standardized"/factorized to the point where
> I can just use some default implementation?

PSCI [1] exists for this, and is supported by Linux.

Mark.

[1] http://infocenter.arm.com/help/topic/com.arm.doc.den0022c/DEN0022C_Power_State_Coordination_Interface.pdf

Standard/common method for booting secondary cores

[Mason]

Hello Dave,

It seems you and Mark posted within seconds of each other :-)

On 21/04/2015 13:41, Dave Martin wrote:
> On Tue, Apr 21, 2015 at 12:43:03AM +0200, Mason wrote:
>> Hello,
>>
>> With the current push for CONFIG_ARCH_MULTIPLATFORM,
>> I'm wondering if things like starting secondary cores
>> have been "standardized"/factorized to the point where
>> I can just use some default implementation?
>>
>> I think the implementation I'm using as a starting
>> point uses SMC calls. I think we are using "TrustZone"
>> where Linux runs in NS (non secure) mode, and a tiny
>> proprietary OS runs in secure mode.
>
> _If_ you're implementing something new

You're talking about software, right?
(The hardware already exists.)
As I mentioned we do have a working implementation using
SMC to request service from the secure OS, but I have some
time to evaluate "better" strategies, as I port the
platform to a newer kernel.

> and have a say in what gets implemented in the firmware,
> please consider PSCI.

In this context, firmware would be the code run by the
secure OS?

> Without this, little standardisation exists.  Low level CPU
> power management must typically involve SMCs, but there's
> no standardisation of their exact semantics, which implies that
> some platform-specific glue will likely be needed.  The best you can
> do is look for one or more backends in the kernel that are
> close to what you need (if any), and if so propose some refactoring
> to extend support to your platform.

My objective is to minimize platform-specific code, I will
look into PSCI.

>> IIUC (which is probably NOT the case), when Linux runs
>> in NS mode, some operations that are typically carried
>> out at boot/init are not allowed, such as
>>
>> - starting secondary cores
>> - configuring the L2 cache controller
>>
>> and this must be done by the secure OS via SMC?
>
> Generally, yes.

Is PSCI involved in L2$ configuration?

>> I'd be happy to be given pointers to internet references
>> and do my own reading. I've bookmarked a few on related
>> subjects:
>>
>> http://events.linuxfoundation.org/sites/events/files/slides/clement-smp-bring-up-on-arm-soc.pdf
>> http://www.linux-arm.org/pub/LinuxPlatform/RealViewLink/Booting_ARM_Linux_SMP_on_MPCore.doc
>> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0388g/Beihjjgb.html
>> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.prd29-genc-009492c/index.html
>
> For PSCI, you can look at http://infocenter.arm.com/help/topic/com.arm.doc.den0022c/index.html.
>
>>
>> Are the arch/arm/common/mcpm_* files relevant?
>
> MCPM is infrastructure for handling _cluster_ powerup/powerdown, in
> kernels where Linux runs in Secure (or otherwise has a high level of
> access to power controls).
>
> If unsure, say N.

Thanks.

Standard/common method for booting secondary cores

[Dave Martin]

On Tue, Apr 21, 2015 at 02:20:52PM +0200, Mason wrote:
> Hello Dave,
> 
> It seems you and Mark posted within seconds of each other :-)

Seems so...  fortunately we didn't argue the opposite to each
other.

> On 21/04/2015 13:41, Dave Martin wrote:
> >On Tue, Apr 21, 2015 at 12:43:03AM +0200, Mason wrote:
> >>Hello,
> >>
> >>With the current push for CONFIG_ARCH_MULTIPLATFORM,
> >>I'm wondering if things like starting secondary cores
> >>have been "standardized"/factorized to the point where
> >>I can just use some default implementation?
> >>
> >>I think the implementation I'm using as a starting
> >>point uses SMC calls. I think we are using "TrustZone"
> >>where Linux runs in NS (non secure) mode, and a tiny
> >>proprietary OS runs in secure mode.
> >
> >_If_ you're implementing something new
> 
> You're talking about software, right?
> (The hardware already exists.)

Yes, PSCI is a software interface.

> As I mentioned we do have a working implementation using
> SMC to request service from the secure OS, but I have some
> time to evaluate "better" strategies, as I port the
> platform to a newer kernel.
> 
> >and have a say in what gets implemented in the firmware,
> >please consider PSCI.
> 
> In this context, firmware would be the code run by the
> secure OS?

Yes, or at least those parts of the Secure World software that provide
essential services to the main OS.

> >Without this, little standardisation exists.  Low level CPU
> >power management must typically involve SMCs, but there's
> >no standardisation of their exact semantics, which implies that
> >some platform-specific glue will likely be needed.  The best you can
> >do is look for one or more backends in the kernel that are
> >close to what you need (if any), and if so propose some refactoring
> >to extend support to your platform.
> 
> My objective is to minimize platform-specific code, I will
> look into PSCI.
> 
> >>IIUC (which is probably NOT the case), when Linux runs
> >>in NS mode, some operations that are typically carried
> >>out at boot/init are not allowed, such as
> >>
> >>- starting secondary cores
> >>- configuring the L2 cache controller
> >>
> >>and this must be done by the secure OS via SMC?
> >
> >Generally, yes.
> 
> Is PSCI involved in L2$ configuration?

That depends on the platform.  Single-cluster systems might
not have to do very much L2 configuration, but it depends on
the hardware -- particularly of the L2 is external.

I suggest you take a look at the PSCI spec first -- this
should make is clearer what the firmware is expected to do.

Cheers
---Dave