[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

If PSCI initializes correctly and PSCI SMP operations are available, use them.
This is required for SMP support in Dom0 on Xen.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: will.deacon at arm.com
CC: arnd at arndb.de
CC: marc.zyngier at arm.com
CC: linux at arm.linux.org.uk
CC: nico at linaro.org
---
 arch/arm/kernel/setup.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 341efaa..c7e50dd 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -770,10 +770,10 @@ void __init setup_arch(char **cmdline_p)
 	psci_init();
 #ifdef CONFIG_SMP
 	if (is_smp()) {
-		if (mdesc->smp)
-			smp_set_ops(mdesc->smp);
-		else if (psci_smp_available())
+		if (psci_smp_available())
 			smp_set_ops(&psci_smp_ops);
+		else
+			smp_set_ops(mdesc->smp);
 		smp_init_cpus();
 	}
 #endif
-- 
1.7.2.5

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

If PSCI initializes correctly and PSCI SMP operations are available, use them.
This is required for SMP support in Dom0 on Xen.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: will.deacon@arm.com
CC: arnd@arndb.de
CC: marc.zyngier@arm.com
CC: linux@arm.linux.org.uk
CC: nico@linaro.org
---
 arch/arm/kernel/setup.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 341efaa..c7e50dd 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -770,10 +770,10 @@ void __init setup_arch(char **cmdline_p)
 	psci_init();
 #ifdef CONFIG_SMP
 	if (is_smp()) {
-		if (mdesc->smp)
-			smp_set_ops(mdesc->smp);
-		else if (psci_smp_available())
+		if (psci_smp_available())
 			smp_set_ops(&psci_smp_ops);
+		else
+			smp_set_ops(mdesc->smp);
 		smp_init_cpus();
 	}
 #endif
-- 
1.7.2.5

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

If PSCI initializes correctly and PSCI SMP operations are available, use them.
This is required for SMP support in Dom0 on Xen.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: will.deacon@arm.com
CC: arnd@arndb.de
CC: marc.zyngier@arm.com
CC: linux@arm.linux.org.uk
CC: nico@linaro.org
---
 arch/arm/kernel/setup.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 341efaa..c7e50dd 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -770,10 +770,10 @@ void __init setup_arch(char **cmdline_p)
 	psci_init();
 #ifdef CONFIG_SMP
 	if (is_smp()) {
-		if (mdesc->smp)
-			smp_set_ops(mdesc->smp);
-		else if (psci_smp_available())
+		if (psci_smp_available())
 			smp_set_ops(&psci_smp_ops);
+		else
+			smp_set_ops(mdesc->smp);
 		smp_init_cpus();
 	}
 #endif
-- 
1.7.2.5

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Fri, 29 Mar 2013, Stefano Stabellini wrote:

> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> This is required for SMP support in Dom0 on Xen.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> CC: will.deacon at arm.com
> CC: arnd at arndb.de
> CC: marc.zyngier at arm.com
> CC: linux at arm.linux.org.uk
> CC: nico at linaro.org

I'd suggest you also include in your series the patch I posted earlier 
providing a runtime mdesc->smp_init method as well.  This way the 
priority order would be:

- If mdesc->smp_init is non null then use that.

- Otherwise, if PSCI is available then use that.

- Otherwise use mdesc->smp.

This way, if the PSCI default has to be overriden (like in the MCPM case 
because it needs to wrap PSCI itself, or to cover Rob's concern) then 
this can be achieved at run time on a per mdesc basis.


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Fri, 29 Mar 2013, Stefano Stabellini wrote:

> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> This is required for SMP support in Dom0 on Xen.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> CC: will.deacon@arm.com
> CC: arnd@arndb.de
> CC: marc.zyngier@arm.com
> CC: linux@arm.linux.org.uk
> CC: nico@linaro.org

I'd suggest you also include in your series the patch I posted earlier 
providing a runtime mdesc->smp_init method as well.  This way the 
priority order would be:

- If mdesc->smp_init is non null then use that.

- Otherwise, if PSCI is available then use that.

- Otherwise use mdesc->smp.

This way, if the PSCI default has to be overriden (like in the MCPM case 
because it needs to wrap PSCI itself, or to cover Rob's concern) then 
this can be achieved at run time on a per mdesc basis.


Nicolas

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
> > If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > This is required for SMP support in Dom0 on Xen.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > CC: will.deacon at arm.com
> > CC: arnd at arndb.de
> > CC: marc.zyngier at arm.com
> > CC: linux at arm.linux.org.uk
> > CC: nico at linaro.org
> 
> I'd suggest you also include in your series the patch I posted earlier 
> providing a runtime mdesc->smp_init method as well.

OK.


> This way the 
> priority order would be:
> 
> - If mdesc->smp_init is non null then use that.
> 
> - Otherwise, if PSCI is available then use that.
> 
> - Otherwise use mdesc->smp.
> 
> This way, if the PSCI default has to be overriden (like in the MCPM case 
> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> this can be achieved at run time on a per mdesc basis.

Actually that's not a bad idea, it could make everybody happy.
What about the following, in this precise order:

- if a xen hypervisor node is present on device tree, use PSCI;
- otherwise if mdesc->smp_init is non null then use it;
- otherwise if PSCI is available then use it;
- otherwise use mdesc->smp.

It's the most practical solution to satisfy everybody's needs.

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
> > If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > This is required for SMP support in Dom0 on Xen.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > CC: will.deacon@arm.com
> > CC: arnd@arndb.de
> > CC: marc.zyngier@arm.com
> > CC: linux@arm.linux.org.uk
> > CC: nico@linaro.org
> 
> I'd suggest you also include in your series the patch I posted earlier 
> providing a runtime mdesc->smp_init method as well.

OK.


> This way the 
> priority order would be:
> 
> - If mdesc->smp_init is non null then use that.
> 
> - Otherwise, if PSCI is available then use that.
> 
> - Otherwise use mdesc->smp.
> 
> This way, if the PSCI default has to be overriden (like in the MCPM case 
> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> this can be achieved at run time on a per mdesc basis.

Actually that's not a bad idea, it could make everybody happy.
What about the following, in this precise order:

- if a xen hypervisor node is present on device tree, use PSCI;
- otherwise if mdesc->smp_init is non null then use it;
- otherwise if PSCI is available then use it;
- otherwise use mdesc->smp.

It's the most practical solution to satisfy everybody's needs.

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Fri, 29 Mar 2013, Stefano Stabellini wrote:

> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > 
> > > If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > > This is required for SMP support in Dom0 on Xen.
> > > 
> > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > CC: will.deacon at arm.com
> > > CC: arnd at arndb.de
> > > CC: marc.zyngier at arm.com
> > > CC: linux at arm.linux.org.uk
> > > CC: nico at linaro.org
> > 
> > I'd suggest you also include in your series the patch I posted earlier 
> > providing a runtime mdesc->smp_init method as well.
> 
> OK.
> 
> 
> > This way the 
> > priority order would be:
> > 
> > - If mdesc->smp_init is non null then use that.
> > 
> > - Otherwise, if PSCI is available then use that.
> > 
> > - Otherwise use mdesc->smp.
> > 
> > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > this can be achieved at run time on a per mdesc basis.
> 
> Actually that's not a bad idea, it could make everybody happy.
> What about the following, in this precise order:
> 
> - if a xen hypervisor node is present on device tree, use PSCI;
> - otherwise if mdesc->smp_init is non null then use it;
> - otherwise if PSCI is available then use it;
> - otherwise use mdesc->smp.
> 
> It's the most practical solution to satisfy everybody's needs.

Maybe I'm missing something obvious, but why can't xen declare a mdesc 
of its own?  Given it is going to tweak the DT passed to the kernel 
anyway that shouldn't be a problem.

That would be more eleguant than adding xen exception hooks in generic 
code.


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Fri, 29 Mar 2013, Stefano Stabellini wrote:

> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > 
> > > If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > > This is required for SMP support in Dom0 on Xen.
> > > 
> > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > CC: will.deacon@arm.com
> > > CC: arnd@arndb.de
> > > CC: marc.zyngier@arm.com
> > > CC: linux@arm.linux.org.uk
> > > CC: nico@linaro.org
> > 
> > I'd suggest you also include in your series the patch I posted earlier 
> > providing a runtime mdesc->smp_init method as well.
> 
> OK.
> 
> 
> > This way the 
> > priority order would be:
> > 
> > - If mdesc->smp_init is non null then use that.
> > 
> > - Otherwise, if PSCI is available then use that.
> > 
> > - Otherwise use mdesc->smp.
> > 
> > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > this can be achieved at run time on a per mdesc basis.
> 
> Actually that's not a bad idea, it could make everybody happy.
> What about the following, in this precise order:
> 
> - if a xen hypervisor node is present on device tree, use PSCI;
> - otherwise if mdesc->smp_init is non null then use it;
> - otherwise if PSCI is available then use it;
> - otherwise use mdesc->smp.
> 
> It's the most practical solution to satisfy everybody's needs.

Maybe I'm missing something obvious, but why can't xen declare a mdesc 
of its own?  Given it is going to tweak the DT passed to the kernel 
anyway that shouldn't be a problem.

That would be more eleguant than adding xen exception hooks in generic 
code.


Nicolas

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
> > On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > 
> > > > If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > > > This is required for SMP support in Dom0 on Xen.
> > > > 
> > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > > CC: will.deacon at arm.com
> > > > CC: arnd at arndb.de
> > > > CC: marc.zyngier at arm.com
> > > > CC: linux at arm.linux.org.uk
> > > > CC: nico at linaro.org
> > > 
> > > I'd suggest you also include in your series the patch I posted earlier 
> > > providing a runtime mdesc->smp_init method as well.
> > 
> > OK.
> > 
> > 
> > > This way the 
> > > priority order would be:
> > > 
> > > - If mdesc->smp_init is non null then use that.
> > > 
> > > - Otherwise, if PSCI is available then use that.
> > > 
> > > - Otherwise use mdesc->smp.
> > > 
> > > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > > this can be achieved at run time on a per mdesc basis.
> > 
> > Actually that's not a bad idea, it could make everybody happy.
> > What about the following, in this precise order:
> > 
> > - if a xen hypervisor node is present on device tree, use PSCI;
> > - otherwise if mdesc->smp_init is non null then use it;
> > - otherwise if PSCI is available then use it;
> > - otherwise use mdesc->smp.
> > 
> > It's the most practical solution to satisfy everybody's needs.
> 
> Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> of its own?  Given it is going to tweak the DT passed to the kernel 
> anyway that shouldn't be a problem.
> 
> That would be more eleguant than adding xen exception hooks in generic 
> code.

I guess I could introduce a Xen-enabled mdesc for all the platforms we intend to
support, but as the list of boards with virtualization extensions grow,
also the list of Xen-enabled mdescs would grow accordingly.
We would end up with dozen (more?) of Xen mdescs.

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
> > On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > 
> > > > If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > > > This is required for SMP support in Dom0 on Xen.
> > > > 
> > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > > CC: will.deacon@arm.com
> > > > CC: arnd@arndb.de
> > > > CC: marc.zyngier@arm.com
> > > > CC: linux@arm.linux.org.uk
> > > > CC: nico@linaro.org
> > > 
> > > I'd suggest you also include in your series the patch I posted earlier 
> > > providing a runtime mdesc->smp_init method as well.
> > 
> > OK.
> > 
> > 
> > > This way the 
> > > priority order would be:
> > > 
> > > - If mdesc->smp_init is non null then use that.
> > > 
> > > - Otherwise, if PSCI is available then use that.
> > > 
> > > - Otherwise use mdesc->smp.
> > > 
> > > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > > this can be achieved at run time on a per mdesc basis.
> > 
> > Actually that's not a bad idea, it could make everybody happy.
> > What about the following, in this precise order:
> > 
> > - if a xen hypervisor node is present on device tree, use PSCI;
> > - otherwise if mdesc->smp_init is non null then use it;
> > - otherwise if PSCI is available then use it;
> > - otherwise use mdesc->smp.
> > 
> > It's the most practical solution to satisfy everybody's needs.
> 
> Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> of its own?  Given it is going to tweak the DT passed to the kernel 
> anyway that shouldn't be a problem.
> 
> That would be more eleguant than adding xen exception hooks in generic 
> code.

I guess I could introduce a Xen-enabled mdesc for all the platforms we intend to
support, but as the list of boards with virtualization extensions grow,
also the list of Xen-enabled mdescs would grow accordingly.
We would end up with dozen (more?) of Xen mdescs.

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Rob Herring]

On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
>> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
>>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
>>>
>>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
>>>> This is required for SMP support in Dom0 on Xen.
>>>>
>>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>>> CC: will.deacon at arm.com
>>>> CC: arnd at arndb.de
>>>> CC: marc.zyngier at arm.com
>>>> CC: linux at arm.linux.org.uk
>>>> CC: nico at linaro.org
>>>
>>> I'd suggest you also include in your series the patch I posted earlier 
>>> providing a runtime mdesc->smp_init method as well.
>>
>> OK.
>>
>>
>>> This way the 
>>> priority order would be:
>>>
>>> - If mdesc->smp_init is non null then use that.
>>>
>>> - Otherwise, if PSCI is available then use that.
>>>
>>> - Otherwise use mdesc->smp.
>>>
>>> This way, if the PSCI default has to be overriden (like in the MCPM case 
>>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
>>> this can be achieved at run time on a per mdesc basis.
>>
>> Actually that's not a bad idea, it could make everybody happy.
>> What about the following, in this precise order:
>>
>> - if a xen hypervisor node is present on device tree, use PSCI;
>> - otherwise if mdesc->smp_init is non null then use it;
>> - otherwise if PSCI is available then use it;
>> - otherwise use mdesc->smp.
>>
>> It's the most practical solution to satisfy everybody's needs.
> 
> Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> of its own?  Given it is going to tweak the DT passed to the kernel 
> anyway that shouldn't be a problem.

Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
only for DomU guests. For Dom0, you still need all the platform specific
code except smp_ops. However, I'm doubtful this would work without other
changes on more complicated platforms like OMAP.

I would say wait to add this until you have platforms that actually need
the first case.

Rob

> 
> That would be more eleguant than adding xen exception hooks in generic 
> code.
> 
> 
> Nicolas
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Rob Herring]

On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
>> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
>>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
>>>
>>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
>>>> This is required for SMP support in Dom0 on Xen.
>>>>
>>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>>> CC: will.deacon@arm.com
>>>> CC: arnd@arndb.de
>>>> CC: marc.zyngier@arm.com
>>>> CC: linux@arm.linux.org.uk
>>>> CC: nico@linaro.org
>>>
>>> I'd suggest you also include in your series the patch I posted earlier 
>>> providing a runtime mdesc->smp_init method as well.
>>
>> OK.
>>
>>
>>> This way the 
>>> priority order would be:
>>>
>>> - If mdesc->smp_init is non null then use that.
>>>
>>> - Otherwise, if PSCI is available then use that.
>>>
>>> - Otherwise use mdesc->smp.
>>>
>>> This way, if the PSCI default has to be overriden (like in the MCPM case 
>>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
>>> this can be achieved at run time on a per mdesc basis.
>>
>> Actually that's not a bad idea, it could make everybody happy.
>> What about the following, in this precise order:
>>
>> - if a xen hypervisor node is present on device tree, use PSCI;
>> - otherwise if mdesc->smp_init is non null then use it;
>> - otherwise if PSCI is available then use it;
>> - otherwise use mdesc->smp.
>>
>> It's the most practical solution to satisfy everybody's needs.
> 
> Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> of its own?  Given it is going to tweak the DT passed to the kernel 
> anyway that shouldn't be a problem.

Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
only for DomU guests. For Dom0, you still need all the platform specific
code except smp_ops. However, I'm doubtful this would work without other
changes on more complicated platforms like OMAP.

I would say wait to add this until you have platforms that actually need
the first case.

Rob

> 
> That would be more eleguant than adding xen exception hooks in generic 
> code.
> 
> 
> Nicolas
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Rob Herring wrote:
> On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > 
> >> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> >>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> >>>
> >>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> >>>> This is required for SMP support in Dom0 on Xen.
> >>>>
> >>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> >>>> CC: will.deacon at arm.com
> >>>> CC: arnd at arndb.de
> >>>> CC: marc.zyngier at arm.com
> >>>> CC: linux at arm.linux.org.uk
> >>>> CC: nico at linaro.org
> >>>
> >>> I'd suggest you also include in your series the patch I posted earlier 
> >>> providing a runtime mdesc->smp_init method as well.
> >>
> >> OK.
> >>
> >>
> >>> This way the 
> >>> priority order would be:
> >>>
> >>> - If mdesc->smp_init is non null then use that.
> >>>
> >>> - Otherwise, if PSCI is available then use that.
> >>>
> >>> - Otherwise use mdesc->smp.
> >>>
> >>> This way, if the PSCI default has to be overriden (like in the MCPM case 
> >>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> >>> this can be achieved at run time on a per mdesc basis.
> >>
> >> Actually that's not a bad idea, it could make everybody happy.
> >> What about the following, in this precise order:
> >>
> >> - if a xen hypervisor node is present on device tree, use PSCI;
> >> - otherwise if mdesc->smp_init is non null then use it;
> >> - otherwise if PSCI is available then use it;
> >> - otherwise use mdesc->smp.
> >>
> >> It's the most practical solution to satisfy everybody's needs.
> > 
> > Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> > of its own?  Given it is going to tweak the DT passed to the kernel 
> > anyway that shouldn't be a problem.
> 
> Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
> only for DomU guests. For Dom0, you still need all the platform specific
> code except smp_ops. However, I'm doubtful this would work without other
> changes on more complicated platforms like OMAP.
> 
> I would say wait to add this until you have platforms that actually need
> the first case.

OK, that is not unreasonable.

What are the platforms that are going to use smp_init? Do we know how do
they intend to use it?

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Rob Herring wrote:
> On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > 
> >> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> >>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> >>>
> >>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> >>>> This is required for SMP support in Dom0 on Xen.
> >>>>
> >>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> >>>> CC: will.deacon@arm.com
> >>>> CC: arnd@arndb.de
> >>>> CC: marc.zyngier@arm.com
> >>>> CC: linux@arm.linux.org.uk
> >>>> CC: nico@linaro.org
> >>>
> >>> I'd suggest you also include in your series the patch I posted earlier 
> >>> providing a runtime mdesc->smp_init method as well.
> >>
> >> OK.
> >>
> >>
> >>> This way the 
> >>> priority order would be:
> >>>
> >>> - If mdesc->smp_init is non null then use that.
> >>>
> >>> - Otherwise, if PSCI is available then use that.
> >>>
> >>> - Otherwise use mdesc->smp.
> >>>
> >>> This way, if the PSCI default has to be overriden (like in the MCPM case 
> >>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> >>> this can be achieved at run time on a per mdesc basis.
> >>
> >> Actually that's not a bad idea, it could make everybody happy.
> >> What about the following, in this precise order:
> >>
> >> - if a xen hypervisor node is present on device tree, use PSCI;
> >> - otherwise if mdesc->smp_init is non null then use it;
> >> - otherwise if PSCI is available then use it;
> >> - otherwise use mdesc->smp.
> >>
> >> It's the most practical solution to satisfy everybody's needs.
> > 
> > Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> > of its own?  Given it is going to tweak the DT passed to the kernel 
> > anyway that shouldn't be a problem.
> 
> Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
> only for DomU guests. For Dom0, you still need all the platform specific
> code except smp_ops. However, I'm doubtful this would work without other
> changes on more complicated platforms like OMAP.
> 
> I would say wait to add this until you have platforms that actually need
> the first case.

OK, that is not unreasonable.

What are the platforms that are going to use smp_init? Do we know how do
they intend to use it?

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Mon, 1 Apr 2013, Stefano Stabellini wrote:

> On Fri, 29 Mar 2013, Rob Herring wrote:
> > On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> > > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > 
> > >> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > >>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > >>>
> > >>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > >>>> This is required for SMP support in Dom0 on Xen.
> > >>>>
> > >>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > >>>> CC: will.deacon at arm.com
> > >>>> CC: arnd at arndb.de
> > >>>> CC: marc.zyngier at arm.com
> > >>>> CC: linux at arm.linux.org.uk
> > >>>> CC: nico at linaro.org
> > >>>
> > >>> I'd suggest you also include in your series the patch I posted earlier 
> > >>> providing a runtime mdesc->smp_init method as well.
> > >>
> > >> OK.
> > >>
> > >>
> > >>> This way the 
> > >>> priority order would be:
> > >>>
> > >>> - If mdesc->smp_init is non null then use that.
> > >>>
> > >>> - Otherwise, if PSCI is available then use that.
> > >>>
> > >>> - Otherwise use mdesc->smp.
> > >>>
> > >>> This way, if the PSCI default has to be overriden (like in the MCPM case 
> > >>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > >>> this can be achieved at run time on a per mdesc basis.
> > >>
> > >> Actually that's not a bad idea, it could make everybody happy.
> > >> What about the following, in this precise order:
> > >>
> > >> - if a xen hypervisor node is present on device tree, use PSCI;
> > >> - otherwise if mdesc->smp_init is non null then use it;
> > >> - otherwise if PSCI is available then use it;
> > >> - otherwise use mdesc->smp.
> > >>
> > >> It's the most practical solution to satisfy everybody's needs.
> > > 
> > > Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> > > of its own?  Given it is going to tweak the DT passed to the kernel 
> > > anyway that shouldn't be a problem.
> > 
> > Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
> > only for DomU guests. For Dom0, you still need all the platform specific
> > code except smp_ops. However, I'm doubtful this would work without other
> > changes on more complicated platforms like OMAP.
> > 
> > I would say wait to add this until you have platforms that actually need
> > the first case.
> 
> OK, that is not unreasonable.
> 
> What are the platforms that are going to use smp_init? Do we know how do
> they intend to use it?

VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
VExpress, the MCPM layer needs to arbitrate power management operations 
on a per cluster basis.  In that case there is a MCPM specific set of 
SMP ops to be used, even if it may end up calling into PSCI.

But the important point is that we don't know beforehand what to use, 
especially with a kernel that can boot on multiple different VExpress 
configurations.  The decision has to be made at run time, and therefore 
a static default or mdesc->smp ops doesn't cut it.


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Mon, 1 Apr 2013, Stefano Stabellini wrote:

> On Fri, 29 Mar 2013, Rob Herring wrote:
> > On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> > > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > 
> > >> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > >>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > >>>
> > >>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > >>>> This is required for SMP support in Dom0 on Xen.
> > >>>>
> > >>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > >>>> CC: will.deacon@arm.com
> > >>>> CC: arnd@arndb.de
> > >>>> CC: marc.zyngier@arm.com
> > >>>> CC: linux@arm.linux.org.uk
> > >>>> CC: nico@linaro.org
> > >>>
> > >>> I'd suggest you also include in your series the patch I posted earlier 
> > >>> providing a runtime mdesc->smp_init method as well.
> > >>
> > >> OK.
> > >>
> > >>
> > >>> This way the 
> > >>> priority order would be:
> > >>>
> > >>> - If mdesc->smp_init is non null then use that.
> > >>>
> > >>> - Otherwise, if PSCI is available then use that.
> > >>>
> > >>> - Otherwise use mdesc->smp.
> > >>>
> > >>> This way, if the PSCI default has to be overriden (like in the MCPM case 
> > >>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > >>> this can be achieved at run time on a per mdesc basis.
> > >>
> > >> Actually that's not a bad idea, it could make everybody happy.
> > >> What about the following, in this precise order:
> > >>
> > >> - if a xen hypervisor node is present on device tree, use PSCI;
> > >> - otherwise if mdesc->smp_init is non null then use it;
> > >> - otherwise if PSCI is available then use it;
> > >> - otherwise use mdesc->smp.
> > >>
> > >> It's the most practical solution to satisfy everybody's needs.
> > > 
> > > Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> > > of its own?  Given it is going to tweak the DT passed to the kernel 
> > > anyway that shouldn't be a problem.
> > 
> > Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
> > only for DomU guests. For Dom0, you still need all the platform specific
> > code except smp_ops. However, I'm doubtful this would work without other
> > changes on more complicated platforms like OMAP.
> > 
> > I would say wait to add this until you have platforms that actually need
> > the first case.
> 
> OK, that is not unreasonable.
> 
> What are the platforms that are going to use smp_init? Do we know how do
> they intend to use it?

VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
VExpress, the MCPM layer needs to arbitrate power management operations 
on a per cluster basis.  In that case there is a MCPM specific set of 
SMP ops to be used, even if it may end up calling into PSCI.

But the important point is that we don't know beforehand what to use, 
especially with a kernel that can boot on multiple different VExpress 
configurations.  The decision has to be made at run time, and therefore 
a static default or mdesc->smp ops doesn't cut it.


Nicolas

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Mon, 1 Apr 2013, Nicolas Pitre wrote:
> On Mon, 1 Apr 2013, Stefano Stabellini wrote:
> 
> > On Fri, 29 Mar 2013, Rob Herring wrote:
> > > On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> > > > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > > 
> > > >> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > > >>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > >>>
> > > >>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > > >>>> This is required for SMP support in Dom0 on Xen.
> > > >>>>
> > > >>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > >>>> CC: will.deacon at arm.com
> > > >>>> CC: arnd at arndb.de
> > > >>>> CC: marc.zyngier at arm.com
> > > >>>> CC: linux at arm.linux.org.uk
> > > >>>> CC: nico at linaro.org
> > > >>>
> > > >>> I'd suggest you also include in your series the patch I posted earlier 
> > > >>> providing a runtime mdesc->smp_init method as well.
> > > >>
> > > >> OK.
> > > >>
> > > >>
> > > >>> This way the 
> > > >>> priority order would be:
> > > >>>
> > > >>> - If mdesc->smp_init is non null then use that.
> > > >>>
> > > >>> - Otherwise, if PSCI is available then use that.
> > > >>>
> > > >>> - Otherwise use mdesc->smp.
> > > >>>
> > > >>> This way, if the PSCI default has to be overriden (like in the MCPM case 
> > > >>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > > >>> this can be achieved at run time on a per mdesc basis.
> > > >>
> > > >> Actually that's not a bad idea, it could make everybody happy.
> > > >> What about the following, in this precise order:
> > > >>
> > > >> - if a xen hypervisor node is present on device tree, use PSCI;
> > > >> - otherwise if mdesc->smp_init is non null then use it;
> > > >> - otherwise if PSCI is available then use it;
> > > >> - otherwise use mdesc->smp.
> > > >>
> > > >> It's the most practical solution to satisfy everybody's needs.
> > > > 
> > > > Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> > > > of its own?  Given it is going to tweak the DT passed to the kernel 
> > > > anyway that shouldn't be a problem.
> > > 
> > > Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
> > > only for DomU guests. For Dom0, you still need all the platform specific
> > > code except smp_ops. However, I'm doubtful this would work without other
> > > changes on more complicated platforms like OMAP.
> > > 
> > > I would say wait to add this until you have platforms that actually need
> > > the first case.
> > 
> > OK, that is not unreasonable.
> > 
> > What are the platforms that are going to use smp_init? Do we know how do
> > they intend to use it?
> 
> VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
> VExpress, the MCPM layer needs to arbitrate power management operations 
> on a per cluster basis.  In that case there is a MCPM specific set of 
> SMP ops to be used, even if it may end up calling into PSCI.
> 
> But the important point is that we don't know beforehand what to use, 
> especially with a kernel that can boot on multiple different VExpress 
> configurations.  The decision has to be made at run time, and therefore 
> a static default or mdesc->smp ops doesn't cut it.

I certainly like the principle and I am in favor of anything that moves
the decisions at runtime. I have pulled the patch in the series, it's
going to be in the next version.

However I am concerned that these platform specific operations won't
work with Xen at all.
I am getting increasingly certain that we need a Xen specific check in
setup_arch to bump up of the priority of PSCI over anything else if Xen
is running. 

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Mon, 1 Apr 2013, Nicolas Pitre wrote:
> On Mon, 1 Apr 2013, Stefano Stabellini wrote:
> 
> > On Fri, 29 Mar 2013, Rob Herring wrote:
> > > On 03/29/2013 12:53 PM, Nicolas Pitre wrote:
> > > > On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > > 
> > > >> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > > >>> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> > > >>>
> > > >>>> If PSCI initializes correctly and PSCI SMP operations are available, use them.
> > > >>>> This is required for SMP support in Dom0 on Xen.
> > > >>>>
> > > >>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > > >>>> CC: will.deacon@arm.com
> > > >>>> CC: arnd@arndb.de
> > > >>>> CC: marc.zyngier@arm.com
> > > >>>> CC: linux@arm.linux.org.uk
> > > >>>> CC: nico@linaro.org
> > > >>>
> > > >>> I'd suggest you also include in your series the patch I posted earlier 
> > > >>> providing a runtime mdesc->smp_init method as well.
> > > >>
> > > >> OK.
> > > >>
> > > >>
> > > >>> This way the 
> > > >>> priority order would be:
> > > >>>
> > > >>> - If mdesc->smp_init is non null then use that.
> > > >>>
> > > >>> - Otherwise, if PSCI is available then use that.
> > > >>>
> > > >>> - Otherwise use mdesc->smp.
> > > >>>
> > > >>> This way, if the PSCI default has to be overriden (like in the MCPM case 
> > > >>> because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > > >>> this can be achieved at run time on a per mdesc basis.
> > > >>
> > > >> Actually that's not a bad idea, it could make everybody happy.
> > > >> What about the following, in this precise order:
> > > >>
> > > >> - if a xen hypervisor node is present on device tree, use PSCI;
> > > >> - otherwise if mdesc->smp_init is non null then use it;
> > > >> - otherwise if PSCI is available then use it;
> > > >> - otherwise use mdesc->smp.
> > > >>
> > > >> It's the most practical solution to satisfy everybody's needs.
> > > > 
> > > > Maybe I'm missing something obvious, but why can't xen declare a mdesc 
> > > > of its own?  Given it is going to tweak the DT passed to the kernel 
> > > > anyway that shouldn't be a problem.
> > > 
> > > Xen does have it's own mdesc. It is (or will be) mach-virt, but that is
> > > only for DomU guests. For Dom0, you still need all the platform specific
> > > code except smp_ops. However, I'm doubtful this would work without other
> > > changes on more complicated platforms like OMAP.
> > > 
> > > I would say wait to add this until you have platforms that actually need
> > > the first case.
> > 
> > OK, that is not unreasonable.
> > 
> > What are the platforms that are going to use smp_init? Do we know how do
> > they intend to use it?
> 
> VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
> VExpress, the MCPM layer needs to arbitrate power management operations 
> on a per cluster basis.  In that case there is a MCPM specific set of 
> SMP ops to be used, even if it may end up calling into PSCI.
> 
> But the important point is that we don't know beforehand what to use, 
> especially with a kernel that can boot on multiple different VExpress 
> configurations.  The decision has to be made at run time, and therefore 
> a static default or mdesc->smp ops doesn't cut it.

I certainly like the principle and I am in favor of anything that moves
the decisions at runtime. I have pulled the patch in the series, it's
going to be in the next version.

However I am concerned that these platform specific operations won't
work with Xen at all.
I am getting increasingly certain that we need a Xen specific check in
setup_arch to bump up of the priority of PSCI over anything else if Xen
is running. 

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Tue, 2 Apr 2013, Stefano Stabellini wrote:

> On Mon, 1 Apr 2013, Nicolas Pitre wrote:
> > On Mon, 1 Apr 2013, Stefano Stabellini wrote:
> > > What are the platforms that are going to use smp_init? Do we know how do
> > > they intend to use it?
> > 
> > VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
> > VExpress, the MCPM layer needs to arbitrate power management operations 
> > on a per cluster basis.  In that case there is a MCPM specific set of 
> > SMP ops to be used, even if it may end up calling into PSCI.
> > 
> > But the important point is that we don't know beforehand what to use, 
> > especially with a kernel that can boot on multiple different VExpress 
> > configurations.  The decision has to be made at run time, and therefore 
> > a static default or mdesc->smp ops doesn't cut it.
> 
> I certainly like the principle and I am in favor of anything that moves
> the decisions at runtime. I have pulled the patch in the series, it's
> going to be in the next version.
> 
> However I am concerned that these platform specific operations won't
> work with Xen at all.
> I am getting increasingly certain that we need a Xen specific check in
> setup_arch to bump up of the priority of PSCI over anything else if Xen
> is running. 

I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
this is going to make an easy match.  KVM might have an easier fit here.

But, in any case, even if the MCPM layer gets involved, if Xen is there 
then PSCI will end up being the ultimate interface anyway.

But let's cross that bridge when we get to it.  For now this is still a 
non existing problem.


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Tue, 2 Apr 2013, Stefano Stabellini wrote:

> On Mon, 1 Apr 2013, Nicolas Pitre wrote:
> > On Mon, 1 Apr 2013, Stefano Stabellini wrote:
> > > What are the platforms that are going to use smp_init? Do we know how do
> > > they intend to use it?
> > 
> > VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
> > VExpress, the MCPM layer needs to arbitrate power management operations 
> > on a per cluster basis.  In that case there is a MCPM specific set of 
> > SMP ops to be used, even if it may end up calling into PSCI.
> > 
> > But the important point is that we don't know beforehand what to use, 
> > especially with a kernel that can boot on multiple different VExpress 
> > configurations.  The decision has to be made at run time, and therefore 
> > a static default or mdesc->smp ops doesn't cut it.
> 
> I certainly like the principle and I am in favor of anything that moves
> the decisions at runtime. I have pulled the patch in the series, it's
> going to be in the next version.
> 
> However I am concerned that these platform specific operations won't
> work with Xen at all.
> I am getting increasingly certain that we need a Xen specific check in
> setup_arch to bump up of the priority of PSCI over anything else if Xen
> is running. 

I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
this is going to make an easy match.  KVM might have an easier fit here.

But, in any case, even if the MCPM layer gets involved, if Xen is there 
then PSCI will end up being the ultimate interface anyway.

But let's cross that bridge when we get to it.  For now this is still a 
non existing problem.


Nicolas

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Dave Martin]

On Tue, Apr 02, 2013 at 12:11:25PM -0400, Nicolas Pitre wrote:
> On Tue, 2 Apr 2013, Stefano Stabellini wrote:
> 
> > On Mon, 1 Apr 2013, Nicolas Pitre wrote:
> > > On Mon, 1 Apr 2013, Stefano Stabellini wrote:
> > > > What are the platforms that are going to use smp_init? Do we know how do
> > > > they intend to use it?
> > > 
> > > VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
> > > VExpress, the MCPM layer needs to arbitrate power management operations 
> > > on a per cluster basis.  In that case there is a MCPM specific set of 
> > > SMP ops to be used, even if it may end up calling into PSCI.
> > > 
> > > But the important point is that we don't know beforehand what to use, 
> > > especially with a kernel that can boot on multiple different VExpress 
> > > configurations.  The decision has to be made at run time, and therefore 
> > > a static default or mdesc->smp ops doesn't cut it.
> > 
> > I certainly like the principle and I am in favor of anything that moves
> > the decisions at runtime. I have pulled the patch in the series, it's
> > going to be in the next version.
> > 
> > However I am concerned that these platform specific operations won't
> > work with Xen at all.
> > I am getting increasingly certain that we need a Xen specific check in
> > setup_arch to bump up of the priority of PSCI over anything else if Xen
> > is running. 
> 
> I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
> this is going to make an easy match.  KVM might have an easier fit here.
> 
> But, in any case, even if the MCPM layer gets involved, if Xen is there 
> then PSCI will end up being the ultimate interface anyway.

Note that big.LITTLE != MCPM.  Virtualisation hosts might be large multi-
cluster systems, but the CPUs might be all of the same type.  MCPM or
similar would me needed for the multi-cluster power management even
though there is no big.LITTLE mix of CPUs.

> But let's cross that bridge when we get to it.  For now this is still a 
> non existing problem.

That's a big open question.  Either the host or hypervisor needs to be
very clever about scheduling guests, or you need to bind each guest virtual
CPU to a specific class of physical CPUs -- so, for example you provide
a guest with an explicit mix of bigs and littles.

All we can say about that for now is that it's a potential research area...

Cheers
---Dave

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Dave Martin]

On Tue, Apr 02, 2013 at 12:11:25PM -0400, Nicolas Pitre wrote:
> On Tue, 2 Apr 2013, Stefano Stabellini wrote:
> 
> > On Mon, 1 Apr 2013, Nicolas Pitre wrote:
> > > On Mon, 1 Apr 2013, Stefano Stabellini wrote:
> > > > What are the platforms that are going to use smp_init? Do we know how do
> > > > they intend to use it?
> > > 
> > > VExpress for one.  When booting on a big.LITTLE system such as TC2 on 
> > > VExpress, the MCPM layer needs to arbitrate power management operations 
> > > on a per cluster basis.  In that case there is a MCPM specific set of 
> > > SMP ops to be used, even if it may end up calling into PSCI.
> > > 
> > > But the important point is that we don't know beforehand what to use, 
> > > especially with a kernel that can boot on multiple different VExpress 
> > > configurations.  The decision has to be made at run time, and therefore 
> > > a static default or mdesc->smp ops doesn't cut it.
> > 
> > I certainly like the principle and I am in favor of anything that moves
> > the decisions at runtime. I have pulled the patch in the series, it's
> > going to be in the next version.
> > 
> > However I am concerned that these platform specific operations won't
> > work with Xen at all.
> > I am getting increasingly certain that we need a Xen specific check in
> > setup_arch to bump up of the priority of PSCI over anything else if Xen
> > is running. 
> 
> I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
> this is going to make an easy match.  KVM might have an easier fit here.
> 
> But, in any case, even if the MCPM layer gets involved, if Xen is there 
> then PSCI will end up being the ultimate interface anyway.

Note that big.LITTLE != MCPM.  Virtualisation hosts might be large multi-
cluster systems, but the CPUs might be all of the same type.  MCPM or
similar would me needed for the multi-cluster power management even
though there is no big.LITTLE mix of CPUs.

> But let's cross that bridge when we get to it.  For now this is still a 
> non existing problem.

That's a big open question.  Either the host or hypervisor needs to be
very clever about scheduling guests, or you need to bind each guest virtual
CPU to a specific class of physical CPUs -- so, for example you provide
a guest with an explicit mix of bigs and littles.

All we can say about that for now is that it's a potential research area...

Cheers
---Dave

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Tue, 9 Apr 2013, Dave Martin wrote:

> On Tue, Apr 02, 2013 at 12:11:25PM -0400, Nicolas Pitre wrote:
> > I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
> > this is going to make an easy match.  KVM might have an easier fit here.
> > 
> > But, in any case, even if the MCPM layer gets involved, if Xen is there 
> > then PSCI will end up being the ultimate interface anyway.
> 
> Note that big.LITTLE != MCPM.  Virtualisation hosts might be large multi-
> cluster systems, but the CPUs might be all of the same type.  MCPM or
> similar would me needed for the multi-cluster power management even
> though there is no big.LITTLE mix of CPUs.

Absolutely!  But in this case, there is no need for Xen to learn about 
the computing capacity differences between different CPU sets.

What I wanted to emphasize is the fact that, if Xen decides to expose a 
b.L topology to guests, then those guests must be b.L aware to make good 
scheduling decisions, etc.  Initially I suspect that guests will be 
confined to the same sets of CPUs to simplify things.  Or it could even 
migrate a guest between little and big CPUs like the switcher does.  
But a single guest probably won't span different CPU classes 
simultaneously initially.  And therefore it is unlikely that MCPM will 
be active inside a Xen guest for quite a while.

> > But let's cross that bridge when we get to it.  For now this is still a 
> > non existing problem.
> 
> That's a big open question.  Either the host or hypervisor needs to be
> very clever about scheduling guests, or you need to bind each guest virtual
> CPU to a specific class of physical CPUs -- so, for example you provide
> a guest with an explicit mix of bigs and littles.
> 
> All we can say about that for now is that it's a potential research area...

Absolutely.


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Tue, 9 Apr 2013, Dave Martin wrote:

> On Tue, Apr 02, 2013 at 12:11:25PM -0400, Nicolas Pitre wrote:
> > I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
> > this is going to make an easy match.  KVM might have an easier fit here.
> > 
> > But, in any case, even if the MCPM layer gets involved, if Xen is there 
> > then PSCI will end up being the ultimate interface anyway.
> 
> Note that big.LITTLE != MCPM.  Virtualisation hosts might be large multi-
> cluster systems, but the CPUs might be all of the same type.  MCPM or
> similar would me needed for the multi-cluster power management even
> though there is no big.LITTLE mix of CPUs.

Absolutely!  But in this case, there is no need for Xen to learn about 
the computing capacity differences between different CPU sets.

What I wanted to emphasize is the fact that, if Xen decides to expose a 
b.L topology to guests, then those guests must be b.L aware to make good 
scheduling decisions, etc.  Initially I suspect that guests will be 
confined to the same sets of CPUs to simplify things.  Or it could even 
migrate a guest between little and big CPUs like the switcher does.  
But a single guest probably won't span different CPU classes 
simultaneously initially.  And therefore it is unlikely that MCPM will 
be active inside a Xen guest for quite a while.

> > But let's cross that bridge when we get to it.  For now this is still a 
> > non existing problem.
> 
> That's a big open question.  Either the host or hypervisor needs to be
> very clever about scheduling guests, or you need to bind each guest virtual
> CPU to a specific class of physical CPUs -- so, for example you provide
> a guest with an explicit mix of bigs and littles.
> 
> All we can say about that for now is that it's a potential research area...

Absolutely.


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Tue, 9 Apr 2013, Dave Martin wrote:

> On Tue, Apr 02, 2013 at 12:11:25PM -0400, Nicolas Pitre wrote:
> > I'm concerned about mixing big.LITTLE and Xen as well.  I don't think 
> > this is going to make an easy match.  KVM might have an easier fit here.
> > 
> > But, in any case, even if the MCPM layer gets involved, if Xen is there 
> > then PSCI will end up being the ultimate interface anyway.
> 
> Note that big.LITTLE != MCPM.  Virtualisation hosts might be large multi-
> cluster systems, but the CPUs might be all of the same type.  MCPM or
> similar would me needed for the multi-cluster power management even
> though there is no big.LITTLE mix of CPUs.

Absolutely!  But in this case, there is no need for Xen to learn about 
the computing capacity differences between different CPU sets.

What I wanted to emphasize is the fact that, if Xen decides to expose a 
b.L topology to guests, then those guests must be b.L aware to make good 
scheduling decisions, etc.  Initially I suspect that guests will be 
confined to the same sets of CPUs to simplify things.  Or it could even 
migrate a guest between little and big CPUs like the switcher does.  
But a single guest probably won't span different CPU classes 
simultaneously initially.  And therefore it is unlikely that MCPM will 
be active inside a Xen guest for quite a while.

> > But let's cross that bridge when we get to it.  For now this is still a 
> > non existing problem.
> 
> That's a big open question.  Either the host or hypervisor needs to be
> very clever about scheduling guests, or you need to bind each guest virtual
> CPU to a specific class of physical CPUs -- so, for example you provide
> a guest with an explicit mix of bigs and littles.
> 
> All we can say about that for now is that it's a potential research area...

Absolutely.


Nicolas

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Fri, 29 Mar 2013, Stefano Stabellini wrote:

> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > This way the 
> > priority order would be:
> > 
> > - If mdesc->smp_init is non null then use that.
> > 
> > - Otherwise, if PSCI is available then use that.
> > 
> > - Otherwise use mdesc->smp.
> > 
> > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > this can be achieved at run time on a per mdesc basis.
> 
> Actually that's not a bad idea, it could make everybody happy.
> What about the following, in this precise order:
> 
> - if a xen hypervisor node is present on device tree, use PSCI;
> - otherwise if mdesc->smp_init is non null then use it;
> - otherwise if PSCI is available then use it;
> - otherwise use mdesc->smp.
> 
> It's the most practical solution to satisfy everybody's needs.

Regardless of my previous email suggesting a mdesc for xen, I still 
don't understand why you need this absolute priority for Xen.  Isn't my 
original suggestion sufficient?

The likely reason why mdesc->smp_init might be needed is to provide an 
extra encapsulation layer before actually using PSCI instead of using it 
directly.  Why would you need to bypass that?


Nicolas

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Nicolas Pitre]

On Fri, 29 Mar 2013, Stefano Stabellini wrote:

> On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > This way the 
> > priority order would be:
> > 
> > - If mdesc->smp_init is non null then use that.
> > 
> > - Otherwise, if PSCI is available then use that.
> > 
> > - Otherwise use mdesc->smp.
> > 
> > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > this can be achieved at run time on a per mdesc basis.
> 
> Actually that's not a bad idea, it could make everybody happy.
> What about the following, in this precise order:
> 
> - if a xen hypervisor node is present on device tree, use PSCI;
> - otherwise if mdesc->smp_init is non null then use it;
> - otherwise if PSCI is available then use it;
> - otherwise use mdesc->smp.
> 
> It's the most practical solution to satisfy everybody's needs.

Regardless of my previous email suggesting a mdesc for xen, I still 
don't understand why you need this absolute priority for Xen.  Isn't my 
original suggestion sufficient?

The likely reason why mdesc->smp_init might be needed is to provide an 
extra encapsulation layer before actually using PSCI instead of using it 
directly.  Why would you need to bypass that?


Nicolas

[PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
> > On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > > This way the 
> > > priority order would be:
> > > 
> > > - If mdesc->smp_init is non null then use that.
> > > 
> > > - Otherwise, if PSCI is available then use that.
> > > 
> > > - Otherwise use mdesc->smp.
> > > 
> > > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > > this can be achieved at run time on a per mdesc basis.
> > 
> > Actually that's not a bad idea, it could make everybody happy.
> > What about the following, in this precise order:
> > 
> > - if a xen hypervisor node is present on device tree, use PSCI;
> > - otherwise if mdesc->smp_init is non null then use it;
> > - otherwise if PSCI is available then use it;
> > - otherwise use mdesc->smp.
> > 
> > It's the most practical solution to satisfy everybody's needs.
> 
> Regardless of my previous email suggesting a mdesc for xen, I still 
> don't understand why you need this absolute priority for Xen.  Isn't my 
> original suggestion sufficient?
> 
> The likely reason why mdesc->smp_init might be needed is to provide an 
> extra encapsulation layer before actually using PSCI instead of using it 
> directly.  Why would you need to bypass that?

Uhm.. maybe I wouldn't, I am not 100% sure TBH.

I am expecting that this "extra encapsulation layer" would read/write
some platform specific registers that Xen doesn't export.
If this is the case, then we would need to bypass it.

But if it's just to mangle parameters, then it should be OK.

Re: [PATCH v4 2/2] arm: prefer PSCI for SMP bringup

[Stefano Stabellini]

On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> On Fri, 29 Mar 2013, Stefano Stabellini wrote:
> 
> > On Fri, 29 Mar 2013, Nicolas Pitre wrote:
> > > This way the 
> > > priority order would be:
> > > 
> > > - If mdesc->smp_init is non null then use that.
> > > 
> > > - Otherwise, if PSCI is available then use that.
> > > 
> > > - Otherwise use mdesc->smp.
> > > 
> > > This way, if the PSCI default has to be overriden (like in the MCPM case 
> > > because it needs to wrap PSCI itself, or to cover Rob's concern) then 
> > > this can be achieved at run time on a per mdesc basis.
> > 
> > Actually that's not a bad idea, it could make everybody happy.
> > What about the following, in this precise order:
> > 
> > - if a xen hypervisor node is present on device tree, use PSCI;
> > - otherwise if mdesc->smp_init is non null then use it;
> > - otherwise if PSCI is available then use it;
> > - otherwise use mdesc->smp.
> > 
> > It's the most practical solution to satisfy everybody's needs.
> 
> Regardless of my previous email suggesting a mdesc for xen, I still 
> don't understand why you need this absolute priority for Xen.  Isn't my 
> original suggestion sufficient?
> 
> The likely reason why mdesc->smp_init might be needed is to provide an 
> extra encapsulation layer before actually using PSCI instead of using it 
> directly.  Why would you need to bypass that?

Uhm.. maybe I wouldn't, I am not 100% sure TBH.

I am expecting that this "extra encapsulation layer" would read/write
some platform specific registers that Xen doesn't export.
If this is the case, then we would need to bypass it.

But if it's just to mangle parameters, then it should be OK.