Re: [CFT] Always enable SMP mode on MP capable CPUs

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Tony Lindgren]

* Russell King - ARM Linux <linux@armlinux.org.uk> [170519 10:19]:
> On Fri, May 19, 2017 at 02:07:28PM -0300, Fabio Estevam wrote:
> > On Thu, May 18, 2017 at 3:09 PM, Fabio Estevam <festevam@gmail.com> wrote:
> > 
> > > I tested this patch on a mx6ul, which is CortexA7 UP using USB serial
> > > download mode.
> > >
> > > In this boot mode the ROM code does not set the SMP bit and I can
> > > notice a very slow boot progress with CONFIG_SMP=n with an original
> > > 4.12-rc1.
> > >
> > > With this patch applied it boots with the correct speed, so:
> > 
> > However the kernel still takes a long time to decompress when the
> > bootloader or ROM does not set the SMP bit.
> > 
> > After it decompress, then it boots quickly.
> > 
> > Should the SMP bit be set inside arch/arm/boot/compressed/head.S?
> 
> We could expand the table to positively identify the SMP capable CPUs
> and enable the SMP bit there, but it's going to add a lot of entries
> there (one for each specific ARMv7 MP CPU) and is going to have to be
> endlessly added to each time a new SMP CPU comes out.

Looks like commit e414690faa81 ("ARM: always enable SMP mode on SMP cores")
causes booting to fail early at least on omap4430 duovero. It's a bit of
a mystery as it is SMP.. And pandboard es (4460) and droid 4 (4430) work
without the conditional write..

Partial revert like below seems to make it boot again.

Regards,

Tony

8< -------------------
diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
index 4d48a4cf563b..ad653ec0835e 100644
--- a/arch/arm/mm/proc-v7.S
+++ b/arch/arm/mm/proc-v7.S
@@ -289,7 +289,8 @@ __v7_ca17mp_setup:
 	mrc	p15, 0, r0, c1, c0, 1
 	orr	r10, r10, #(1 << 6)		@ Enable SMP/nAMP mode
 	orr	r10, r10, r0			@ Set required bits
-	mcr	p15, 0, r10, c1, c0, 1		@ No, update register
+	teq	r10, r0				@ Were they already set?
+	mcrne	p15, 0, r10, c1, c0, 1		@ No, update register
 	b	__v7_setup_cont
 
 /*

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Russell King - ARM Linux]

On Wed, May 24, 2017 at 08:38:47AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@armlinux.org.uk> [170519 10:19]:
> > We could expand the table to positively identify the SMP capable CPUs
> > and enable the SMP bit there, but it's going to add a lot of entries
> > there (one for each specific ARMv7 MP CPU) and is going to have to be
> > endlessly added to each time a new SMP CPU comes out.
> 
> Looks like commit e414690faa81 ("ARM: always enable SMP mode on SMP cores")
> causes booting to fail early at least on omap4430 duovero. It's a bit of
> a mystery as it is SMP.. And pandboard es (4460) and droid 4 (4430) work
> without the conditional write..
> 
> Partial revert like below seems to make it boot again.

Thanks for testing, I'll integrate your change, and it should be in
tonight's linux-next.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Tony Lindgren]

* Russell King - ARM Linux <linux@armlinux.org.uk> [170525 07:51]:
> On Wed, May 24, 2017 at 08:38:47AM -0700, Tony Lindgren wrote:
> > * Russell King - ARM Linux <linux@armlinux.org.uk> [170519 10:19]:
> > > We could expand the table to positively identify the SMP capable CPUs
> > > and enable the SMP bit there, but it's going to add a lot of entries
> > > there (one for each specific ARMv7 MP CPU) and is going to have to be
> > > endlessly added to each time a new SMP CPU comes out.
> > 
> > Looks like commit e414690faa81 ("ARM: always enable SMP mode on SMP cores")
> > causes booting to fail early at least on omap4430 duovero. It's a bit of
> > a mystery as it is SMP.. And pandboard es (4460) and droid 4 (4430) work
> > without the conditional write..
> > 
> > Partial revert like below seems to make it boot again.
> 
> Thanks for testing, I'll integrate your change, and it should be in
> tonight's linux-next.

OK. That may not work on am437x though as it's UP.

Afzal, did the change that I posted work for you?

Regards,

Tony

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Russell King - ARM Linux]

On Thu, May 25, 2017 at 08:11:03AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@armlinux.org.uk> [170525 07:51]:
> > On Wed, May 24, 2017 at 08:38:47AM -0700, Tony Lindgren wrote:
> > > * Russell King - ARM Linux <linux@armlinux.org.uk> [170519 10:19]:
> > > > We could expand the table to positively identify the SMP capable CPUs
> > > > and enable the SMP bit there, but it's going to add a lot of entries
> > > > there (one for each specific ARMv7 MP CPU) and is going to have to be
> > > > endlessly added to each time a new SMP CPU comes out.
> > > 
> > > Looks like commit e414690faa81 ("ARM: always enable SMP mode on SMP cores")
> > > causes booting to fail early at least on omap4430 duovero. It's a bit of
> > > a mystery as it is SMP.. And pandboard es (4460) and droid 4 (4430) work
> > > without the conditional write..
> > > 
> > > Partial revert like below seems to make it boot again.
> > 
> > Thanks for testing, I'll integrate your change, and it should be in
> > tonight's linux-next.
> 
> OK. That may not work on am437x though as it's UP.

Maybe this is just too complicated to grasp - in which case just test
tonight's linux-next on *all* your boards irrespective of the CPU, and
let me know if there's a regression due to this patch, giving me full
details of your CPUs (CPU ID numbers, number of CPUs, etc.)  If no one
reports a regression that doesn't get fixed, I'll merge it during the
next merge window.

(I can't believe that this is proving to be soo complicated...)

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Tony Lindgren]

* Russell King - ARM Linux <linux@armlinux.org.uk> [170525 11:09]:
> On Thu, May 25, 2017 at 08:11:03AM -0700, Tony Lindgren wrote:
> > 
> > OK. That may not work on am437x though as it's UP.
> 
> Maybe this is just too complicated to grasp - in which case just test
> tonight's linux-next on *all* your boards irrespective of the CPU, and
> let me know if there's a regression due to this patch, giving me full
> details of your CPUs (CPU ID numbers, number of CPUs, etc.)  If no one
> reports a regression that doesn't get fixed, I'll merge it during the
> next merge window.

OK so got my am437x-idk back up and running again. I'm kind of
running out of space here..

And as suspected it won't boot with these changes. It's a UP core:

CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: TI AM437x Industrial Development Kit

Regards,

Tony

Re: [CFT] Always enable SMP mode on MP capable CPUs

[afzal mohammed]

Hi,

On Thu, May 25, 2017 at 03:13:33PM -0700, Tony Lindgren wrote:

> OK so got my am437x-idk back
> 
> And as suspected it won't boot with these changes.

Tested here too with your additional changes included & seeing same
behavior here as well on AM437x.

As an aside, when Russell posted this patch a week back itself was
suspecting this. i had a certain view on AM437x w.r.t SMP issue back
while officially working on it & since now not officially involved
with the issue at hand, i was uncomfortable dealing with this. But as
Russell's patch was not noticed by others w.r.t AM437x, upon being
able to confirm it yesterday, i at least wanted to bring it to notice
& runaway from the scene.

Regards
afzal

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Russell King - ARM Linux]

On Fri, May 26, 2017 at 04:52:10PM +0530, afzal mohammed wrote:
> Hi,
> 
> On Thu, May 25, 2017 at 03:13:33PM -0700, Tony Lindgren wrote:
> 
> > OK so got my am437x-idk back
> > 
> > And as suspected it won't boot with these changes.
> 
> Tested here too with your additional changes included & seeing same
> behavior here as well on AM437x.
> 
> As an aside, when Russell posted this patch a week back itself was
> suspecting this. i had a certain view on AM437x w.r.t SMP issue back
> while officially working on it & since now not officially involved
> with the issue at hand, i was uncomfortable dealing with this. But as
> Russell's patch was not noticed by others w.r.t AM437x, upon being
> able to confirm it yesterday, i at least wanted to bring it to notice
> & runaway from the scene.

Right, so I think the summary is: the kernel can't set the SMP bit
correctly - it doesn't have the information at this point in time
to know definitively whether the bit needs to be set or not.

So, those who are suffering issues with their SMP cores in uniprocessor
systems need to ensure that their boot loader correctly configures the
CPU cores.

Thanks for testing.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Tony Lindgren]

* Russell King - ARM Linux <linux@armlinux.org.uk> [170527 06:26]:
> On Fri, May 26, 2017 at 04:52:10PM +0530, afzal mohammed wrote:
> > Hi,
> > 
> > On Thu, May 25, 2017 at 03:13:33PM -0700, Tony Lindgren wrote:
> > 
> > > OK so got my am437x-idk back
> > > 
> > > And as suspected it won't boot with these changes.
> > 
> > Tested here too with your additional changes included & seeing same
> > behavior here as well on AM437x.
> > 
> > As an aside, when Russell posted this patch a week back itself was
> > suspecting this. i had a certain view on AM437x w.r.t SMP issue back
> > while officially working on it & since now not officially involved
> > with the issue at hand, i was uncomfortable dealing with this. But as
> > Russell's patch was not noticed by others w.r.t AM437x, upon being
> > able to confirm it yesterday, i at least wanted to bring it to notice
> > & runaway from the scene.
> 
> Right, so I think the summary is: the kernel can't set the SMP bit
> correctly - it doesn't have the information at this point in time
> to know definitively whether the bit needs to be set or not.
> 
> So, those who are suffering issues with their SMP cores in uniprocessor
> systems need to ensure that their boot loader correctly configures the
> CPU cores.

Russell, current Linux next still has commit e7cf836e9757 ("ARM:
always enable SMP mode on SMP cores"). You're planning to drop
it, right?

It causes a regression on am437x to not boot as we can't set the
SMP bit on it.

Regards,

Tony

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Russell King - ARM Linux]

On Wed, May 31, 2017 at 07:39:15AM -0700, Tony Lindgren wrote:
> * Russell King - ARM Linux <linux@armlinux.org.uk> [170527 06:26]:
> > On Fri, May 26, 2017 at 04:52:10PM +0530, afzal mohammed wrote:
> > > Hi,
> > > 
> > > On Thu, May 25, 2017 at 03:13:33PM -0700, Tony Lindgren wrote:
> > > 
> > > > OK so got my am437x-idk back
> > > > 
> > > > And as suspected it won't boot with these changes.
> > > 
> > > Tested here too with your additional changes included & seeing same
> > > behavior here as well on AM437x.
> > > 
> > > As an aside, when Russell posted this patch a week back itself was
> > > suspecting this. i had a certain view on AM437x w.r.t SMP issue back
> > > while officially working on it & since now not officially involved
> > > with the issue at hand, i was uncomfortable dealing with this. But as
> > > Russell's patch was not noticed by others w.r.t AM437x, upon being
> > > able to confirm it yesterday, i at least wanted to bring it to notice
> > > & runaway from the scene.
> > 
> > Right, so I think the summary is: the kernel can't set the SMP bit
> > correctly - it doesn't have the information at this point in time
> > to know definitively whether the bit needs to be set or not.
> > 
> > So, those who are suffering issues with their SMP cores in uniprocessor
> > systems need to ensure that their boot loader correctly configures the
> > CPU cores.
> 
> Russell, current Linux next still has commit e7cf836e9757 ("ARM:
> always enable SMP mode on SMP cores"). You're planning to drop
> it, right?

Please have some patience.

Correct, it should be gone from tonight's linux-next.

The UK has two holidays in May, one of them was this Monday.  I dropped
the patch yesterday, which is the first opportunity since sending that
mail, but due to other pressures yesterday, my tree was in no state to
be pushed out for linux-next's grab last night.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Tony Lindgren]

* Russell King - ARM Linux <linux@armlinux.org.uk> [170531 08:03]:
> On Wed, May 31, 2017 at 07:39:15AM -0700, Tony Lindgren wrote:
> > * Russell King - ARM Linux <linux@armlinux.org.uk> [170527 06:26]:
> > > On Fri, May 26, 2017 at 04:52:10PM +0530, afzal mohammed wrote:
> > > > Hi,
> > > > 
> > > > On Thu, May 25, 2017 at 03:13:33PM -0700, Tony Lindgren wrote:
> > > > 
> > > > > OK so got my am437x-idk back
> > > > > 
> > > > > And as suspected it won't boot with these changes.
> > > > 
> > > > Tested here too with your additional changes included & seeing same
> > > > behavior here as well on AM437x.
> > > > 
> > > > As an aside, when Russell posted this patch a week back itself was
> > > > suspecting this. i had a certain view on AM437x w.r.t SMP issue back
> > > > while officially working on it & since now not officially involved
> > > > with the issue at hand, i was uncomfortable dealing with this. But as
> > > > Russell's patch was not noticed by others w.r.t AM437x, upon being
> > > > able to confirm it yesterday, i at least wanted to bring it to notice
> > > > & runaway from the scene.
> > > 
> > > Right, so I think the summary is: the kernel can't set the SMP bit
> > > correctly - it doesn't have the information at this point in time
> > > to know definitively whether the bit needs to be set or not.
> > > 
> > > So, those who are suffering issues with their SMP cores in uniprocessor
> > > systems need to ensure that their boot loader correctly configures the
> > > CPU cores.
> > 
> > Russell, current Linux next still has commit e7cf836e9757 ("ARM:
> > always enable SMP mode on SMP cores"). You're planning to drop
> > it, right?
> 
> Please have some patience.
> 
> Correct, it should be gone from tonight's linux-next.
> 
> The UK has two holidays in May, one of them was this Monday.  I dropped
> the patch yesterday, which is the first opportunity since sending that
> mail, but due to other pressures yesterday, my tree was in no state to
> be pushed out for linux-next's grab last night.

OK thanks for confirming!

Tony

Re: [CFT] Always enable SMP mode on MP capable CPUs

[afzal mohammed]

To make those handling AM43x aware, To'ing l-o

Hi,

On Thu, May 18, 2017 at 11:52:10AM +0100, Russell King - ARM Linux wrote:
> As a result of a recent bug report, it has been found that certain CPUs
> must always have SMP mode enabled in order for the caches to work.
> 
> Remove the conditional on setting the SMP bit(s).
> 
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> ---

> I'm aware of a Cortex
> A9 CPU out there that does mis-identify itself as SMP capable but
> isn't:
> 
>         @ Core indicates it is SMP. Check for Aegis SOC where a single
>         @ Cortex-A9 CPU is present but SMP operations fault.
> 
> This will also need testing.

With this change, AM437x, i.e. the above mentioned SoC stops
booting (multi_v7 config)

afzal

> 
>  arch/arm/mm/proc-v7.S | 8 ++------
>  1 file changed, 2 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
> index 01d64c0b2563..4d48a4cf563b 100644
> --- a/arch/arm/mm/proc-v7.S
> +++ b/arch/arm/mm/proc-v7.S
> @@ -286,14 +286,10 @@ ENDPROC(cpu_pj4b_do_resume)
>  	stmia	r12, {r1-r6, lr}		@ v7_invalidate_l1 touches r0-r6
>  	bl      v7_invalidate_l1
>  	ldmia	r12, {r1-r6, lr}
> -#ifdef CONFIG_SMP
> +	mrc	p15, 0, r0, c1, c0, 1
>  	orr	r10, r10, #(1 << 6)		@ Enable SMP/nAMP mode
> -	ALT_SMP(mrc	p15, 0, r0, c1, c0, 1)
> -	ALT_UP(mov	r0, r10)		@ fake it for UP
>  	orr	r10, r10, r0			@ Set required bits
> -	teq	r10, r0				@ Were they already set?
> -	mcrne	p15, 0, r10, c1, c0, 1		@ No, update register
> -#endif
> +	mcr	p15, 0, r10, c1, c0, 1		@ No, update register
>  	b	__v7_setup_cont
>  
>  /*
> 
> -- 
> RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
> FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
> according to speedtest.net.
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [CFT] Always enable SMP mode on MP capable CPUs

[Tony Lindgren]

* Russell King - ARM Linux <linux@armlinux.org.uk> [170525 10:00]:
> On Thu, May 25, 2017 at 09:15:19AM -0700, Florian Fainelli wrote:
> > On 05/18/2017 03:52 AM, Russell King - ARM Linux wrote:
> > > As a result of a recent bug report, it has been found that certain CPUs
> > > must always have SMP mode enabled in order for the caches to work.
> > > 
> > > Remove the conditional on setting the SMP bit(s).
> > > 
> > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > > ---
> > > This needs to be tested on:
> > > 
> > > - Cortex A5MP
> > > - Cortex A9MP
> > > - Cortex R7MP
> > > - Cortex A7MP
> > > - Cortex A12MP
> > > - Cortex A15MP
> > > - Cortex A17MP
> > > - Brahma B15
> > 
> > Sorry just saw this, what kind of test do you want me to run on B15?
> > Should I build a !SMP kernel, or force a SMP kernel with maxcpus=1?
> 
> What I'm after is testing on any single-core systems with these SMP
> capable cores.  If the core never appears in a single-core configuration,
> then please let me know so it can be crossed off the list.
> 
> With the bug that crept in fixed (as pointed out by Tony) there is no
> difference for kernels built with SMP enabled and detected as a SMP
> capable CPU.

At least am437x needs to be tested as it's UP with some quirks.

Regards,

Tony