Possible regression in kexec on ARM ARMv6 and ARMv7 cores

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Simon Horman]

Hi Will, Hi All,

it appears that "ARM: proc: add definition of cpu_reset for
 ARMv6 and ARMv7 cores" (f4daf06fc23b99df5ca5b3e892428b91e148cc52),
which was introduced for 3.1-rc1, causes a regression and that
kexec no longer works on ARM. The board that I am testing
on is a Renesas Mackerel which has an SH7372 (ARMv7) processor.

I have included the patch below for reference.


commit f4daf06fc23b99df5ca5b3e892428b91e148cc52
Author: Will Deacon <will.deacon@arm.com>
Date:   Mon Jun 6 12:27:34 2011 +0100

    ARM: proc: add definition of cpu_reset for ARMv6 and ARMv7 cores
    
    This patch adds simple definitions of cpu_reset for ARMv6 and ARMv7
    cores, which disable the MMU via the SCTLR.
    
    Signed-off-by: Will Deacon <will.deacon@arm.com>

diff --git a/arch/arm/mm/proc-v6.S b/arch/arm/mm/proc-v6.S
index 5ec1543..aedf3c5 100644
--- a/arch/arm/mm/proc-v6.S
+++ b/arch/arm/mm/proc-v6.S
@@ -56,6 +56,11 @@ ENTRY(cpu_v6_proc_fin)
  */
 	.align	5
 ENTRY(cpu_v6_reset)
+	mrc	p15, 0, r1, c1, c0, 0		@ ctrl register
+	bic	r1, r1, #0x1			@ ...............m
+	mcr	p15, 0, r1, c1, c0, 0		@ disable MMU
+	mov	r1, #0
+	mcr	p15, 0, r1, c7, c5, 4		@ ISB
 	mov	pc, r0
 
 /*
diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S
index 5932854..54d1a63 100644
--- a/arch/arm/mm/proc-v7.S
+++ b/arch/arm/mm/proc-v7.S
@@ -58,9 +58,16 @@ ENDPROC(cpu_v7_proc_fin)
  *	to what would be the reset vector.
  *
  *	- loc   - location to jump to for soft reset
+ *
+ *	This code must be executed using a flat identity mapping with
+ *      caches disabled.
  */
 	.align	5
 ENTRY(cpu_v7_reset)
+	mrc	p15, 0, r1, c1, c0, 0		@ ctrl register
+	bic	r1, r1, #0x1			@ ...............m
+	mcr	p15, 0, r1, c1, c0, 0		@ disable MMU
+	isb
 	mov	pc, r0
 ENDPROC(cpu_v7_reset)
 

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Will Deacon]

On Thu, Oct 20, 2011 at 05:24:45AM +0100, Simon Horman wrote:
> Hi Will, Hi All,

Hi Simon,

> it appears that "ARM: proc: add definition of cpu_reset for
>  ARMv6 and ARMv7 cores" (f4daf06fc23b99df5ca5b3e892428b91e148cc52),
> which was introduced for 3.1-rc1, causes a regression and that
> kexec no longer works on ARM. The board that I am testing
> on is a Renesas Mackerel which has an SH7372 (ARMv7) processor.

Wow, I'm surprised the old code worked at all on an ARMv7 CPU! It's certainly
highly unlikely to work by the letter of the architecture, so I guess the
planets aligned in just the right way for your particular implementation.

Please can you try my kexec/mmu-off patches? They should add the bits and
pieces you need for kexec to work reliably on a UP system. I plan to finish
these off next week in Prague so hopefully they'll hit mainline in the near
future.

https://github.com/wdeacon/linux-wd/commits/kexec/mmu-off

Will

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Simon Horman]

On Thu, Oct 20, 2011 at 08:01:06AM +0100, Will Deacon wrote:
> On Thu, Oct 20, 2011 at 05:24:45AM +0100, Simon Horman wrote:
> > Hi Will, Hi All,
> 
> Hi Simon,

Hi Will,

> > it appears that "ARM: proc: add definition of cpu_reset for
> >  ARMv6 and ARMv7 cores" (f4daf06fc23b99df5ca5b3e892428b91e148cc52),
> > which was introduced for 3.1-rc1, causes a regression and that
> > kexec no longer works on ARM. The board that I am testing
> > on is a Renesas Mackerel which has an SH7372 (ARMv7) processor.
> 
> Wow, I'm surprised the old code worked at all on an ARMv7 CPU! It's certainly
> highly unlikely to work by the letter of the architecture, so I guess the
> planets aligned in just the right way for your particular implementation.
> 
> Please can you try my kexec/mmu-off patches? They should add the bits and
> pieces you need for kexec to work reliably on a UP system. I plan to finish
> these off next week in Prague so hopefully they'll hit mainline in the near
> future.
> 
> https://github.com/wdeacon/linux-wd/commits/kexec/mmu-off

Thanks, I will try your patches and let you know how I go.

You mention UP in particular, which I will test.
Do you have any thoughts on SMP?

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Simon Horman]

On Thu, Oct 20, 2011 at 05:08:24PM +0900, Simon Horman wrote:
> On Thu, Oct 20, 2011 at 08:01:06AM +0100, Will Deacon wrote:
> > On Thu, Oct 20, 2011 at 05:24:45AM +0100, Simon Horman wrote:
> > > Hi Will, Hi All,
> > 
> > Hi Simon,
> 
> Hi Will,
> 
> > > it appears that "ARM: proc: add definition of cpu_reset for
> > >  ARMv6 and ARMv7 cores" (f4daf06fc23b99df5ca5b3e892428b91e148cc52),
> > > which was introduced for 3.1-rc1, causes a regression and that
> > > kexec no longer works on ARM. The board that I am testing
> > > on is a Renesas Mackerel which has an SH7372 (ARMv7) processor.
> > 
> > Wow, I'm surprised the old code worked at all on an ARMv7 CPU! It's certainly
> > highly unlikely to work by the letter of the architecture, so I guess the
> > planets aligned in just the right way for your particular implementation.
> > 
> > Please can you try my kexec/mmu-off patches? They should add the bits and
> > pieces you need for kexec to work reliably on a UP system. I plan to finish
> > these off next week in Prague so hopefully they'll hit mainline in the near
> > future.
> > 
> > https://github.com/wdeacon/linux-wd/commits/kexec/mmu-off
> 
> Thanks, I will try your patches and let you know how I go.

I have tested the kexec/mmu-off branch of your tree on the UP board
I mentioned above and kexec works :)

> You mention UP in particular, which I will test.
> Do you have any thoughts on SMP?

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Will Deacon]

On Fri, Oct 21, 2011 at 09:34:26AM +0100, Simon Horman wrote:
> I have tested the kexec/mmu-off branch of your tree on the UP board
> I mentioned above and kexec works :)

Hurrah! Thanks for giving it a spin.

> > You mention UP in particular, which I will test.
> > Do you have any thoughts on SMP?

Yes. Currently [i.e. on my branch above] you can do SMP kexec using the CPU
hotplug platform_cpu_kill callback to take down the CPUs in some
platform-specific manner.

The more difficult case is when you want to offline the secondary CPUs into
a pen and then boot them in the new kernel. I did get some of this working,
but there are outstanding issues with whether the pen should be at a fixed
location or not. If not, then we need a way to tell the new kernel where it
is, which may involve updating the DT blob...

Will

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Simon Horman]

On Fri, Oct 21, 2011 at 09:46:41AM +0100, Will Deacon wrote:
> On Fri, Oct 21, 2011 at 09:34:26AM +0100, Simon Horman wrote:
> > I have tested the kexec/mmu-off branch of your tree on the UP board
> > I mentioned above and kexec works :)
> 
> Hurrah! Thanks for giving it a spin.
> 
> > > You mention UP in particular, which I will test.
> > > Do you have any thoughts on SMP?
> 
> Yes. Currently [i.e. on my branch above] you can do SMP kexec using the CPU
> hotplug platform_cpu_kill callback to take down the CPUs in some
> platform-specific manner.

Thanks, I will take a look into it.

> The more difficult case is when you want to offline the secondary CPUs into
> a pen and then boot them in the new kernel. I did get some of this working,
> but there are outstanding issues with whether the pen should be at a fixed
> location or not. If not, then we need a way to tell the new kernel where it
> is, which may involve updating the DT blob...

Is the implication that the (working) callback method does not
give the second kernel any secondary CPUs?

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Will Deacon]

On Fri, Oct 21, 2011 at 09:59:58AM +0100, Simon Horman wrote:
> On Fri, Oct 21, 2011 at 09:46:41AM +0100, Will Deacon wrote:
> > The more difficult case is when you want to offline the secondary CPUs into
> > a pen and then boot them in the new kernel. I did get some of this working,
> > but there are outstanding issues with whether the pen should be at a fixed
> > location or not. If not, then we need a way to tell the new kernel where it
> > is, which may involve updating the DT blob...
> 
> Is the implication that the (working) callback method does not
> give the second kernel any secondary CPUs?

What the callback does it up to you. You just need to make sure that you can
cope with the kernel disappearing later on, so you'll probably need some
hardware mechanism for offlining a CPU (for example, placing it in SRAM
or cutting the power).

Will

Possible regression in kexec on ARM ARMv6 and ARMv7 cores

[Simon Horman]

On Fri, Oct 21, 2011 at 10:15:24AM +0100, Will Deacon wrote:
> On Fri, Oct 21, 2011 at 09:59:58AM +0100, Simon Horman wrote:
> > On Fri, Oct 21, 2011 at 09:46:41AM +0100, Will Deacon wrote:
> > > The more difficult case is when you want to offline the secondary CPUs into
> > > a pen and then boot them in the new kernel. I did get some of this working,
> > > but there are outstanding issues with whether the pen should be at a fixed
> > > location or not. If not, then we need a way to tell the new kernel where it
> > > is, which may involve updating the DT blob...
> > 
> > Is the implication that the (working) callback method does not
> > give the second kernel any secondary CPUs?
> 
> What the callback does it up to you. You just need to make sure that you can
> cope with the kernel disappearing later on, so you'll probably need some
> hardware mechanism for offlining a CPU (for example, placing it in SRAM
> or cutting the power).

Thanks, it seems that I now have a bit of work to do implementing
a callback.