[PATCH/RFT 0/4] ARM: shmobile: R-Car Gen2: Allow booting secondary CPU cores in debug mode

[PATCH/RFT 0/4] ARM: shmobile: R-Car Gen2: Allow booting secondary CPU cores in debug mode

[Geert Uytterhoeven]

	Hi Simon, Magnus,

This patch series is an attempt to allow booting secondary CPU cores on
R-Car Gen2 when hardware debug mode is enabled. In this mode, reset
requests derived from power-shutoff to the AP-system CPU cores must be
enabled before the AP-system cores first resume from power-shutoff. Else
resume may fail, causing the system to hang during boot. Currently we
avoid the hang by prohibiting booting secondary CPU cores when hardware
debug mode is enabled.

On all R-Car Gen2 SoCs, hardware debug mode is enabled by setting
MD21=1.  On both Koelsch and Lager, this is done by setting mode switch
SW8-4 to OFF.

Unfortunately the hang is not easy to reproduce: I only saw it (on
Koelsch) during real cold boot (power off during the night), and even
then it's not guaranteed to trigger. Pressing the reset button
afterwards recovers the system, and a subsequent boot will succeed
(incl. secondary CPU core boot).

This series configures the reset requests as documented in the R-Car
Gen2 datasheet, and removes the check for MD21 during secondary CPU
bringup.  It was inspired by CPU-specific patches in the BSP by
Nakamura-san.

This series has been boot-tested on r8a7791/koelsch (both debug mode and
normal mode), on r8a7790/lager and r8a7793/gose (normal mode only), and
on r8a7794/alt (normal mode UP only).

Thanks!

Geert Uytterhoeven (4):
  ARM: shmobile: apmu: Add more register documentation
  ARM: shmobile: apmu: Add debug resource reset for secondary CPU boot
  ARM: shmobile: apmu: Allow booting secondary CPU cores in debug mode
  ARM: shmobile: r8a7791: Allow booting secondary CPU cores in debug
    mode

 arch/arm/mach-shmobile/platsmp-apmu.c | 41 ++++++++++++++++++++---------------
 arch/arm/mach-shmobile/smp-r8a7791.c  | 14 +-----------
 2 files changed, 25 insertions(+), 30 deletions(-)

-- 
1.9.1

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

[PATCH 1/4] ARM: shmobile: apmu: Add more register documentation

[Geert Uytterhoeven]

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/platsmp-apmu.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/arm/mach-shmobile/platsmp-apmu.c b/arch/arm/mach-shmobile/platsmp-apmu.c
index 0c6bb458b7a45128..933f9b9024050223 100644
--- a/arch/arm/mach-shmobile/platsmp-apmu.c
+++ b/arch/arm/mach-shmobile/platsmp-apmu.c
@@ -31,9 +31,15 @@ static struct {
 	int bit;
 } apmu_cpus[NR_CPUS];
 
-#define WUPCR_OFFS 0x10
-#define PSTR_OFFS 0x40
-#define CPUNCR_OFFS(n) (0x100 + (0x10 * (n)))
+#define WUPCR_OFFS	 0x10		/* Wake Up Control Register */
+#define PSTR_OFFS	 0x40		/* Power Status Register */
+#define CPUNCR_OFFS(n)	(0x100 + (0x10 * (n)))
+					/* CPUn Power Status Control Register */
+
+/* Power Status Register */
+#define CPUNST(r, n)	(((r) >> (n * 4)) & 3)	/* CPUn Status Bit */
+#define CPUST_RUN	0		/* Run Mode */
+#define CPUST_STANDBY	3		/* CoreStandby Mode */
 
 static int __maybe_unused apmu_power_on(void __iomem *p, int bit)
 {
@@ -59,7 +65,7 @@ static int __maybe_unused apmu_power_off_poll(void __iomem *p, int bit)
 	int k;
 
 	for (k = 0; k < 1000; k++) {
-		if (((readl_relaxed(p + PSTR_OFFS) >> (bit * 4)) & 0x03) == 3)
+		if (CPUNST(readl_relaxed(p + PSTR_OFFS), bit) == CPUST_STANDBY)
 			return 1;
 
 		mdelay(1);
-- 
1.9.1

[PATCH/RFT 2/4] ARM: shmobile: apmu: Add debug resource reset for secondary CPU boot

[Geert Uytterhoeven]

In debug mode (MD21=1), reset requests derived from power-shutoff to the
AP-system CPU cores must be enabled before the AP-system cores first
resume from power-shutoff. Else resume may fail, causing the system to
hang during boot.

As setting these bits is a no-op in normal mode, there's no need to
check the actual state of MD21 first.

Inspired by CPU-specific patches in the BSP by Hisashi Nakamura
<hisashi.nakamura.ak@renesas.com>.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/platsmp-apmu.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/arm/mach-shmobile/platsmp-apmu.c b/arch/arm/mach-shmobile/platsmp-apmu.c
index 933f9b9024050223..7e4ca6788be521dc 100644
--- a/arch/arm/mach-shmobile/platsmp-apmu.c
+++ b/arch/arm/mach-shmobile/platsmp-apmu.c
@@ -35,12 +35,18 @@ static struct {
 #define PSTR_OFFS	 0x40		/* Power Status Register */
 #define CPUNCR_OFFS(n)	(0x100 + (0x10 * (n)))
 					/* CPUn Power Status Control Register */
+#define DBGRCR_OFFS	0x180		/* Debug Resource Reset Control Reg. */
 
 /* Power Status Register */
 #define CPUNST(r, n)	(((r) >> (n * 4)) & 3)	/* CPUn Status Bit */
 #define CPUST_RUN	0		/* Run Mode */
 #define CPUST_STANDBY	3		/* CoreStandby Mode */
 
+/* Debug Resource Reset Control Register */
+#define DBGCPUREN	BIT(24)		/* CPU Other Reset Request Enable */
+#define DBGCPUNREN(n)	BIT((n) + 20)	/* CPUn Reset Request Enable */
+#define DBGCPUPREN	BIT(19)		/* CPU Peripheral Reset Req. Enable */
+
 static int __maybe_unused apmu_power_on(void __iomem *p, int bit)
 {
 	/* request power on */
@@ -84,6 +90,8 @@ static int __maybe_unused apmu_wrap(int cpu, int (*fn)(void __iomem *p, int cpu)
 #ifdef CONFIG_SMP
 static void apmu_init_cpu(struct resource *res, int cpu, int bit)
 {
+	u32 x;
+
 	if ((cpu >= ARRAY_SIZE(apmu_cpus)) || apmu_cpus[cpu].iomem)
 		return;
 
@@ -91,6 +99,11 @@ static void apmu_init_cpu(struct resource *res, int cpu, int bit)
 	apmu_cpus[cpu].bit = bit;
 
 	pr_debug("apmu ioremap %d %d %pr\n", cpu, bit, res);
+
+	/* Setup for debug mode */
+	x = readl(apmu_cpus[cpu].iomem + DBGRCR_OFFS);
+	x |= DBGCPUREN | DBGCPUNREN(bit) | DBGCPUPREN;
+	writel(x, apmu_cpus[cpu].iomem + DBGRCR_OFFS);
 }
 
 static void apmu_parse_cfg(void (*fn)(struct resource *res, int cpu, int bit),
-- 
1.9.1

[PATCH/RFT 3/4] ARM: shmobile: apmu: Allow booting secondary CPU cores in debug mode

[Geert Uytterhoeven]

Now debug resource reset is handled properly, allow booting secondary
CPU cores when hardware debug mode is enabled (MD21=1) on SoCs using the
"renesas,apmu" enable method.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/platsmp-apmu.c | 14 +-------------
 1 file changed, 1 insertion(+), 13 deletions(-)

diff --git a/arch/arm/mach-shmobile/platsmp-apmu.c b/arch/arm/mach-shmobile/platsmp-apmu.c
index 7e4ca6788be521dc..e19266844e16126b 100644
--- a/arch/arm/mach-shmobile/platsmp-apmu.c
+++ b/arch/arm/mach-shmobile/platsmp-apmu.c
@@ -216,21 +216,9 @@ static void __init shmobile_smp_apmu_prepare_cpus_dt(unsigned int max_cpus)
 	rcar_gen2_pm_init();
 }
 
-static int shmobile_smp_apmu_boot_secondary_md21(unsigned int cpu,
-						 struct task_struct *idle)
-{
-	/* Error out when hardware debug mode is enabled */
-	if (rcar_gen2_read_mode_pins() & BIT(21)) {
-		pr_warn("Unable to boot CPU%u when MD21 is set\n", cpu);
-		return -ENOTSUPP;
-	}
-
-	return shmobile_smp_apmu_boot_secondary(cpu, idle);
-}
-
 static struct smp_operations apmu_smp_ops __initdata = {
 	.smp_prepare_cpus	= shmobile_smp_apmu_prepare_cpus_dt,
-	.smp_boot_secondary	= shmobile_smp_apmu_boot_secondary_md21,
+	.smp_boot_secondary	= shmobile_smp_apmu_boot_secondary,
 #ifdef CONFIG_HOTPLUG_CPU
 	.cpu_can_disable	= shmobile_smp_cpu_can_disable,
 	.cpu_die		= shmobile_smp_apmu_cpu_die,
-- 
1.9.1

[PATCH/RFT 4/4] ARM: shmobile: r8a7791: Allow booting secondary CPU cores in debug mode

[Geert Uytterhoeven]

Now debug resource reset is handled properly, allow booting secondary
CPU cores when hardware debug mode is enabled (MD21=1, SW8-4=OFF on
koelsch) on legacy r8a7791.

Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
---
 arch/arm/mach-shmobile/smp-r8a7791.c | 14 +-------------
 1 file changed, 1 insertion(+), 13 deletions(-)

diff --git a/arch/arm/mach-shmobile/smp-r8a7791.c b/arch/arm/mach-shmobile/smp-r8a7791.c
index 2d6417af73b5b722..2948c22cfc53026c 100644
--- a/arch/arm/mach-shmobile/smp-r8a7791.c
+++ b/arch/arm/mach-shmobile/smp-r8a7791.c
@@ -42,21 +42,9 @@ static void __init r8a7791_smp_prepare_cpus(unsigned int max_cpus)
 	rcar_gen2_pm_init();
 }
 
-static int r8a7791_smp_boot_secondary(unsigned int cpu,
-				      struct task_struct *idle)
-{
-	/* Error out when hardware debug mode is enabled */
-	if (rcar_gen2_read_mode_pins() & BIT(21)) {
-		pr_warn("Unable to boot CPU%u when MD21 is set\n", cpu);
-		return -ENOTSUPP;
-	}
-
-	return shmobile_smp_apmu_boot_secondary(cpu, idle);
-}
-
 const struct smp_operations r8a7791_smp_ops __initconst = {
 	.smp_prepare_cpus	= r8a7791_smp_prepare_cpus,
-	.smp_boot_secondary	= r8a7791_smp_boot_secondary,
+	.smp_boot_secondary	= shmobile_smp_apmu_boot_secondary,
 #ifdef CONFIG_HOTPLUG_CPU
 	.cpu_can_disable	= shmobile_smp_cpu_can_disable,
 	.cpu_die		= shmobile_smp_apmu_cpu_die,
-- 
1.9.1

Re: [PATCH/RFT 0/4] ARM: shmobile: R-Car Gen2: Allow booting secondary CPU cores in debug mode

[Geert Uytterhoeven]

Hi Simon, Magnus,

On Mon, Aug 22, 2016 at 4:44 PM, Geert Uytterhoeven
<geert+renesas@glider.be> wrote:
> This patch series is an attempt to allow booting secondary CPU cores on
> R-Car Gen2 when hardware debug mode is enabled. In this mode, reset
> requests derived from power-shutoff to the AP-system CPU cores must be
> enabled before the AP-system cores first resume from power-shutoff. Else
> resume may fail, causing the system to hang during boot. Currently we
> avoid the hang by prohibiting booting secondary CPU cores when hardware
> debug mode is enabled.
>
> On all R-Car Gen2 SoCs, hardware debug mode is enabled by setting
> MD21=1.  On both Koelsch and Lager, this is done by setting mode switch
> SW8-4 to OFF.
>
> Unfortunately the hang is not easy to reproduce: I only saw it (on
> Koelsch) during real cold boot (power off during the night), and even
> then it's not guaranteed to trigger. Pressing the reset button
> afterwards recovers the system, and a subsequent boot will succeed
> (incl. secondary CPU core boot).
>
> This series configures the reset requests as documented in the R-Car
> Gen2 datasheet, and removes the check for MD21 during secondary CPU
> bringup.  It was inspired by CPU-specific patches in the BSP by
> Nakamura-san.
>
> This series has been boot-tested on r8a7791/koelsch (both debug mode and
> normal mode), on r8a7790/lager and r8a7793/gose (normal mode only), and
> on r8a7794/alt (normal mode UP only).

Any comments?
Any objection to applying this series?

I've been running my Koelsch with MD21=1 since I posted this series,
and it has been included in renesas-drivers since the beginning of September.

My main motivation to push this is that it removes two more users of
rcar_gen2_read_mode_pins(). After this, the only remaining user is the
clock driver, invoked from rcar_gen2_timer_init().

Thanks!

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH/RFT 0/4] ARM: shmobile: R-Car Gen2: Allow booting secondary CPU cores in debug mode

[Magnus Damm]

Hi Geert,

On Tue, Sep 27, 2016 at 9:37 PM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> Hi Simon, Magnus,
>
> On Mon, Aug 22, 2016 at 4:44 PM, Geert Uytterhoeven
> <geert+renesas@glider.be> wrote:
>> This patch series is an attempt to allow booting secondary CPU cores on
>> R-Car Gen2 when hardware debug mode is enabled. In this mode, reset
>> requests derived from power-shutoff to the AP-system CPU cores must be
>> enabled before the AP-system cores first resume from power-shutoff. Else
>> resume may fail, causing the system to hang during boot. Currently we
>> avoid the hang by prohibiting booting secondary CPU cores when hardware
>> debug mode is enabled.
>>
>> On all R-Car Gen2 SoCs, hardware debug mode is enabled by setting
>> MD21=1.  On both Koelsch and Lager, this is done by setting mode switch
>> SW8-4 to OFF.
>>
>> Unfortunately the hang is not easy to reproduce: I only saw it (on
>> Koelsch) during real cold boot (power off during the night), and even
>> then it's not guaranteed to trigger. Pressing the reset button
>> afterwards recovers the system, and a subsequent boot will succeed
>> (incl. secondary CPU core boot).
>>
>> This series configures the reset requests as documented in the R-Car
>> Gen2 datasheet, and removes the check for MD21 during secondary CPU
>> bringup.  It was inspired by CPU-specific patches in the BSP by
>> Nakamura-san.
>>
>> This series has been boot-tested on r8a7791/koelsch (both debug mode and
>> normal mode), on r8a7790/lager and r8a7793/gose (normal mode only), and
>> on r8a7794/alt (normal mode UP only).
>
> Any comments?
> Any objection to applying this series?
>
> I've been running my Koelsch with MD21=1 since I posted this series,
> and it has been included in renesas-drivers since the beginning of September.
>
> My main motivation to push this is that it removes two more users of
> rcar_gen2_read_mode_pins(). After this, the only remaining user is the
> clock driver, invoked from rcar_gen2_timer_init().

I have no objections, but I'm curious if the series received enough
testing (with debug mode enabled) on earlier R-Car Gen2 platforms like
r8a7790/lager?

Cheers,

/ magnus

Re: [PATCH/RFT 0/4] ARM: shmobile: R-Car Gen2: Allow booting secondary CPU cores in debug mode

[Geert Uytterhoeven]

Hi Magnus,

On Fri, Sep 30, 2016 at 9:04 AM, Magnus Damm <magnus.damm@gmail.com> wrote:
> On Tue, Sep 27, 2016 at 9:37 PM, Geert Uytterhoeven
> <geert@linux-m68k.org> wrote:
>> On Mon, Aug 22, 2016 at 4:44 PM, Geert Uytterhoeven
>> <geert+renesas@glider.be> wrote:
>>> This patch series is an attempt to allow booting secondary CPU cores on
>>> R-Car Gen2 when hardware debug mode is enabled. In this mode, reset
>>> requests derived from power-shutoff to the AP-system CPU cores must be
>>> enabled before the AP-system cores first resume from power-shutoff. Else
>>> resume may fail, causing the system to hang during boot. Currently we
>>> avoid the hang by prohibiting booting secondary CPU cores when hardware
>>> debug mode is enabled.
>>>
>>> On all R-Car Gen2 SoCs, hardware debug mode is enabled by setting
>>> MD21=1.  On both Koelsch and Lager, this is done by setting mode switch
>>> SW8-4 to OFF.
>>>
>>> Unfortunately the hang is not easy to reproduce: I only saw it (on
>>> Koelsch) during real cold boot (power off during the night), and even
>>> then it's not guaranteed to trigger. Pressing the reset button
>>> afterwards recovers the system, and a subsequent boot will succeed
>>> (incl. secondary CPU core boot).
>>>
>>> This series configures the reset requests as documented in the R-Car
>>> Gen2 datasheet, and removes the check for MD21 during secondary CPU
>>> bringup.  It was inspired by CPU-specific patches in the BSP by
>>> Nakamura-san.
>>>
>>> This series has been boot-tested on r8a7791/koelsch (both debug mode and
>>> normal mode), on r8a7790/lager and r8a7793/gose (normal mode only), and
>>> on r8a7794/alt (normal mode UP only).
>>
>> Any comments?
>> Any objection to applying this series?
>>
>> I've been running my Koelsch with MD21=1 since I posted this series,
>> and it has been included in renesas-drivers since the beginning of September.
>>
>> My main motivation to push this is that it removes two more users of
>> rcar_gen2_read_mode_pins(). After this, the only remaining user is the
>> clock driver, invoked from rcar_gen2_timer_init().
>
> I have no objections, but I'm curious if the series received enough
> testing (with debug mode enabled) on earlier R-Car Gen2 platforms like
> r8a7790/lager?

Let's see what Hiep has to say, who tests Lager with both debug mode
enabled and disabled...

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH/RFT 0/4] ARM: shmobile: R-Car Gen2: Allow booting secondary CPU cores in debug mode

[Magnus Damm]

Hi Geert,

On Fri, Sep 30, 2016 at 4:09 PM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> Hi Magnus,
>
> On Fri, Sep 30, 2016 at 9:04 AM, Magnus Damm <magnus.damm@gmail.com> wrote:
>> On Tue, Sep 27, 2016 at 9:37 PM, Geert Uytterhoeven
>> <geert@linux-m68k.org> wrote:
>>> On Mon, Aug 22, 2016 at 4:44 PM, Geert Uytterhoeven
>>> <geert+renesas@glider.be> wrote:
>>>> This patch series is an attempt to allow booting secondary CPU cores on
>>>> R-Car Gen2 when hardware debug mode is enabled. In this mode, reset
>>>> requests derived from power-shutoff to the AP-system CPU cores must be
>>>> enabled before the AP-system cores first resume from power-shutoff. Else
>>>> resume may fail, causing the system to hang during boot. Currently we
>>>> avoid the hang by prohibiting booting secondary CPU cores when hardware
>>>> debug mode is enabled.
>>>>
>>>> On all R-Car Gen2 SoCs, hardware debug mode is enabled by setting
>>>> MD21=1.  On both Koelsch and Lager, this is done by setting mode switch
>>>> SW8-4 to OFF.
>>>>
>>>> Unfortunately the hang is not easy to reproduce: I only saw it (on
>>>> Koelsch) during real cold boot (power off during the night), and even
>>>> then it's not guaranteed to trigger. Pressing the reset button
>>>> afterwards recovers the system, and a subsequent boot will succeed
>>>> (incl. secondary CPU core boot).
>>>>
>>>> This series configures the reset requests as documented in the R-Car
>>>> Gen2 datasheet, and removes the check for MD21 during secondary CPU
>>>> bringup.  It was inspired by CPU-specific patches in the BSP by
>>>> Nakamura-san.
>>>>
>>>> This series has been boot-tested on r8a7791/koelsch (both debug mode and
>>>> normal mode), on r8a7790/lager and r8a7793/gose (normal mode only), and
>>>> on r8a7794/alt (normal mode UP only).
>>>
>>> Any comments?
>>> Any objection to applying this series?
>>>
>>> I've been running my Koelsch with MD21=1 since I posted this series,
>>> and it has been included in renesas-drivers since the beginning of September.
>>>
>>> My main motivation to push this is that it removes two more users of
>>> rcar_gen2_read_mode_pins(). After this, the only remaining user is the
>>> clock driver, invoked from rcar_gen2_timer_init().
>>
>> I have no objections, but I'm curious if the series received enough
>> testing (with debug mode enabled) on earlier R-Car Gen2 platforms like
>> r8a7790/lager?
>
> Let's see what Hiep has to say, who tests Lager with both debug mode
> enabled and disabled...

Good idea! We probably want to know about ES version too.

Cheers,

/ magnus