5.7-rc interrupt_return Unrecoverable exception 380

5.7-rc interrupt_return Unrecoverable exception 380

[Hugh Dickins]

Hi Nick,

I've been getting an "Unrecoverable exception 380" after a few hours
of load on the G5 (yes, that G5!) with 5.7-rc: when interrupt_return
checks lazy_irq_pending, it crashes at check_preemption_disabled+0x24
with CONFIG_DEBUG_PREEMPT=y.

check_preemption_disabled():
lib/smp_processor_id.c:13
   0:	7c 08 02 a6 	mflr    r0
   4:	fb e1 ff f8 	std     r31,-8(r1)
   8:	fb 61 ff d8 	std     r27,-40(r1)
   c:	fb 81 ff e0 	std     r28,-32(r1)
  10:	fb a1 ff e8 	std     r29,-24(r1)
  14:	fb c1 ff f0 	std     r30,-16(r1)
get_current():
arch/powerpc/include/asm/current.h:20
  18:	eb ed 01 88 	ld      r31,392(r13)
check_preemption_disabled():
lib/smp_processor_id.c:13
  1c:	f8 01 00 10 	std     r0,16(r1)
  20:	f8 21 ff 61 	stdu    r1,-160(r1)
__read_once_size():
include/linux/compiler.h:199
  24:	81 3f 00 00 	lwz     r9,0(r31)
check_preemption_disabled():
lib/smp_processor_id.c:14
  28:	a3 cd 00 02 	lhz     r30,2(r13)

I don't read ppc assembly, and have not jotted down the registers,
but hope you can make sense of it. I get around it with the patch
below (just avoiding the debug), but have no idea whether it's a
necessary fix or a hacky workaround.

Hugh

--- 5.7-rc3/arch/powerpc/include/asm/hw_irq.h	2020-04-12 16:24:29.802769727 -0700
+++ linux/arch/powerpc/include/asm/hw_irq.h	2020-04-27 11:31:10.000000000 -0700
@@ -252,7 +252,7 @@ static inline bool arch_irqs_disabled(vo
 
 static inline bool lazy_irq_pending(void)
 {
-	return !!(get_paca()->irq_happened & ~PACA_IRQ_HARD_DIS);
+	return !!(local_paca->irq_happened & ~PACA_IRQ_HARD_DIS);
 }
 
 /*

Re: 5.7-rc interrupt_return Unrecoverable exception 380

[Nicholas Piggin]

Excerpts from Hugh Dickins's message of May 2, 2020 6:38 am:
> Hi Nick,
> 
> I've been getting an "Unrecoverable exception 380" after a few hours
> of load on the G5 (yes, that G5!) with 5.7-rc: when interrupt_return
> checks lazy_irq_pending, it crashes at check_preemption_disabled+0x24
> with CONFIG_DEBUG_PREEMPT=y.
> 
> check_preemption_disabled():
> lib/smp_processor_id.c:13
>    0:	7c 08 02 a6 	mflr    r0
>    4:	fb e1 ff f8 	std     r31,-8(r1)
>    8:	fb 61 ff d8 	std     r27,-40(r1)
>    c:	fb 81 ff e0 	std     r28,-32(r1)
>   10:	fb a1 ff e8 	std     r29,-24(r1)
>   14:	fb c1 ff f0 	std     r30,-16(r1)
> get_current():
> arch/powerpc/include/asm/current.h:20
>   18:	eb ed 01 88 	ld      r31,392(r13)
> check_preemption_disabled():
> lib/smp_processor_id.c:13
>   1c:	f8 01 00 10 	std     r0,16(r1)
>   20:	f8 21 ff 61 	stdu    r1,-160(r1)
> __read_once_size():
> include/linux/compiler.h:199
>   24:	81 3f 00 00 	lwz     r9,0(r31)
> check_preemption_disabled():
> lib/smp_processor_id.c:14
>   28:	a3 cd 00 02 	lhz     r30,2(r13)
> 
> I don't read ppc assembly, and have not jotted down the registers,
> but hope you can make sense of it. I get around it with the patch
> below (just avoiding the debug), but have no idea whether it's a
> necessary fix or a hacky workaround.

Hi Hugh,

Thanks for the report, nice catch. Your fix is actually the correct one 
(well, we probably want a __lazy_irq_pending() variant which is to be 
used in these cases).

Problem is MSR[RI] is cleared here, ready to do the last few things for 
interrupt return where we're not allowed to take any other interrupts.

SLB interrupts can happen just about anywhere aside from kernel text, 
global variables, and stack. When that hits, it appears to be 
unrecoverable due to RI=0.

We could clear just MSR[EE] for asynchronous interrupts, then check 
lazy_irq_pending(), and then clear MSR[RI] ready to return, and the
SLB miss in the debug check would be fine. But that's two mtmsr 
instructions, which is slower. So we'll skip the check.

I tested hash, and preempt, possibly even preempt+hash, but clearly not 
preempt+preempt_debug+hash+slb thrashing!

Thanks,
Nick

> 
> Hugh
> 
> --- 5.7-rc3/arch/powerpc/include/asm/hw_irq.h	2020-04-12 16:24:29.802769727 -0700
> +++ linux/arch/powerpc/include/asm/hw_irq.h	2020-04-27 11:31:10.000000000 -0700
> @@ -252,7 +252,7 @@ static inline bool arch_irqs_disabled(vo
>  
>  static inline bool lazy_irq_pending(void)
>  {
> -	return !!(get_paca()->irq_happened & ~PACA_IRQ_HARD_DIS);
> +	return !!(local_paca->irq_happened & ~PACA_IRQ_HARD_DIS);
>  }
>  
>  /*
>