Re: panic kexec broken on ARM64?

[Marc Zyngier <marc.zyngier@arm.com>]

On Wed, 06 Jun 2018 12:37:02 +0100,
James Morse wrote:
> 
> Hi Stefan,
> 
> On 06/06/18 08:02, Stefan Wahren wrote:
> > Am 05.06.2018 um 19:46 schrieb James Morse:
> >> On 05/06/18 09:01, Petr Tesarik wrote:
> >>> I attached a hardware debugger and found
> >>> out that all CPU cores were stopped except one which was stuck in the
> >>> idle thread. It seems that irq_set_irqchip_state() may sleep, which is
> >>> definitely not safe after a kernel panic.
> 
> >> I don't know much about irqchip stuff, but __irq_get_desc_lock() takes a
> >> raw_spin_lock(), and calls gic_irq_get_irqchip_state() which is just poking
> >> around in mmio registers, this should all be safe unless you re-entered the same
> >> code.
> 
> >>> If I'm right, then this is broken in general, but I have only ever seen
> >>> it on RPi 3 Model B+ (even RPi3 Model B works fine), so the issue may
> >>> be more subtle.
> 
> >> Is there a hardware difference around the interrupt controller on these?
> 
> > No, but the RPi 3 B has a different USB network chip on board (smsc95xx, Fast
> > ethernet) instead of lan78xx (Gigabit ethernet).
> 
> Bingo: its the lan78xx driver that is sleeping from the irqchip
> callbacks; The smsc95xx driver doesn't have a struct irq_chip, which
> is why the RPi-3-B doesn't do this.
> 
> It may be valid for kdump to only teardown the 'root irqdomain' (if
> that even means anything). I assume these secondary irqchip's would
> have a summary-interrupt that goes to another irqchip. But I can't
> see a way to tell them apart..,

There is none. A cascaded irqchip is just like a root irqchip, just
that its output line is connected to another irqchip. But we have no
easy way to identify the parent. Also, this particular driver looks
quite creative (it reinvents the wheel for chained interrupts -- see
intr_complete and lan78xx_status), meaning that even if we could have
a magic way of identify a chained irqchip, we'd miss that one. Broken.

> I think we need to wait until after the merge window for Marc's
> wisdom on this!

Overall, I can't think of an easy fix. We have a few options, but none
of them involve a centralised change:

1) We provide a reset infrastructure for irqchips, with an opt-in
   mechanism. This involves changing the way we teardown irqs at
   crash-time, and we'd then need some notion of reset ordering (think
   of the layered ITS and GICv3, for example).

2) We provide a way to identify interrupts that are ultimately backed
   by a root controller, which implies walking down the hierarchy for
   each one of them. Fairly expensive, but minimal in way of changes
   in the crash code. Requires a per-irqchip flag, but ordering comes
   in for free.

3) We do the same as (2), but at the irqdomain level. Not sure that's
   any better, and it may be even more complicated and bring back some
   ordering issues.

I'm currently angling for (2), with (1) as a final hammer option once
we have nuked all the individual interrupts (useful for the GICv3
redistributor case).

Thoughts?

	M.

-- 
Jazz is not dead, it just smell funny.

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec