From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from moutng.kundenserver.de ([212.227.126.171]) by bombadil.infradead.org with esmtp (Exim 4.68 #1 (Red Hat Linux)) id 1L3DFs-0003cS-BZ for kexec@lists.infradead.org; Thu, 20 Nov 2008 17:23:48 +0000 From: Arnd Bergmann Subject: Re: [PATCH] powerpc/mpic: don't reset affinity for secondary MPIC on boot Date: Thu, 20 Nov 2008 18:23:24 +0100 References: <200811191450.59361.arnd@arndb.de> <1227166240.7185.207.camel@pasglop> In-Reply-To: <1227166240.7185.207.camel@pasglop> MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200811201823.25263.arnd@arndb.de> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: kexec-bounces@lists.infradead.org Errors-To: kexec-bounces+dwmw2=infradead.org@lists.infradead.org To: Benjamin Herrenschmidt Cc: linuxppc-dev@ozlabs.org, paulus@samba.org, kexec@lists.infradead.org, cbe-oss-dev@ozlabs.org, Max Krasnyansky Kexec/kdump currently fails on the IBM QS2x blades when the kexec happens on a CPU other than the initial boot CPU. It turns out that this is the result of mpic_init trying to set affinity of each interrupt vector to the current boot CPU. As far as I can tell, the same problem is likely to exist on any secondary MPIC, because they have to deliver interrupts to the first output all the time. There are two potential solutions for this: either not set up affinity at all for secondary MPICs, or assume that a single CPU output is connected to the upstream interrupt controller and hardcode affinity to that per architecture. This patch implements the second approach, defaulting to the first output. An architecture that has a secondary MPIC connected upstream using a different output needs to set the mpic default_dest to the correct value before calling mpic_init. Signed-off-by: Arnd Bergmann --- include/asm/mpic.h | 3 +++ sysdev/mpic.c | 9 +++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) On Thursday 20 November 2008, Benjamin Herrenschmidt wrote: > I would rather, for non primary, set it to a cpu provided as > either a new argument or an mpic struct member initially set to 1 with > an accessor to change it if necessary. How about this one? > Or should we define a flag to have it read it at init time from the chip ? I don't understand. This is init time, where we write it to the chip, what do you mean with reading from it? If I understand you correctly, we can't trust that value to start with. Arnd <>< diff --git a/arch/powerpc/include/asm/mpic.h b/arch/powerpc/include/asm/mpic.h index fe566a3..543d51c 100644 --- a/arch/powerpc/include/asm/mpic.h +++ b/arch/powerpc/include/asm/mpic.h @@ -295,6 +295,9 @@ struct mpic /* Protected sources */ unsigned long *protected; + /* destination for non-primary MPICs */ + int default_dest; + #ifdef CONFIG_MPIC_WEIRD /* Pointer to HW info array */ u32 *hw_set; diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c index 8e3478c..cf154c9 100644 --- a/arch/powerpc/sysdev/mpic.c +++ b/arch/powerpc/sysdev/mpic.c @@ -1220,6 +1220,7 @@ void __init mpic_set_default_senses(struct mpic *mpic, u8 *senses, int count) void __init mpic_init(struct mpic *mpic) { int i; + int cpu; BUG_ON(mpic->num_sources == 0); @@ -1262,6 +1263,11 @@ void __init mpic_init(struct mpic *mpic) mpic_pasemi_msi_init(mpic); + if (mpic->flags & MPIC_PRIMARY) + cpu = hard_smp_processor_id(); + else + cpu = mpic->default_dest; + for (i = 0; i < mpic->num_sources; i++) { /* start with vector = source number, and masked */ u32 vecpri = MPIC_VECPRI_MASK | i | @@ -1272,8 +1278,7 @@ void __init mpic_init(struct mpic *mpic) continue; /* init hw */ mpic_irq_write(i, MPIC_INFO(IRQ_VECTOR_PRI), vecpri); - mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION), - 1 << hard_smp_processor_id()); + mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION), 1 << cpu); } /* Init spurious vector */ _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: From: Arnd Bergmann To: Benjamin Herrenschmidt Subject: Re: [PATCH] powerpc/mpic: don't reset affinity for secondary MPIC on boot Date: Thu, 20 Nov 2008 18:23:24 +0100 References: <200811191450.59361.arnd@arndb.de> <1227166240.7185.207.camel@pasglop> In-Reply-To: <1227166240.7185.207.camel@pasglop> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Message-Id: <200811201823.25263.arnd@arndb.de> Cc: linuxppc-dev@ozlabs.org, paulus@samba.org, kexec@lists.infradead.org, cbe-oss-dev@ozlabs.org, Max Krasnyansky List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Kexec/kdump currently fails on the IBM QS2x blades when the kexec happens on a CPU other than the initial boot CPU. It turns out that this is the result of mpic_init trying to set affinity of each interrupt vector to the current boot CPU. As far as I can tell, the same problem is likely to exist on any secondary MPIC, because they have to deliver interrupts to the first output all the time. There are two potential solutions for this: either not set up affinity at all for secondary MPICs, or assume that a single CPU output is connected to the upstream interrupt controller and hardcode affinity to that per architecture. This patch implements the second approach, defaulting to the first output. An architecture that has a secondary MPIC connected upstream using a different output needs to set the mpic default_dest to the correct value before calling mpic_init. Signed-off-by: Arnd Bergmann --- include/asm/mpic.h | 3 +++ sysdev/mpic.c | 9 +++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) On Thursday 20 November 2008, Benjamin Herrenschmidt wrote: > I would rather, for non primary, set it to a cpu provided as > either a new argument or an mpic struct member initially set to 1 with > an accessor to change it if necessary. How about this one? > Or should we define a flag to have it read it at init time from the chip ? I don't understand. This is init time, where we write it to the chip, what do you mean with reading from it? If I understand you correctly, we can't trust that value to start with. Arnd <>< diff --git a/arch/powerpc/include/asm/mpic.h b/arch/powerpc/include/asm/mpic.h index fe566a3..543d51c 100644 --- a/arch/powerpc/include/asm/mpic.h +++ b/arch/powerpc/include/asm/mpic.h @@ -295,6 +295,9 @@ struct mpic /* Protected sources */ unsigned long *protected; + /* destination for non-primary MPICs */ + int default_dest; + #ifdef CONFIG_MPIC_WEIRD /* Pointer to HW info array */ u32 *hw_set; diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c index 8e3478c..cf154c9 100644 --- a/arch/powerpc/sysdev/mpic.c +++ b/arch/powerpc/sysdev/mpic.c @@ -1220,6 +1220,7 @@ void __init mpic_set_default_senses(struct mpic *mpic, u8 *senses, int count) void __init mpic_init(struct mpic *mpic) { int i; + int cpu; BUG_ON(mpic->num_sources == 0); @@ -1262,6 +1263,11 @@ void __init mpic_init(struct mpic *mpic) mpic_pasemi_msi_init(mpic); + if (mpic->flags & MPIC_PRIMARY) + cpu = hard_smp_processor_id(); + else + cpu = mpic->default_dest; + for (i = 0; i < mpic->num_sources; i++) { /* start with vector = source number, and masked */ u32 vecpri = MPIC_VECPRI_MASK | i | @@ -1272,8 +1278,7 @@ void __init mpic_init(struct mpic *mpic) continue; /* init hw */ mpic_irq_write(i, MPIC_INFO(IRQ_VECTOR_PRI), vecpri); - mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION), - 1 << hard_smp_processor_id()); + mpic_irq_write(i, MPIC_INFO(IRQ_DESTINATION), 1 << cpu); } /* Init spurious vector */