* [PATCH RFC] Debug handling of early spurious interrupts
@ 2007-07-17 10:09 Fernando Luis Vázquez Cao
2007-07-18 22:46 ` Andrew Morton
0 siblings, 1 reply; 13+ messages in thread
From: Fernando Luis Vázquez Cao @ 2007-07-17 10:09 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, kexec
With the advent of kdump it is possible that device drivers receive
interrupts generated in the context of a previous kernel. Ideally
quiescing the underlying devices should suffice but not all drivers
do this, either because it is not possible or because they did not
contemplate this case. Thus drivers ought to be able to handle
interrupts coming in as soon as the interrupt handler is registered.
Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp>
---
diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22/kernel/irq/manage.c
--- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22/kernel/irq/manage.c 2007-07-17 18:37:24.000000000 +0900
@@ -537,6 +537,29 @@ int request_irq(unsigned int irq, irq_ha
select_smp_affinity(irq);
+#if defined(CONFIG_DEBUG_PENDING_IRQ) || defined(CONFIG_DEBUG_SHIRQ)
+#ifndef CONFIG_DEBUG_PENDING_IRQ
+ if (irqflags & IRQF_SHARED) {
+ /*
+ * It's a shared IRQ -- the driver ought to be prepared for it
+ * to happen immediately, so let's make sure....
+ * We do this before actually registering it, to make sure that
+ * a 'real' IRQ doesn't run in parallel with our fake.
+ */
+#endif /* !CONFIG_DEBUG_PENDING_IRQ */
+ if (irqflags & IRQF_DISABLED) {
+ unsigned long flags;
+
+ local_irq_save(flags);
+ handler(irq, dev_id);
+ local_irq_restore(flags);
+ } else
+ handler(irq, dev_id);
+#ifndef CONFIG_DEBUG_PENDING_IRQ
+ }
+#endif /* !CONFIG_DEBUG_PENDING_IRQ */
+#endif /* CONFIG_DEBUG_PENDING_IRQ || CONFIG_DEBUG_SHIRQ */
+
#ifdef CONFIG_DEBUG_SHIRQ
if (irqflags & IRQF_SHARED) {
/*
diff -urNp linux-2.6.22-orig/lib/Kconfig.debug linux-2.6.22/lib/Kconfig.debug
--- linux-2.6.22-orig/lib/Kconfig.debug 2007-07-09 08:32:17.000000000 +0900
+++ linux-2.6.22/lib/Kconfig.debug 2007-07-17 18:52:00.000000000 +0900
@@ -77,6 +77,17 @@ config DEBUG_KERNEL
Say Y here if you are developing drivers or trying to debug and
identify kernel problems.
+config DEBUG_PENDING_IRQ
+ bool "Debug handling of early spurious interrupts"
+ depends on DEBUG_KERNEL && GENERIC_HARDIRQS
+ help
+ With the advent of kdump it is possible that device drivers receive
+ interrupts generated in the context of a previous kernel. Ideally
+ quiescing the underlying devices should suffice but not all drivers
+ do this, either because it is not possible or because they did not
+ contemplate this case. Thus drivers ought to be able to handle
+ interrupts coming in as soon as the interrupt handler is registered.
+
config DEBUG_SHIRQ
bool "Debug shared IRQ handlers"
depends on DEBUG_KERNEL && GENERIC_HARDIRQS
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH RFC] Debug handling of early spurious interrupts 2007-07-17 10:09 [PATCH RFC] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao @ 2007-07-18 22:46 ` Andrew Morton 2007-07-20 1:54 ` Fernando Luis Vázquez Cao 0 siblings, 1 reply; 13+ messages in thread From: Andrew Morton @ 2007-07-18 22:46 UTC (permalink / raw) To: Fernando Luis Vázquez Cao; +Cc: linux-kernel, kexec On Tue, 17 Jul 2007 19:09:57 +0900 Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote: > With the advent of kdump it is possible that device drivers receive > interrupts generated in the context of a previous kernel. Ideally > quiescing the underlying devices should suffice but not all drivers > do this, either because it is not possible or because they did not > contemplate this case. Thus drivers ought to be able to handle > interrupts coming in as soon as the interrupt handler is registered. > > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> > --- > > diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22/kernel/irq/manage.c > --- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-09 08:32:17.000000000 +0900 > +++ linux-2.6.22/kernel/irq/manage.c 2007-07-17 18:37:24.000000000 +0900 > @@ -537,6 +537,29 @@ int request_irq(unsigned int irq, irq_ha > > select_smp_affinity(irq); > > +#if defined(CONFIG_DEBUG_PENDING_IRQ) || defined(CONFIG_DEBUG_SHIRQ) > +#ifndef CONFIG_DEBUG_PENDING_IRQ > + if (irqflags & IRQF_SHARED) { > + /* > + * It's a shared IRQ -- the driver ought to be prepared for it > + * to happen immediately, so let's make sure.... > + * We do this before actually registering it, to make sure that > + * a 'real' IRQ doesn't run in parallel with our fake. > + */ > +#endif /* !CONFIG_DEBUG_PENDING_IRQ */ > + if (irqflags & IRQF_DISABLED) { > + unsigned long flags; > + > + local_irq_save(flags); > + handler(irq, dev_id); > + local_irq_restore(flags); > + } else > + handler(irq, dev_id); > +#ifndef CONFIG_DEBUG_PENDING_IRQ > + } > +#endif /* !CONFIG_DEBUG_PENDING_IRQ */ > +#endif /* CONFIG_DEBUG_PENDING_IRQ || CONFIG_DEBUG_SHIRQ */ Even if we were going to merge this functionality as-is, I'd ask for some sort of refactoring to fix up that ifdef maze. But more substantial issues: - This is presented as a "debug" feature, but it isn't a debug feature at all - it is new functionality which is unrelated to kernel development. Also, it is a "debug" feature which provides no debugging! At the very least, one would expect to see it emit a printk to tell people that we have some driver which needs fixing. Also, this not-really-a-debug-feature is undesirably coupled with a real debugging feature: CONFIG_DEBUG_PENDING_IRQ. - Does this new feature really need its own Kconfig setting? Why not enable it unconditionally? request_irq() isn't exactly performance-critical. - If poss, we really do want to find some way of emitting a warning when we detect such a device driver. Like, call the handler and if it returned IRQ_HANDLED, start shouting. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH RFC] Debug handling of early spurious interrupts 2007-07-18 22:46 ` Andrew Morton @ 2007-07-20 1:54 ` Fernando Luis Vázquez Cao 2007-07-20 2:02 ` [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ Fernando Luis Vázquez Cao 0 siblings, 1 reply; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-20 1:54 UTC (permalink / raw) To: Andrew Morton; +Cc: linux-kernel, kexec On Wed, 2007-07-18 at 15:46 -0700, Andrew Morton wrote: > On Tue, 17 Jul 2007 19:09:57 +0900 > Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote: > > > With the advent of kdump it is possible that device drivers receive > > interrupts generated in the context of a previous kernel. Ideally > > quiescing the underlying devices should suffice but not all drivers > > do this, either because it is not possible or because they did not > > contemplate this case. Thus drivers ought to be able to handle > > interrupts coming in as soon as the interrupt handler is registered. > > > > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> > > --- > > > > diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22/kernel/irq/manage.c > > --- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-09 08:32:17.000000000 +0900 > > +++ linux-2.6.22/kernel/irq/manage.c 2007-07-17 18:37:24.000000000 +0900 > > @@ -537,6 +537,29 @@ int request_irq(unsigned int irq, irq_ha > > > > select_smp_affinity(irq); > > > > +#if defined(CONFIG_DEBUG_PENDING_IRQ) || defined(CONFIG_DEBUG_SHIRQ) > > +#ifndef CONFIG_DEBUG_PENDING_IRQ > > + if (irqflags & IRQF_SHARED) { > > + /* > > + * It's a shared IRQ -- the driver ought to be prepared for it > > + * to happen immediately, so let's make sure.... > > + * We do this before actually registering it, to make sure that > > + * a 'real' IRQ doesn't run in parallel with our fake. > > + */ > > +#endif /* !CONFIG_DEBUG_PENDING_IRQ */ > > + if (irqflags & IRQF_DISABLED) { > > + unsigned long flags; > > + > > + local_irq_save(flags); > > + handler(irq, dev_id); > > + local_irq_restore(flags); > > + } else > > + handler(irq, dev_id); > > +#ifndef CONFIG_DEBUG_PENDING_IRQ > > + } > > +#endif /* !CONFIG_DEBUG_PENDING_IRQ */ > > +#endif /* CONFIG_DEBUG_PENDING_IRQ || CONFIG_DEBUG_SHIRQ */ > > Even if we were going to merge this functionality as-is, I'd ask for some > sort of refactoring to fix up that ifdef maze. I a absolutely agree. My first impulse was to get rid of all the cpp kludge including the Kconfig setting CONFIG_DEBUG_SHIRQ, since, as you pointed out, request_irq() is not really performance critical. Unfortunately for the RFC I decided to be conservative and ended up with an "ifdef maze". Thank you for the heads-up. > But more substantial issues: > > - This is presented as a "debug" feature, but it isn't a debug feature at > all - it is new functionality which is unrelated to kernel development. Yup. > Also, it is a "debug" feature which provides no debugging! At the very > least, one would expect to see it emit a printk to tell people that we > have some driver which needs fixing. I am afraid that in some occasions the kernel may panic inside the interrupt handler, but I agree that we need to print a meaningful message for the general case (i.e. something goes wrong but we can recover). I will do that. > Also, this not-really-a-debug-feature is undesirably coupled with a > real debugging feature: CONFIG_DEBUG_PENDING_IRQ. In the new version of the patch I will remove both CONFIG_DEBUG_PENDING_IRQ and CONFIG_DEBUG_SHIRQ. request_irq() and setup_irq() are not fast paths and free_irq() much less so. > - Does this new feature really need its own Kconfig setting? Why not enable > it unconditionally? request_irq() isn't exactly performance-critical. I guess the Kconfig setting is not needed. In fact, by enabling this feature unconditionally we would have _everyone_ (unknowing) testing an area which is a major pain point for kdump. I am not sure this is an acceptable default for all systems though. Opinions welcome. > - If poss, we really do want to find some way of emitting a warning when > we detect such a device driver. Like, call the handler and if it > returned IRQ_HANDLED, start shouting. I will do that and submit updated patches. Thank you for your feedback, Andrew! ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ 2007-07-20 1:54 ` Fernando Luis Vázquez Cao @ 2007-07-20 2:02 ` Fernando Luis Vázquez Cao 2007-07-20 2:20 ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao 0 siblings, 1 reply; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-20 2:02 UTC (permalink / raw) To: Andrew Morton; +Cc: kexec, linux-kernel request_irq() and setup_irq() are not fast paths and free_irq() much less so. In fact, by enabling this feature unconditionally we would have _everyone_ (unknowingly) testing devices drivers, which hopefully will result in more bug-reports and, in turn, better drivers. Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> --- diff -urNp linux-2.6.22-orig/arch/arm/configs/at91sam9rlek_defconfig linux-2.6.22-pendirq/arch/arm/configs/at91sam9rlek_defconfig --- linux-2.6.22-orig/arch/arm/configs/at91sam9rlek_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/arm/configs/at91sam9rlek_defconfig 2007-07-19 16:49:54.000000000 +0900 @@ -906,7 +906,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/arm/configs/picotux200_defconfig linux-2.6.22-pendirq/arch/arm/configs/picotux200_defconfig --- linux-2.6.22-orig/arch/arm/configs/picotux200_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/arm/configs/picotux200_defconfig 2007-07-19 16:50:00.000000000 +0900 @@ -1293,7 +1293,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/arm/configs/s3c2410_defconfig linux-2.6.22-pendirq/arch/arm/configs/s3c2410_defconfig --- linux-2.6.22-orig/arch/arm/configs/s3c2410_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/arm/configs/s3c2410_defconfig 2007-07-19 16:50:06.000000000 +0900 @@ -1368,7 +1368,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=16 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/avr32/configs/atngw100_defconfig linux-2.6.22-pendirq/arch/avr32/configs/atngw100_defconfig --- linux-2.6.22-orig/arch/avr32/configs/atngw100_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/avr32/configs/atngw100_defconfig 2007-07-19 16:50:11.000000000 +0900 @@ -925,7 +925,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/avr32/configs/atstk1002_defconfig linux-2.6.22-pendirq/arch/avr32/configs/atstk1002_defconfig --- linux-2.6.22-orig/arch/avr32/configs/atstk1002_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/avr32/configs/atstk1002_defconfig 2007-07-19 16:50:16.000000000 +0900 @@ -756,7 +756,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/i386/defconfig linux-2.6.22-pendirq/arch/i386/defconfig --- linux-2.6.22-orig/arch/i386/defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/i386/defconfig 2007-07-19 16:50:21.000000000 +0900 @@ -1424,7 +1424,6 @@ CONFIG_UNUSED_SYMBOLS=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/ia64/configs/tiger_defconfig linux-2.6.22-pendirq/arch/ia64/configs/tiger_defconfig --- linux-2.6.22-orig/arch/ia64/configs/tiger_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/ia64/configs/tiger_defconfig 2007-07-19 16:50:26.000000000 +0900 @@ -1318,7 +1318,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=20 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/ia64/configs/zx1_defconfig linux-2.6.22-pendirq/arch/ia64/configs/zx1_defconfig --- linux-2.6.22-orig/arch/ia64/configs/zx1_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/ia64/configs/zx1_defconfig 2007-07-19 16:50:31.000000000 +0900 @@ -1533,7 +1533,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=17 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/ia64/defconfig linux-2.6.22-pendirq/arch/ia64/defconfig --- linux-2.6.22-orig/arch/ia64/defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/ia64/defconfig 2007-07-19 16:50:36.000000000 +0900 @@ -1482,7 +1482,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=20 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/mips/configs/bigsur_defconfig linux-2.6.22-pendirq/arch/mips/configs/bigsur_defconfig --- linux-2.6.22-orig/arch/mips/configs/bigsur_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/mips/configs/bigsur_defconfig 2007-07-19 16:50:42.000000000 +0900 @@ -1003,7 +1003,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=16 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/mips/configs/mipssim_defconfig linux-2.6.22-pendirq/arch/mips/configs/mipssim_defconfig --- linux-2.6.22-orig/arch/mips/configs/mipssim_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/mips/configs/mipssim_defconfig 2007-07-19 16:50:54.000000000 +0900 @@ -855,7 +855,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_UNUSED_SYMBOLS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 # CONFIG_DETECT_SOFTLOCKUP is not set # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/mips/configs/pnx8550-jbs_defconfig linux-2.6.22-pendirq/arch/mips/configs/pnx8550-jbs_defconfig --- linux-2.6.22-orig/arch/mips/configs/pnx8550-jbs_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/mips/configs/pnx8550-jbs_defconfig 2007-07-19 16:51:08.000000000 +0900 @@ -1205,7 +1205,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/mips/configs/pnx8550-stb810_defconfig linux-2.6.22-pendirq/arch/mips/configs/pnx8550-stb810_defconfig --- linux-2.6.22-orig/arch/mips/configs/pnx8550-stb810_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/mips/configs/pnx8550-stb810_defconfig 2007-07-19 16:51:33.000000000 +0900 @@ -1195,7 +1195,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set CONFIG_HEADERS_CHECK=y CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/mips/configs/yosemite_defconfig linux-2.6.22-pendirq/arch/mips/configs/yosemite_defconfig --- linux-2.6.22-orig/arch/mips/configs/yosemite_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/mips/configs/yosemite_defconfig 2007-07-19 16:51:38.000000000 +0900 @@ -816,7 +816,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/cell_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/cell_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/cell_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/cell_defconfig 2007-07-19 16:51:59.000000000 +0900 @@ -1468,7 +1468,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set # CONFIG_DETECT_SOFTLOCKUP is not set # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/celleb_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/celleb_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/celleb_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/celleb_defconfig 2007-07-19 16:52:06.000000000 +0900 @@ -1295,7 +1295,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/chrp32_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/chrp32_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/chrp32_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/chrp32_defconfig 2007-07-19 16:52:12.000000000 +0900 @@ -1454,7 +1454,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/ebony_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/ebony_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/ebony_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/ebony_defconfig 2007-07-19 16:52:16.000000000 +0900 @@ -772,7 +772,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/g5_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/g5_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/g5_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/g5_defconfig 2007-07-19 16:52:22.000000000 +0900 @@ -1645,7 +1645,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/holly_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/holly_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/holly_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/holly_defconfig 2007-07-19 16:52:26.000000000 +0900 @@ -985,7 +985,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/iseries_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/iseries_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/iseries_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/iseries_defconfig 2007-07-19 16:52:30.000000000 +0900 @@ -1147,7 +1147,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/linkstation_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/linkstation_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/linkstation_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/linkstation_defconfig 2007-07-19 16:52:34.000000000 +0900 @@ -1530,7 +1530,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/lite5200_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/lite5200_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/lite5200_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/lite5200_defconfig 2007-07-19 16:52:38.000000000 +0900 @@ -893,7 +893,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/maple_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/maple_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/maple_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/maple_defconfig 2007-07-19 16:52:43.000000000 +0900 @@ -1169,7 +1169,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8272_ads_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8272_ads_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8272_ads_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8272_ads_defconfig 2007-07-19 16:52:47.000000000 +0900 @@ -844,7 +844,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8313_rdb_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8313_rdb_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8313_rdb_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8313_rdb_defconfig 2007-07-19 16:52:51.000000000 +0900 @@ -1336,7 +1336,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8540_ads_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8540_ads_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8540_ads_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8540_ads_defconfig 2007-07-19 16:52:56.000000000 +0900 @@ -772,7 +772,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8544_ds_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8544_ds_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8544_ds_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8544_ds_defconfig 2007-07-19 16:53:00.000000000 +0900 @@ -963,7 +963,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8560_ads_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8560_ads_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8560_ads_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8560_ads_defconfig 2007-07-19 16:53:04.000000000 +0900 @@ -852,7 +852,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8568mds_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8568mds_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8568mds_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8568mds_defconfig 2007-07-19 16:53:08.000000000 +0900 @@ -890,7 +890,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc85xx_cds_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc85xx_cds_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc85xx_cds_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc85xx_cds_defconfig 2007-07-19 16:53:11.000000000 +0900 @@ -905,7 +905,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/mpc8641_hpcn_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/mpc8641_hpcn_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/mpc8641_hpcn_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/mpc8641_hpcn_defconfig 2007-07-19 16:53:15.000000000 +0900 @@ -962,7 +962,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/pasemi_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/pasemi_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/pasemi_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/pasemi_defconfig 2007-07-19 16:53:27.000000000 +0900 @@ -1646,7 +1646,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/pmac32_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/pmac32_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/pmac32_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/pmac32_defconfig 2007-07-19 16:53:30.000000000 +0900 @@ -1885,7 +1885,6 @@ CONFIG_MAGIC_SYSRQ=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/ppc64_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/ppc64_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/ppc64_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/ppc64_defconfig 2007-07-19 16:53:34.000000000 +0900 @@ -1801,7 +1801,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/ps3_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/ps3_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/ps3_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/ps3_defconfig 2007-07-19 16:53:40.000000000 +0900 @@ -1091,7 +1091,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/powerpc/configs/pseries_defconfig linux-2.6.22-pendirq/arch/powerpc/configs/pseries_defconfig --- linux-2.6.22-orig/arch/powerpc/configs/pseries_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/powerpc/configs/pseries_defconfig 2007-07-19 16:54:16.000000000 +0900 @@ -1580,7 +1580,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/sh/configs/r7780rp_defconfig linux-2.6.22-pendirq/arch/sh/configs/r7780rp_defconfig --- linux-2.6.22-orig/arch/sh/configs/r7780rp_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/sh/configs/r7780rp_defconfig 2007-07-19 16:54:27.000000000 +0900 @@ -1235,7 +1235,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/sh/configs/r7785rp_defconfig linux-2.6.22-pendirq/arch/sh/configs/r7785rp_defconfig --- linux-2.6.22-orig/arch/sh/configs/r7785rp_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/sh/configs/r7785rp_defconfig 2007-07-19 16:54:31.000000000 +0900 @@ -1238,7 +1238,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 # CONFIG_DETECT_SOFTLOCKUP is not set # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/sh/configs/se7712_defconfig linux-2.6.22-pendirq/arch/sh/configs/se7712_defconfig --- linux-2.6.22-orig/arch/sh/configs/se7712_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/sh/configs/se7712_defconfig 2007-07-19 16:54:34.000000000 +0900 @@ -993,7 +993,6 @@ CONFIG_ENABLE_MUST_CHECK=y # CONFIG_DEBUG_FS is not set # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_LOG_BUF_SHIFT=14 # CONFIG_DETECT_SOFTLOCKUP is not set # CONFIG_SCHEDSTATS is not set diff -urNp linux-2.6.22-orig/arch/sh64/configs/cayman_defconfig linux-2.6.22-pendirq/arch/sh64/configs/cayman_defconfig --- linux-2.6.22-orig/arch/sh64/configs/cayman_defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/sh64/configs/cayman_defconfig 2007-07-19 16:54:22.000000000 +0900 @@ -999,7 +999,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y CONFIG_SCHEDSTATS=y # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/sparc64/defconfig linux-2.6.22-pendirq/arch/sparc64/defconfig --- linux-2.6.22-orig/arch/sparc64/defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/sparc64/defconfig 2007-07-19 16:54:38.000000000 +0900 @@ -1463,7 +1463,6 @@ CONFIG_MAGIC_SYSRQ=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y CONFIG_SCHEDSTATS=y # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/arch/x86_64/defconfig linux-2.6.22-pendirq/arch/x86_64/defconfig --- linux-2.6.22-orig/arch/x86_64/defconfig 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/arch/x86_64/defconfig 2007-07-19 16:54:43.000000000 +0900 @@ -1522,7 +1522,6 @@ CONFIG_UNUSED_SYMBOLS=y CONFIG_DEBUG_FS=y # CONFIG_HEADERS_CHECK is not set CONFIG_DEBUG_KERNEL=y -# CONFIG_DEBUG_SHIRQ is not set CONFIG_DETECT_SOFTLOCKUP=y # CONFIG_SCHEDSTATS is not set # CONFIG_TIMER_STATS is not set diff -urNp linux-2.6.22-orig/drivers/char/cyclades.c linux-2.6.22-pendirq/drivers/char/cyclades.c --- linux-2.6.22-orig/drivers/char/cyclades.c 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/drivers/char/cyclades.c 2007-07-19 17:11:48.000000000 +0900 @@ -1408,7 +1408,7 @@ static irqreturn_t cyy_interrupt(int irq card_base_addr = cinfo->base_addr; index = cinfo->bus_index; - /* card was not initialized yet (e.g. DEBUG_SHIRQ) */ + /* card was not initialized yet (e.g. shared IRQ) */ if (unlikely(card_base_addr == NULL)) return IRQ_HANDLED; diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c --- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c 2007-07-19 17:23:55.000000000 +0900 @@ -1424,10 +1424,11 @@ e1000_open(struct net_device *netdev) e1000_check_mng_mode(&adapter->hw)) e1000_get_hw_control(adapter); - /* before we allocate an interrupt, we must be ready to handle it. - * Setting DEBUG_SHIRQ in the kernel makes it fire an interrupt - * as soon as we call pci_request_irq, so we have to setup our - * clean_rx handler before we do so. */ + /* Before we allocate an interrupt, we must be ready to handle it. + * Any pending interrupts (coming from a device sharing our IRQ line or + * generated in the context of a previous kernel if this is a kdump + * kernel, for example) will come in as soon as we call pci_request_irq, + * so we have to setup our clean_rx handler before we do so. */ e1000_configure(adapter); err = e1000_request_irq(adapter); diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c --- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/kernel/irq/manage.c 2007-07-19 17:04:50.000000000 +0900 @@ -374,14 +374,12 @@ int setup_irq(unsigned int irq, struct i return 0; mismatch: -#ifdef CONFIG_DEBUG_SHIRQ if (!(new->flags & IRQF_PROBE_SHARED)) { printk(KERN_ERR "IRQ handler type mismatch for IRQ %d\n", irq); if (old_name) printk(KERN_ERR "current handler: %s\n", old_name); dump_stack(); } -#endif spin_unlock_irqrestore(&desc->lock, flags); return -EBUSY; } @@ -454,7 +452,6 @@ void free_irq(unsigned int irq, void *de spin_unlock_irqrestore(&desc->lock, flags); return; } -#ifdef CONFIG_DEBUG_SHIRQ if (handler) { /* * It's a shared IRQ -- the driver ought to be prepared for it @@ -464,7 +461,6 @@ void free_irq(unsigned int irq, void *de */ handler(irq, dev_id); } -#endif } EXPORT_SYMBOL(free_irq); @@ -537,7 +533,6 @@ int request_irq(unsigned int irq, irq_ha select_smp_affinity(irq); -#ifdef CONFIG_DEBUG_SHIRQ if (irqflags & IRQF_SHARED) { /* * It's a shared IRQ -- the driver ought to be prepared for it @@ -554,7 +549,6 @@ int request_irq(unsigned int irq, irq_ha } else handler(irq, dev_id); } -#endif retval = setup_irq(irq, action); if (retval) diff -urNp linux-2.6.22-orig/lib/Kconfig.debug linux-2.6.22-pendirq/lib/Kconfig.debug --- linux-2.6.22-orig/lib/Kconfig.debug 2007-07-09 08:32:17.000000000 +0900 +++ linux-2.6.22-pendirq/lib/Kconfig.debug 2007-07-19 17:05:52.000000000 +0900 @@ -77,15 +77,6 @@ config DEBUG_KERNEL Say Y here if you are developing drivers or trying to debug and identify kernel problems. -config DEBUG_SHIRQ - bool "Debug shared IRQ handlers" - depends on DEBUG_KERNEL && GENERIC_HARDIRQS - help - Enable this to generate a spurious interrupt as soon as a shared - interrupt handler is registered, and just before one is deregistered. - Drivers ought to be able to handle interrupts coming in at those - points; some don't and need to be caught. - config DETECT_SOFTLOCKUP bool "Detect Soft Lockups" depends on DEBUG_KERNEL && !S390 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 2/2] Debug handling of early spurious interrupts 2007-07-20 2:02 ` [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ Fernando Luis Vázquez Cao @ 2007-07-20 2:20 ` Fernando Luis Vázquez Cao 2007-07-20 21:43 ` Andrew Morton 2007-07-25 9:18 ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao 0 siblings, 2 replies; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-20 2:20 UTC (permalink / raw) To: Andrew Morton; +Cc: kexec, linux-kernel With the advent of kdump it is possible that device drivers receive interrupts generated in the context of a previous kernel. Ideally quiescing the underlying devices should suffice but not all drivers do this, either because it is not possible or because they did not contemplate this case. Thus drivers ought to be able to handle interrupts coming in as soon as the interrupt handler is registered. Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> --- diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c --- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-19 18:18:53.000000000 +0900 +++ linux-2.6.22-pendirq/kernel/irq/manage.c 2007-07-19 19:43:41.000000000 +0900 @@ -533,21 +533,32 @@ int request_irq(unsigned int irq, irq_ha select_smp_affinity(irq); - if (irqflags & IRQF_SHARED) { - /* - * It's a shared IRQ -- the driver ought to be prepared for it - * to happen immediately, so let's make sure.... - * We do this before actually registering it, to make sure that - * a 'real' IRQ doesn't run in parallel with our fake - */ - if (irqflags & IRQF_DISABLED) { - unsigned long flags; + /* + * With the advent of kdump it possible that device drivers receive + * interrupts generated in the context of a previous kernel. Ideally + * quiescing the underlying devices should suffice but not all drivers + * do this, either because it is not possible or because they did not + * contemplate this case. Thus drivers ought to be able to handle + * interrupts coming in as soon as the interrupt handler is registered. + * + * Besides, if it is a shared IRQ the driver ought to be prepared for + * it to happen immediately too. + * + * We do this before actually registering it, to make sure that + * a 'real' IRQ doesn't run in parallel with our fake. + */ + if (irqflags & IRQF_DISABLED) { + unsigned long flags; - local_irq_save(flags); - handler(irq, dev_id); - local_irq_restore(flags); - } else - handler(irq, dev_id); + local_irq_save(flags); + retval = handler(irq, dev_id); + local_irq_restore(flags); + } else + retval = handler(irq, dev_id); + if (retval == IRQ_HANDLED) { + printk(KERN_WARNING + "%s (IRQ %d) handled a spurious interrupt\n", + devname, irq); } retval = setup_irq(irq, action); ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] Debug handling of early spurious interrupts 2007-07-20 2:20 ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao @ 2007-07-20 21:43 ` Andrew Morton 2007-07-30 9:58 ` Fernando Luis Vázquez Cao 2007-07-25 9:18 ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao 1 sibling, 1 reply; 13+ messages in thread From: Andrew Morton @ 2007-07-20 21:43 UTC (permalink / raw) To: Fernando Luis Vázquez Cao; +Cc: kexec, linux-kernel On Fri, 20 Jul 2007 11:20:43 +0900 Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote: > With the advent of kdump it is possible that device drivers receive > interrupts generated in the context of a previous kernel. Ideally > quiescing the underlying devices should suffice but not all drivers do > this, either because it is not possible or because they did not > contemplate this case. Thus drivers ought to be able to handle > interrupts coming in as soon as the interrupt handler is registered. > > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> > --- > > diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c > --- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-19 18:18:53.000000000 +0900 > +++ linux-2.6.22-pendirq/kernel/irq/manage.c 2007-07-19 19:43:41.000000000 +0900 > @@ -533,21 +533,32 @@ int request_irq(unsigned int irq, irq_ha > > select_smp_affinity(irq); > > - if (irqflags & IRQF_SHARED) { > - /* > - * It's a shared IRQ -- the driver ought to be prepared for it > - * to happen immediately, so let's make sure.... > - * We do this before actually registering it, to make sure that > - * a 'real' IRQ doesn't run in parallel with our fake > - */ > - if (irqflags & IRQF_DISABLED) { > - unsigned long flags; > + /* > + * With the advent of kdump it possible that device drivers receive > + * interrupts generated in the context of a previous kernel. Ideally > + * quiescing the underlying devices should suffice but not all drivers > + * do this, either because it is not possible or because they did not > + * contemplate this case. Thus drivers ought to be able to handle > + * interrupts coming in as soon as the interrupt handler is registered. > + * > + * Besides, if it is a shared IRQ the driver ought to be prepared for > + * it to happen immediately too. > + * > + * We do this before actually registering it, to make sure that > + * a 'real' IRQ doesn't run in parallel with our fake. > + */ > + if (irqflags & IRQF_DISABLED) { > + unsigned long flags; > > - local_irq_save(flags); > - handler(irq, dev_id); > - local_irq_restore(flags); > - } else > - handler(irq, dev_id); > + local_irq_save(flags); > + retval = handler(irq, dev_id); > + local_irq_restore(flags); > + } else > + retval = handler(irq, dev_id); > + if (retval == IRQ_HANDLED) { > + printk(KERN_WARNING > + "%s (IRQ %d) handled a spurious interrupt\n", > + devname, irq); > } > This change means that we'll run the irq handler at request_irq()-time even for non-shared interrupts. This is a bit of a worry. See, shared-interrupt handlers are required to be able to cope with being called when their device isn't interrupting. But nobody ever said that non-shared interrupt handlers need to be able to cope with that. Hence these non-shared handlers are within their rights to emit warning printks, go BUG or accuse the operator of having busted hardware. So bad things might happen because of this change. And if they do, they will take a loooong time to be discovered, because non-shared interrupt handlers tend to dwell in crufty old drivers which not many people use. So I'm wondering if it would be more prudent to only do all this for shared handlers? ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] Debug handling of early spurious interrupts 2007-07-20 21:43 ` Andrew Morton @ 2007-07-30 9:58 ` Fernando Luis Vázquez Cao 2007-07-30 18:22 ` Andrew Morton 0 siblings, 1 reply; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-30 9:58 UTC (permalink / raw) To: Andrew Morton; +Cc: kexec, linux-kernel On Fri, 2007-07-20 at 14:43 -0700, Andrew Morton wrote: > On Fri, 20 Jul 2007 11:20:43 +0900 > Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote: > > > With the advent of kdump it is possible that device drivers receive > > interrupts generated in the context of a previous kernel. Ideally > > quiescing the underlying devices should suffice but not all drivers do > > this, either because it is not possible or because they did not > > contemplate this case. Thus drivers ought to be able to handle > > interrupts coming in as soon as the interrupt handler is registered. > > > > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> > > --- > > > > diff -urNp linux-2.6.22-orig/kernel/irq/manage.c linux-2.6.22-pendirq/kernel/irq/manage.c > > --- linux-2.6.22-orig/kernel/irq/manage.c 2007-07-19 18:18:53.000000000 +0900 > > +++ linux-2.6.22-pendirq/kernel/irq/manage.c 2007-07-19 19:43:41.000000000 +0900 > > @@ -533,21 +533,32 @@ int request_irq(unsigned int irq, irq_ha > > > > select_smp_affinity(irq); > > > > - if (irqflags & IRQF_SHARED) { > > - /* > > - * It's a shared IRQ -- the driver ought to be prepared for it > > - * to happen immediately, so let's make sure.... > > - * We do this before actually registering it, to make sure that > > - * a 'real' IRQ doesn't run in parallel with our fake > > - */ > > - if (irqflags & IRQF_DISABLED) { > > - unsigned long flags; > > + /* > > + * With the advent of kdump it possible that device drivers receive > > + * interrupts generated in the context of a previous kernel. Ideally > > + * quiescing the underlying devices should suffice but not all drivers > > + * do this, either because it is not possible or because they did not > > + * contemplate this case. Thus drivers ought to be able to handle > > + * interrupts coming in as soon as the interrupt handler is registered. > > + * > > + * Besides, if it is a shared IRQ the driver ought to be prepared for > > + * it to happen immediately too. > > + * > > + * We do this before actually registering it, to make sure that > > + * a 'real' IRQ doesn't run in parallel with our fake. > > + */ > > + if (irqflags & IRQF_DISABLED) { > > + unsigned long flags; > > > > - local_irq_save(flags); > > - handler(irq, dev_id); > > - local_irq_restore(flags); > > - } else > > - handler(irq, dev_id); > > + local_irq_save(flags); > > + retval = handler(irq, dev_id); > > + local_irq_restore(flags); > > + } else > > + retval = handler(irq, dev_id); > > + if (retval == IRQ_HANDLED) { > > + printk(KERN_WARNING > > + "%s (IRQ %d) handled a spurious interrupt\n", > > + devname, irq); > > } > > > > This change means that we'll run the irq handler at request_irq()-time even > for non-shared interrupts. > > This is a bit of a worry. See, shared-interrupt handlers are required to > be able to cope with being called when their device isn't interrupting. > But nobody ever said that non-shared interrupt handlers need to be able to > cope with that. > > Hence these non-shared handlers are within their rights to emit warning > printks, go BUG or accuse the operator of having busted hardware. > > So bad things might happen because of this change. And if they do, they > will take a loooong time to be discovered, because non-shared interrupt > handlers tend to dwell in crufty old drivers which not many people use. > > So I'm wondering if it would be more prudent to only do all this for shared > handlers? I have been testing this patches on all my machines, which spans three different architectures: i386, x86_64 (EM64T and Opteron), and ia64. The good news is that nothing catastrophic happened and, in the process, I managed to find some somewhat buggy interrupt handlers. I will present a brief summary of my findings here. That said, I have to admit that these are still preliminary results and have not had time to dig deep into all the issues. I would like hear from you first. - Test environment -- x86_64 [EM64T] CPU: Intel(R) Xeon(TM) CPU 3.20GHz (2 cpus) SCSI: Adaptec AIC-7902B U320 IDE: Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE Controller -- x86_64 [Opteron] CPU: AMD Opteron(tm) Processor 240 (2 cpus) IDE: Advanced Micro Devices [AMD] AMD-8111 IDE (rev 03) -- ia64 CPU: Itanium 2 Madison (2 cpus) SCSI: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI IDE: Intel Corporation 82801DB (ICH4) IDE Controller -- i386 CPU: Intel(R) Xeon(TM) CPU 2.40GHz (2 cpus) IDE: Intel Corporation 82801EB/ER (ICH5/ICH5R) IDE Controller (rev 02) - Test results for i386 --i8042 (IRQ 12) handled a spurious interrupt -> i8042 The culprit here was the i8042_aux_test_irq interrupt handler unconditionally returning IRQ_HANDLED despite being marked as IRQF_SHARED. I submitted a fix and it is currently sitting in the 'for-linus' branch of Dmitry Torokhov's input tree (Dmitry, Alan, Vojtech, thank you for reviewing the patch!). -- eth0 (IRQ 16) handled a spurious interrupt -> e1000 The problem here is that the network card is active before it has been assigned an interrupt line. This means that if the card detects an event such as a link status change it will update the status register (ICR) by setting the corresponding flag, even if request_irq has not been invoked yet. In such a case, once the interrupt handler is set if a spurious interrupt comes in it will be handled. The reason is that the handler checks ICR to determine whether the underlying the device has generated an interrupt or not, and the flag that indicates a link status change will happen to be set. I have sent a patch that clears the ICR before requesting an IRQ, but according to Auke Kok there is a small window during which the device could generate another link status change interrupt. I am looking for a way to prevent the device from reacting to such events before an IRQ has been allocated, but still do not know if it is possible. -- rtc (IRQ 8) handled a spurious interrupt -- parport0 (IRQ 7) handled a spurious interrupt -- MPU401 UART (IRQ 10) handled a spurious interrupt I have not looked into these three yet but it seems that in some cases it is not possible to determine whether an interrupt has actually occurred. - Test results for x86_64 [EM64T] -- i8042 (IRQ 12) handled a spurious interrupt -> i8042 Ditto. -- eth0 (IRQ 54) handled a spurious interrupt -> e1000 Ditto. -- rtc (IRQ 8) handled a spurious interrupt Ditto. - Test results for x86_64 [Opteron] -- i8042 (IRQ 12) handled a spurious interrupt -> i8042 Ditto. -- rtc (IRQ 8) handled a spurious interrupt Ditto. -- parport0 (IRQ 7) handled a spurious interrupt Ditto. - Test results for ia64 -- eth0 (IRQ 48) handled a spurious interrupt -> e1000 Ditto. Besides these patches have exposed bugs in other people's machines too. Valdis Kletnieks discovered a problem in his tpm driver and it seems that the TPM guys are already looking into it. - Conclusion These patches can affect device drivers in several ways and, so far, I have identified 5 possible scenarios: 1. Shared handlers that behave as expected and ignore spurious interrupts. 2. Shared handlers that unconditionally handle any incoming interrupt. 3. Shared handlers that cannot distinguish between interrupts that it needs to handle and interrupts generated by other devices in certain situations. 4. Non-shared handlers that behave as expected and ignore spurious interrupts. 5. Drivers for legacy devices in which it is not possible to determine whether a given interrupt has actually come from the underlying device. Obviously, 1) and 4) are the ideal cases and (probably) the majority of the drivers fall into this group. 2) means that the driver cannot cohabit harmoniously with other drivers and it would end up handling any interrupt coming in from that interrupt line regardless of its origin. i8042 used to be belong to this group, but the aforementioned patch should have solved it. Most devices have a interrupt pending bit in some port or an interrupt cause register (ICR) that can be checked by the driver to determine what interrupts it needs to handle. The problem is that some devices are operational before request_irq has been called and react to external events by updating the interrupt pending bit or the ICR. This means that after actually registering the interrupt handler, any interrupt coming in would be handled. 3) refers to such cases. By the way, the e1000 driver seems to be suffering this issue which causes the error message above. When it comes to 5) there is not much we can do. Sometimes you simply cannot tell. The bottom line is that we have to decide how two tackle 2), 3) and 5). My gut feeling is that in most cases 2) and 3) constitute a bug, because if the driver is not fixable then IRQF_SHARED should not have been set in the first place. Regarding 5), there is no choice, so it seems we would need to mark such drivers somehow (another possible solution would be calling the interrupt handler from request_irq only when IRQF_SHARED is set, because shared handlers should not be affected by this problem). If we find drivers that are not fixable we should probably consider this new behaviour on a per-driver basis, as Andrew suggested. This would probably require passing a new flag into request_irq. Besides, when such a driver is detected we should emit a warning that it may not work properly in a kdump kernel. I would appreciate your comments on this. - Fernando ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] Debug handling of early spurious interrupts 2007-07-30 9:58 ` Fernando Luis Vázquez Cao @ 2007-07-30 18:22 ` Andrew Morton 2007-07-31 2:25 ` Fernando Luis Vázquez Cao 0 siblings, 1 reply; 13+ messages in thread From: Andrew Morton @ 2007-07-30 18:22 UTC (permalink / raw) To: Fernando Luis Vázquez Cao; +Cc: kexec, linux-kernel On Mon, 30 Jul 2007 18:58:14 +0900 Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote: > > > > So bad things might happen because of this change. And if they do, they > > will take a loooong time to be discovered, because non-shared interrupt > > handlers tend to dwell in crufty old drivers which not many people use. > > > > So I'm wondering if it would be more prudent to only do all this for shared > > handlers? > > I have been testing this patches on all my machines, which spans three > different architectures: i386, x86_64 (EM64T and Opteron), and ia64. > > The good news is that nothing catastrophic happened and, in the process, > I managed to find some somewhat buggy interrupt handlers. > > I will present a brief summary of my findings here. That's quite a lot of breakage for such a small sample of machines. I do suspect that if we were to merge this lot into mainline, all sorts of bad stuff would happen. otoh, as you point out, pretty much everthing which goes wrong is due to incorrect or dubious driver behaviour, and there is value in weeding these things out. But the problem with this process is that we're weeding things out at runtime. Some drivers don't get used by many people and users of some architectures (esp embedded) tend to lag kernel.org by a long time. So it could be years before all the fallout from this change is finally wrapped up. > > If we find drivers that are not fixable we should probably consider this > new behaviour on a per-driver basis, as Andrew suggested. We haven't found any such yet, have we? > This would > probably require passing a new flag into request_irq. Besides, when such > a driver is detected we should emit a warning that it may not work > properly in a kdump kernel. > > I would appreciate your comments on this. Oh well, let's just keep maintaining it in -mm for now, gather more information so that we can make a more informed decision later on. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] Debug handling of early spurious interrupts 2007-07-30 18:22 ` Andrew Morton @ 2007-07-31 2:25 ` Fernando Luis Vázquez Cao 2007-07-31 4:46 ` Andrew Morton 0 siblings, 1 reply; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-31 2:25 UTC (permalink / raw) To: Andrew Morton; +Cc: kexec, linux-kernel On Mon, 2007-07-30 at 11:22 -0700, Andrew Morton wrote: > On Mon, 30 Jul 2007 18:58:14 +0900 > Fernando Luis V__zquez Cao <fernando@oss.ntt.co.jp> wrote: > > > > > > > So bad things might happen because of this change. And if they do, they > > > will take a loooong time to be discovered, because non-shared interrupt > > > handlers tend to dwell in crufty old drivers which not many people use. > > > > > > So I'm wondering if it would be more prudent to only do all this for shared > > > handlers? > > > > I have been testing this patches on all my machines, which spans three > > different architectures: i386, x86_64 (EM64T and Opteron), and ia64. > > > > The good news is that nothing catastrophic happened and, in the process, > > I managed to find some somewhat buggy interrupt handlers. > > > > I will present a brief summary of my findings here. > > That's quite a lot of breakage for such a small sample of machines. I do > suspect that if we were to merge this lot into mainline, all sorts of bad > stuff would happen. Yup. > otoh, as you point out, pretty much everthing which goes wrong is due to > incorrect or dubious driver behaviour, and there is value in weeding these > things out. > > But the problem with this process is that we're weeding things out at > runtime. Some drivers don't get used by many people and users of some > architectures (esp embedded) tend to lag kernel.org by a long time. So it > could be years before all the fallout from this change is finally wrapped > up. Yes, that is a big concern. However, the same embedded people is starting to use both kexec and kdump, so they may suffer the issues we are trying to weed out anyway, even if these patches are not applied. The difference is that with this new functionality it is possible to catch potential problems relatively easily, because any incorrect behaviour this may cause will be easily reproducible and, in most cases, will reveal itself early at boot time. As things stand now, I guess we will keep seeing occasional crashes and strange behaviour in kexec-booted kernels, which in some cases will be due to incorrect handling of spurious interrupts. Besides, such problems are really difficult to reproduce because, commonly, we would need to hit an obscure corner case. > > If we find drivers that are not fixable we should probably consider this > > new behaviour on a per-driver basis, as Andrew suggested. > > We haven't found any such yet, have we? Not yet, fortunately. > > This would > > probably require passing a new flag into request_irq. Besides, when such > > a driver is detected we should emit a warning that it may not work > > properly in a kdump kernel. > > > > I would appreciate your comments on this. > > Oh well, let's just keep maintaining it in -mm for now, gather more > information so that we can make a more informed decision later on. Makes sense. I will look into all the issues I have found so far, do some more testing (I think a new approach is needed to speed up testing of these issues...), and get back to you. Thank you. Fernando ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 2/2] Debug handling of early spurious interrupts 2007-07-31 2:25 ` Fernando Luis Vázquez Cao @ 2007-07-31 4:46 ` Andrew Morton 0 siblings, 0 replies; 13+ messages in thread From: Andrew Morton @ 2007-07-31 4:46 UTC (permalink / raw) To: Fernando Luis Vázquez Cao; +Cc: kexec, linux-kernel On Tue, 31 Jul 2007 11:25:26 +0900 Fernando Luis Vázquez Cao <fernando@oss.ntt.co.jp> wrote: > > runtime. Some drivers don't get used by many people and users of some > > architectures (esp embedded) tend to lag kernel.org by a long time. So it > > could be years before all the fallout from this change is finally wrapped > > up. > Yes, that is a big concern. However, the same embedded people is > starting to use both kexec and kdump, so they may suffer the issues we > are trying to weed out anyway, even if these patches are not applied. > The difference is that with this new functionality it is possible to > catch potential problems relatively easily, because any incorrect > behaviour this may cause will be easily reproducible and, in most cases, > will reveal itself early at boot time. > > As things stand now, I guess we will keep seeing occasional crashes and > strange behaviour in kexec-booted kernels, which in some cases will be > due to incorrect handling of spurious interrupts. Besides, such problems > are really difficult to reproduce because, commonly, we would need to > hit an obscure corner case. Please have a think about what we can do to aid this debugging. For example, add an uncondtional printk into request_irq() for now which tells us what irq is being registered - a print_symbol() of the irq_handler_t would be pretty good, although I suspect there are a lot of interrupt handlers with the same name.. ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH RFC] e1000: clear ICR before requesting an IRQ line 2007-07-20 2:20 ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao 2007-07-20 21:43 ` Andrew Morton @ 2007-07-25 9:18 ` Fernando Luis Vázquez Cao 2007-07-25 15:27 ` Kok, Auke 1 sibling, 1 reply; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-25 9:18 UTC (permalink / raw) To: e1000-devel Cc: Andrew Morton, linux-kernel, john.ronciak, jesse.brandeburg, jeffrey.t.kirsher, auke-jan.h.kok I made an interesting finding while testing the two patches below. http://lkml.org/lkml/2007/7/19/685 http://lkml.org/lkml/2007/7/19/687 These patches modify the traditional CONFIG_DEBUG_KERNEL in such a way that the request_irq prints a warning if after calling the handler it returned IRQ_HANDLED . The code looks like this: int request_irq(unsigned int irq, irq_handler_t handler, unsigned long irqflags, const char *devname, void *dev_id) ..... if (irqflags & IRQF_DISABLED) { unsigned long flags; local_irq_save(flags); retval = handler(irq, dev_id); local_irq_restore(flags); } else retval = handler(irq, dev_id); if (retval == IRQ_HANDLED) { printk(KERN_WARNING "%s (IRQ %d) handled a spurious interrupt\n", devname, irq); } ..... I discovered that the e1000 driver handles the "fake" interrupt, which, in principle, is not correct because it obviously isn't a real interrupt and it could have been an interrupt coming from another device that is sharing the IRQ line. The problem is that the interrupt handler assumes that if ICR!=0 it was its device who generated the interrupt and, consequently, it should be handled. But, unfortunately, that is not always the case. If the network link is active when we open the device (e1000_open) the ICR will have the E1000_ICR_LSC bit set (by the way, is this the expected behavior?). This means that _any_ interrupt coming in after allocating our interrupt (e1000_request_irq) will be handled, no matter where it came from. The solution I came up with is clearing the ICR before calling request_irq. I have to admit that I am not familiar enough with this driver, so it is quite likely that this is not the right fix. I would appreciate your comments on this. Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> --- diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c --- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c 2007-07-19 18:18:53.000000000 +0900 +++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c 2007-07-25 17:22:54.000000000 +0900 @@ -1378,6 +1378,17 @@ e1000_alloc_queues(struct e1000_adapter } /** + * e1000_clear_interrupts + * @adapter: address of board private structure + * + * Mask interrupts + **/ +static void +e1000_clear_interrupts(struct e1000_adapter *adapter) { + E1000_READ_REG(&adapter->hw, ICR); +} + +/** * e1000_open - Called when a network interface is made active * @netdev: network interface device structure * @@ -1431,6 +1442,9 @@ e1000_open(struct net_device *netdev) * so we have to setup our clean_rx handler before we do so. */ e1000_configure(adapter); + /* Discard any possible pending interrupts. */ + e1000_clear_interrupts(adapter); + err = e1000_request_irq(adapter); if (err) goto err_req_irq; ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH RFC] e1000: clear ICR before requesting an IRQ line 2007-07-25 9:18 ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao @ 2007-07-25 15:27 ` Kok, Auke 2007-07-26 1:34 ` Fernando Luis Vázquez Cao 0 siblings, 1 reply; 13+ messages in thread From: Kok, Auke @ 2007-07-25 15:27 UTC (permalink / raw) To: Fernando Luis Vázquez Cao Cc: e1000-devel, Andrew Morton, linux-kernel, john.ronciak, jesse.brandeburg, jeffrey.t.kirsher Fernando Luis Vázquez Cao wrote: > I made an interesting finding while testing the two patches below. > > http://lkml.org/lkml/2007/7/19/685 > http://lkml.org/lkml/2007/7/19/687 > > These patches modify the traditional CONFIG_DEBUG_KERNEL in such a way > that the request_irq prints a warning if after calling the handler it > returned IRQ_HANDLED . > > The code looks like this: > > int request_irq(unsigned int irq, irq_handler_t handler, > unsigned long irqflags, const char *devname, void *dev_id) > ..... > if (irqflags & IRQF_DISABLED) { > unsigned long flags; > > local_irq_save(flags); > retval = handler(irq, dev_id); > local_irq_restore(flags); > } else > retval = handler(irq, dev_id); > if (retval == IRQ_HANDLED) { > printk(KERN_WARNING > "%s (IRQ %d) handled a spurious interrupt\n", > devname, irq); > } > ..... > > I discovered that the e1000 driver handles the "fake" interrupt, which, > in principle, is not correct because it obviously isn't a real interrupt > and it could have been an interrupt coming from another device that is > sharing the IRQ line. > > The problem is that the interrupt handler assumes that if ICR!=0 it was > its device who generated the interrupt and, consequently, it should be > handled. But, unfortunately, that is not always the case. If the network > link is active when we open the device (e1000_open) the ICR will have > the E1000_ICR_LSC bit set (by the way, is this the expected behavior?). yes. is it really a problem though? > This means that _any_ interrupt coming in after allocating our interrupt > (e1000_request_irq) will be handled, no matter where it came from. we actually generate this LSC interrupt ourselves in the driver, to make sure that we cascade into the watchdog which then enables or disables the link code based on the link status change. This allows us to _not_ do any link checking in _open and makes things a bit more simple. > The solution I came up with is clearing the ICR before calling > request_irq. I have to admit that I am not familiar enough with this > driver, so it is quite likely that this is not the right fix. I would > appreciate your comments on this. Clearing the ICR before requesting an irq might not work - at the same time the device could generate another LSC irq... Of course, we probably should just schedule some delayed work to run our watchdog in e1000_open, but I haven't checked if that actually works. Auke > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> > --- > > diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c > --- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c 2007-07-19 18:18:53.000000000 +0900 > +++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c 2007-07-25 17:22:54.000000000 +0900 > @@ -1378,6 +1378,17 @@ e1000_alloc_queues(struct e1000_adapter > } > > /** > + * e1000_clear_interrupts > + * @adapter: address of board private structure > + * > + * Mask interrupts > + **/ > +static void > +e1000_clear_interrupts(struct e1000_adapter *adapter) { > + E1000_READ_REG(&adapter->hw, ICR); > +} > + > +/** > * e1000_open - Called when a network interface is made active > * @netdev: network interface device structure > * > @@ -1431,6 +1442,9 @@ e1000_open(struct net_device *netdev) > * so we have to setup our clean_rx handler before we do so. */ > e1000_configure(adapter); > > + /* Discard any possible pending interrupts. */ > + e1000_clear_interrupts(adapter); > + > err = e1000_request_irq(adapter); > if (err) > goto err_req_irq; ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH RFC] e1000: clear ICR before requesting an IRQ line 2007-07-25 15:27 ` Kok, Auke @ 2007-07-26 1:34 ` Fernando Luis Vázquez Cao 0 siblings, 0 replies; 13+ messages in thread From: Fernando Luis Vázquez Cao @ 2007-07-26 1:34 UTC (permalink / raw) To: Kok, Auke Cc: e1000-devel, Andrew Morton, linux-kernel, john.ronciak, jesse.brandeburg, jeffrey.t.kirsher On Wed, 2007-07-25 at 08:27 -0700, Kok, Auke wrote: > Fernando Luis Vázquez Cao wrote: > > I made an interesting finding while testing the two patches below. > > > > http://lkml.org/lkml/2007/7/19/685 > > http://lkml.org/lkml/2007/7/19/687 > > > > These patches modify the traditional CONFIG_DEBUG_KERNEL in such a way > > that the request_irq prints a warning if after calling the handler it > > returned IRQ_HANDLED . > > > > The code looks like this: > > > > int request_irq(unsigned int irq, irq_handler_t handler, > > unsigned long irqflags, const char *devname, void *dev_id) > > ..... > > if (irqflags & IRQF_DISABLED) { > > unsigned long flags; > > > > local_irq_save(flags); > > retval = handler(irq, dev_id); > > local_irq_restore(flags); > > } else > > retval = handler(irq, dev_id); > > if (retval == IRQ_HANDLED) { > > printk(KERN_WARNING > > "%s (IRQ %d) handled a spurious interrupt\n", > > devname, irq); > > } > > ..... > > > > I discovered that the e1000 driver handles the "fake" interrupt, which, > > in principle, is not correct because it obviously isn't a real interrupt > > and it could have been an interrupt coming from another device that is > > sharing the IRQ line. > > > > The problem is that the interrupt handler assumes that if ICR!=0 it was > > its device who generated the interrupt and, consequently, it should be > > handled. But, unfortunately, that is not always the case. If the network > > link is active when we open the device (e1000_open) the ICR will have > > the E1000_ICR_LSC bit set (by the way, is this the expected behavior?). > > yes. is it really a problem though? It seems we may end up handling spurious interrupts or interrupts coming from another devices. > > This means that _any_ interrupt coming in after allocating our interrupt > > (e1000_request_irq) will be handled, no matter where it came from. > > we actually generate this LSC interrupt ourselves in the driver, to make sure > that we cascade into the watchdog which then enables or disables the link code > based on the link status change. This allows us to _not_ do any link checking in > _open and makes things a bit more simple. I am not referring to the LSC interrupt the driver itself generates in e1000_open just before returning. The ICR is masked (ICR==0) after executing the driver probe (e1000_probe), but when we enter e1000_open the E1000_ICR_LSC bit will already be set, before the function even starts executing. I also observed that when the link is active the line /* fire a link status change interrupt to start the watchdog */ E1000_WRITE_REG(&adapter->hw, ICS, E1000_ICS_LSC); is redundant because the E1000_ICS_LSC bit is already set. In fact, the irq handler gets invoked twice in a row with the interrupt cause being a link status change. > > The solution I came up with is clearing the ICR before calling > > request_irq. I have to admit that I am not familiar enough with this > > driver, so it is quite likely that this is not the right fix. I would > > appreciate your comments on this. > > Clearing the ICR before requesting an irq might not work - at the same time the > device could generate another LSC irq... Is it not possible to prevent the device from generating interrupts until we call request_irq? Thank you for your feedback! - Fernando > Of course, we probably should just schedule some delayed work to run our > watchdog in e1000_open, but I haven't checked if that actually works. > > > Auke > > > Signed-off-by: Fernando Luis Vazquez Cao <fernando@oss.ntt.co.jp> > > --- > > > > diff -urNp linux-2.6.22-orig/drivers/net/e1000/e1000_main.c linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c > > --- linux-2.6.22-orig/drivers/net/e1000/e1000_main.c 2007-07-19 18:18:53.000000000 +0900 > > +++ linux-2.6.22-pendirq/drivers/net/e1000/e1000_main.c 2007-07-25 17:22:54.000000000 +0900 > > @@ -1378,6 +1378,17 @@ e1000_alloc_queues(struct e1000_adapter > > } > > > > /** > > + * e1000_clear_interrupts > > + * @adapter: address of board private structure > > + * > > + * Mask interrupts > > + **/ > > +static void > > +e1000_clear_interrupts(struct e1000_adapter *adapter) { > > + E1000_READ_REG(&adapter->hw, ICR); > > +} > > + > > +/** > > * e1000_open - Called when a network interface is made active > > * @netdev: network interface device structure > > * > > @@ -1431,6 +1442,9 @@ e1000_open(struct net_device *netdev) > > * so we have to setup our clean_rx handler before we do so. */ > > e1000_configure(adapter); > > > > + /* Discard any possible pending interrupts. */ > > + e1000_clear_interrupts(adapter); > > + > > err = e1000_request_irq(adapter); > > if (err) > > goto err_req_irq; ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2007-07-31 4:47 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2007-07-17 10:09 [PATCH RFC] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao 2007-07-18 22:46 ` Andrew Morton 2007-07-20 1:54 ` Fernando Luis Vázquez Cao 2007-07-20 2:02 ` [PATCH 1/2] Remove Kconfig setting CONFIG_DEBUG_SHIRQ Fernando Luis Vázquez Cao 2007-07-20 2:20 ` [PATCH 2/2] Debug handling of early spurious interrupts Fernando Luis Vázquez Cao 2007-07-20 21:43 ` Andrew Morton 2007-07-30 9:58 ` Fernando Luis Vázquez Cao 2007-07-30 18:22 ` Andrew Morton 2007-07-31 2:25 ` Fernando Luis Vázquez Cao 2007-07-31 4:46 ` Andrew Morton 2007-07-25 9:18 ` [PATCH RFC] e1000: clear ICR before requesting an IRQ line Fernando Luis Vázquez Cao 2007-07-25 15:27 ` Kok, Auke 2007-07-26 1:34 ` Fernando Luis Vázquez Cao
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox