[PATCH -rt] powerpc update

[PATCH -rt] powerpc update

[Daniel Walker]

This updates arch/powerpc/ so that it boots PREEMPT_RT on a Apple 
G4 PowerBook. It should also get other powerpc boards functional to
some extent. 

Pay close attention to the fasteoi interrupt threading. I added usage 
of mask/unmask instead of using level handling, which worked well on PPC.

There is still a todo to update highmems kmap_atomic() for realtime. 
So don't turn on highmem. 

Signed-Off-By: Daniel Walker <dwalker@mvista.com>

---
 arch/powerpc/Kconfig.debug         |    4 ++++
 arch/powerpc/kernel/idle.c         |    2 +-
 drivers/macintosh/adb.c            |   13 +++++++------
 drivers/net/sungem.c               |    4 +---
 include/asm-powerpc/hw_irq.h       |    2 +-
 include/asm-powerpc/pmac_feature.h |    2 +-
 kernel/irq/chip.c                  |   17 +++++++++++------
 7 files changed, 26 insertions(+), 18 deletions(-)

Index: linux-2.6.18/arch/powerpc/Kconfig.debug
===================================================================
--- linux-2.6.18.orig/arch/powerpc/Kconfig.debug
+++ linux-2.6.18/arch/powerpc/Kconfig.debug
@@ -2,6 +2,10 @@ menu "Kernel hacking"
 
 source "lib/Kconfig.debug"
 
+config TRACE_IRQFLAGS_SUPPORT
+	bool
+	default y
+
 config DEBUG_STACKOVERFLOW
 	bool "Check for stack overflows"
 	depends on DEBUG_KERNEL && PPC64
Index: linux-2.6.18/arch/powerpc/kernel/idle.c
===================================================================
--- linux-2.6.18.orig/arch/powerpc/kernel/idle.c
+++ linux-2.6.18/arch/powerpc/kernel/idle.c
@@ -82,7 +82,7 @@ void cpu_idle(void)
 		ppc64_runlatch_on();
 		if (cpu_should_die())
 			cpu_die();
-		preempt_enable_no_resched();
+		__preempt_enable_no_resched();
 		schedule();
 		preempt_disable();
 	}
Index: linux-2.6.18/drivers/macintosh/adb.c
===================================================================
--- linux-2.6.18.orig/drivers/macintosh/adb.c
+++ linux-2.6.18/drivers/macintosh/adb.c
@@ -256,6 +256,8 @@ adb_probe_task(void *x)
 	sigprocmask(SIG_BLOCK, &blocked, NULL);
 	flush_signals(current);
 
+	down(&adb_probe_mutex);
+
 	printk(KERN_INFO "adb: starting probe task...\n");
 	do_adb_reset_bus();
 	printk(KERN_INFO "adb: finished probe task...\n");
@@ -282,7 +284,9 @@ adb_reset_bus(void)
 		return 0;
 	}
 
-	down(&adb_probe_mutex);
+	if (adb_got_sleep)
+		return 0;
+
 	schedule_work(&adb_reset_work);
 	return 0;
 }
@@ -347,23 +351,21 @@ adb_notify_sleep(struct pmu_sleep_notifi
 	
 	switch (when) {
 	case PBOOK_SLEEP_REQUEST:
+		/* Signal to discontiue probing  */
 		adb_got_sleep = 1;
-		/* We need to get a lock on the probe thread */
-		down(&adb_probe_mutex);
 		/* Stop autopoll */
 		if (adb_controller->autopoll)
 			adb_controller->autopoll(0);
 		ret = blocking_notifier_call_chain(&adb_client_list,
 				ADB_MSG_POWERDOWN, NULL);
 		if (ret & NOTIFY_STOP_MASK) {
-			up(&adb_probe_mutex);
+			adb_got_sleep = 0;
 			return PBOOK_SLEEP_REFUSE;
 		}
 		break;
 	case PBOOK_SLEEP_REJECT:
 		if (adb_got_sleep) {
 			adb_got_sleep = 0;
-			up(&adb_probe_mutex);
 			adb_reset_bus();
 		}
 		break;
@@ -372,7 +374,6 @@ adb_notify_sleep(struct pmu_sleep_notifi
 		break;
 	case PBOOK_WAKE:
 		adb_got_sleep = 0;
-		up(&adb_probe_mutex);
 		adb_reset_bus();
 		break;
 	}
Index: linux-2.6.18/drivers/net/sungem.c
===================================================================
--- linux-2.6.18.orig/drivers/net/sungem.c
+++ linux-2.6.18/drivers/net/sungem.c
@@ -1037,10 +1037,8 @@ static int gem_start_xmit(struct sk_buff
 			(csum_stuff_off << 21));
 	}
 
-	local_irq_save(flags);
-	if (!spin_trylock(&gp->tx_lock)) {
+	if (!spin_trylock_irqsave(&gp->tx_lock, flags)) {
 		/* Tell upper layer to requeue */
-		local_irq_restore(flags);
 		return NETDEV_TX_LOCKED;
 	}
 	/* We raced with gem_do_stop() */
Index: linux-2.6.18/include/asm-powerpc/hw_irq.h
===================================================================
--- linux-2.6.18.orig/include/asm-powerpc/hw_irq.h
+++ linux-2.6.18/include/asm-powerpc/hw_irq.h
@@ -85,7 +85,7 @@ static inline void raw_local_irq_save_pt
 
 #endif /* CONFIG_PPC_ISERIES */
 
-#include <linux/trace_irqflags.h>
+#include <linux/irqflags.h>
 
 #define mask_irq(irq)						\
 	({							\
Index: linux-2.6.18/include/asm-powerpc/pmac_feature.h
===================================================================
--- linux-2.6.18.orig/include/asm-powerpc/pmac_feature.h
+++ linux-2.6.18/include/asm-powerpc/pmac_feature.h
@@ -378,7 +378,7 @@ extern struct macio_chip* macio_find(str
  * Those are exported by pmac feature for internal use by arch code
  * only like the platform function callbacks, do not use directly in drivers
  */
-extern spinlock_t feature_lock;
+extern raw_spinlock_t feature_lock;
 extern struct device_node *uninorth_node;
 extern u32 __iomem *uninorth_base;
 
Index: linux-2.6.18/kernel/irq/chip.c
===================================================================
--- linux-2.6.18.orig/kernel/irq/chip.c
+++ linux-2.6.18/kernel/irq/chip.c
@@ -386,13 +386,19 @@ handle_fasteoi_irq(unsigned int irq, str
 	}
 
 	desc->status |= IRQ_INPROGRESS;
-	/*
-	 * fasteoi should not be used for threaded IRQ handlers!
-	 */
+
 	if (redirect_hardirq(desc)) {
-		WARN_ON_ONCE(1);
-		goto out_unlock;
+		if (desc->chip->mask)
+			desc->chip->mask(irq);
+		else
+			/*
+			 * Threading on eoi chips w/o a mask/unmask is not
+			 * likely to work well.
+			 */
+			WARN_ON_ONCE(1);
+		goto out;
 	}
+
 	desc->status &= ~IRQ_PENDING;
 	spin_unlock(&desc->lock);
 
@@ -404,7 +410,6 @@ handle_fasteoi_irq(unsigned int irq, str
 	desc->status &= ~IRQ_INPROGRESS;
 out:
 	desc->chip->eoi(irq);
-out_unlock:
 	spin_unlock(&desc->lock);
 }
 
--

Re: [PATCH -rt] powerpc update

[Ingo Molnar]

* Daniel Walker <dwalker@mvista.com> wrote:

> Pay close attention to the fasteoi interrupt threading. I added usage 
> of mask/unmask instead of using level handling, which worked well on 
> PPC.

this is wrong - it should be doing mask+ack.

also note that you changed:

> -		goto out_unlock;

to:

> +		goto out;

and you even tried to hide your tracks:

>  out:
>  	desc->chip->eoi(irq);
> -out_unlock:
>  	spin_unlock(&desc->lock);

:-)

really, the ->eoi() op should only be called for true fasteoi cases. 
What we want here is to turn the fasteoi handler into a handler that 
does mask+ack and then unmask. Not 'mask+eoi ... unmask' as your patch 
does.

	Ingo

Re: [PATCH -rt] powerpc update

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> really, the ->eoi() op should only be called for true fasteoi cases. 
> What we want here is to turn the fasteoi handler into a handler that 
> does mask+ack and then unmask. Not 'mask+eoi ... unmask' as your patch 
> does.

i.e. like the much simpler patch below. That's what i've applied to the 
soon-to-be-rt6 tree, plus the other bits of your patch. Can you confirm 
this works well on PPC?

	Ingo

Index: linux-rt.q/kernel/irq/chip.c
===================================================================
--- linux-rt.q.orig/kernel/irq/chip.c
+++ linux-rt.q/kernel/irq/chip.c
@@ -386,13 +386,15 @@ handle_fasteoi_irq(unsigned int irq, str
 	}
 
 	desc->status |= IRQ_INPROGRESS;
+
 	/*
-	 * fasteoi should not be used for threaded IRQ handlers!
+	 * In the threaded case we fall back to a mask+ack sequence:
 	 */
 	if (redirect_hardirq(desc)) {
-		WARN_ON_ONCE(1);
+		mask_ack_irq(desc, irq);
 		goto out_unlock;
 	}
+
 	desc->status &= ~IRQ_PENDING;
 	spin_unlock(&desc->lock);
 

Re: [PATCH -rt] powerpc update

[Ingo Molnar]

* Daniel Walker <dwalker@mvista.com> wrote:

> > also note that you changed:
> 
> > > -		goto out_unlock;
> > 
> > to:
> > 
> > > +		goto out;
> > 
> > and you even tried to hide your tracks:
> 
> hiding something? I made note of these changes specifically so you 
> would review them.

hey, it was a joke:

> > :-)

	Ingo

Re: [PATCH -rt] powerpc update

[Daniel Walker]

On Wed, 2006-10-18 at 09:28 +0200, Ingo Molnar wrote:
> * Daniel Walker <dwalker@mvista.com> wrote:
> 
> > Pay close attention to the fasteoi interrupt threading. I added usage 
> > of mask/unmask instead of using level handling, which worked well on 
> > PPC.
> 
> this is wrong - it should be doing mask+ack.

The main reason I did it this way is cause the current threaded eoi
expected the line to be masked. So if you happen to have a eoi that's
threaded you get a warning then the interrupt hangs. 

> also note that you changed:

> > -		goto out_unlock;
> 
> to:
> 
> > +		goto out;
> 
> and you even tried to hide your tracks:

hiding something? I made note of these changes specifically so you would
review them.

Daniel

Re: [PATCH -rt] powerpc update

[Ingo Molnar]

* Daniel Walker <dwalker@mvista.com> wrote:

> On Wed, 2006-10-18 at 09:28 +0200, Ingo Molnar wrote:
> > * Daniel Walker <dwalker@mvista.com> wrote:
> > 
> > > Pay close attention to the fasteoi interrupt threading. I added usage 
> > > of mask/unmask instead of using level handling, which worked well on 
> > > PPC.
> > 
> > this is wrong - it should be doing mask+ack.
> 
> The main reason I did it this way is cause the current threaded eoi 
> expected the line to be masked. So if you happen to have a eoi that's 
> threaded you get a warning then the interrupt hangs.

does that in fact happen on -rt6? If yes, could you send the warning 
that is produced?

	Ingo

Re: [PATCH -rt] powerpc update

[Daniel Walker]

On Wed, 2006-10-18 at 16:33 +0200, Ingo Molnar wrote:
> * Daniel Walker <dwalker@mvista.com> wrote:
> 
> > On Wed, 2006-10-18 at 09:28 +0200, Ingo Molnar wrote:
> > > * Daniel Walker <dwalker@mvista.com> wrote:
> > > 
> > > > Pay close attention to the fasteoi interrupt threading. I added usage 
> > > > of mask/unmask instead of using level handling, which worked well on 
> > > > PPC.
> > > 
> > > this is wrong - it should be doing mask+ack.
> > 
> > The main reason I did it this way is cause the current threaded eoi 
> > expected the line to be masked. So if you happen to have a eoi that's 
> > threaded you get a warning then the interrupt hangs.
> 
> does that in fact happen on -rt6? If yes, could you send the warning 
> that is produced?
> 
> 	Ingo

The warning was the WARN_ON_ONCE from the section below (which is gone
is -rt6)

        if (redirect_hardirq(desc)) {
-               WARN_ON_ONCE(1);
+               mask_ack_irq(desc, irq);
                goto out_unlock;
        }
+

I haven't tried it, but I'd think it will work. That's assuming you have
an "ack" which I hear some PowerPC don't (Benjamin Herrenschmidt talked
about having no "ack" when discussing fasteoi a few month back).

Daniel

Re: [PATCH -rt] powerpc update

[Daniel Walker]

On Wed, 2006-10-18 at 16:33 +0200, Ingo Molnar wrote:
> * Daniel Walker <dwalker@mvista.com> wrote:
> 
> > On Wed, 2006-10-18 at 09:28 +0200, Ingo Molnar wrote:
> > > * Daniel Walker <dwalker@mvista.com> wrote:
> > > 
> > > > Pay close attention to the fasteoi interrupt threading. I added usage 
> > > > of mask/unmask instead of using level handling, which worked well on 
> > > > PPC.
> > > 
> > > this is wrong - it should be doing mask+ack.
> > 
> > The main reason I did it this way is cause the current threaded eoi 
> > expected the line to be masked. So if you happen to have a eoi that's 
> > threaded you get a warning then the interrupt hangs.
> 
> does that in fact happen on -rt6? If yes, could you send the warning 
> that is produced?

Here's the kernel messages (from Sergei who is CC'd)
---
Subject: 2.6.18-rt6 failure on MPC8540ADS

   My PPC32 board doesn't boot with 2.6.18-rt6, I'm getting this nice trace:

Time: timebase clocksource has been installed.
Unable to handle kernel paging request for instruction fetch
Faulting instruction address: 0x00000000
Oops: Kernel access of bad area, sig: 11 [#1]
PREEMPT
Modules linked in:
NIP: 00000000 LR: C0051AA8 CTR: 00000000
REGS: c0615cf0 TRAP: 0400   Not tainted  (2.6.18_dev)
MSR: 00021000 <ME>  CR: 42028088  XER: 20000000
TASK = cffea600[4] 'softirq-timer/0' THREAD: c0614000
GPR00: C0051A94 C0615DA0 CFFEA600 00000035 C0339CC0 00000035 00000013 00000003
GPR08: 00010000 00000000 FCFED080 C0480035 28028084 00000000 0FFFE700 FFFFFFFF
GPR16: FFFD7C1E FFFFFFFF FFFD7B78 C02E5930 00000000 00028488 00000000 FFFFFFFF
GPR24: C033FB1C C0699498 00008000 C0615DE0 C06BBCA0 C0614000 00000035 C0323AC0
NIP [00000000] _start+0x40000000/0x20
LR [C0051AA8] handle_fasteoi_irq+0x170/0x1b8
Call Trace:
[C0615DA0] [C0051A94] handle_fasteoi_irq+0x15c/0x1b8 (unreliable)
[C0615DC0] [C00045A4] do_IRQ+0x9c/0xbc
[C0615DD0] [C000E0C8] ret_from_except+0x0/0x18
[C0615E90] [C01B4A90] phy_write+0x2c/0x6c
[C0615EC0] [C01B6450] marvell_config_aneg+0x24/0xc8
[C0615ED0] [C01B48F8] phy_start_aneg+0x44/0xcc
[C0615EF0] [C01B50E8] phy_timer+0xd8/0x588
[C0615F10] [C0031EC4] run_timer_softirq+0x224/0x1048
[C0615F80] [C002D1EC] ksoftirqd+0x100/0x1ac
[C0615FC0] [C003F36C] kthread+0xc0/0xfc
[C0615FF0] [C000EC4C] original_kernel_thread+0x44/0x60
Instruction dump:
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX
  <0>Kernel panic - not syncing: Fatal exception in interrupt
Call Trace:
[C0615C30] [C0006DA8] show_stack+0x3c/0x1a0 (unreliable)
[C0615C60] [C0026484] panic+0x98/0x170
[C0615CB0] [C000BF60] die+0x120/0x130
[C0615CD0] [C0010D38] bad_page_fault+0xcc/0xe8
[C0615CE0] [C000DF24] handle_page_fault+0x7c/0x80
[C0615DA0] [C0051A94] handle_fasteoi_irq+0x15c/0x1b8
[C0615DC0] [C00045A4] do_IRQ+0x9c/0xbc
[C0615DD0] [C000E0C8] ret_from_except+0x0/0x18
[C0615E90] [C01B4A90] phy_write+0x2c/0x6c
[C0615EC0] [C01B6450] marvell_config_aneg+0x24/0xc8
[C0615ED0] [C01B48F8] phy_start_aneg+0x44/0xcc
[C0615EF0] [C01B50E8] phy_timer+0xd8/0x588
[C0615F10] [C0031EC4] run_timer_softirq+0x224/0x1048
[C0615F80] [C002D1EC] ksoftirqd+0x100/0x1ac
[C0615FC0] [C003F36C] kthread+0xc0/0xfc
[C0615FF0] [C000EC4C] original_kernel_thread+0x44/0x60

    After looking into the source, I figured out that handle_fasteoi_irq() is 
trying to call ack() handler for the irq_chip but arch/powerpc/sysdev/mpic.c 
(driving OpenPIC-compatible chips) doesn't have it (and I suppose is not 
obliged to since it's using the "fasteoi" flow)...

WBR, Sergei

Re: [PATCH -rt] powerpc update

[Sergei Shtylyov]

Hello.

Ingo Molnar wrote:
> * Daniel Walker <dwalker@mvista.com> wrote:

>>Pay close attention to the fasteoi interrupt threading. I added usage 
>>of mask/unmask instead of using level handling, which worked well on 
>>PPC.

> this is wrong - it should be doing mask+ack.

    It's what it was doing effectively. And what was wrong was calling ack() 
which OpenPIC driver didn't (and was not obliged to) support.

> also note that you changed:

>>-		goto out_unlock;

> to:

>>+		goto out;

> and you even tried to hide your tracks:
> 
> 
>> out:
>> 	desc->chip->eoi(irq);
>>-out_unlock:
>> 	spin_unlock(&desc->lock);

> :-)

> really, the ->eoi() op should only be called for true fasteoi cases. 

    Why is that? eoi() is effectively the same as ack() in this case. I must 
note that what's the "standard" 8259 driver is doing in mask_ack() seems 
misleading since it actually masks IRQ and sends EOI there.

> What we want here is to turn the fasteoi handler into a handler that 
> does mask+ack and then unmask. Not 'mask+eoi ... unmask' as your patch 
> does.

    That's effectively the same for OpenPIC. Maybe that implemetation just 
didn't look graceful but it was *correct*. And the current one is at least 
incomplete.

    I can see 3 ways to get out of this situation now:

1. Revert this change and use mask() + eoi() approach suggested by Daniel.

2. Add the ack() handler to OpenPIC driver -- and point it to mpic_eoi().

3. Do the same as x86 APIC driver does and use level/egde flows instead of 
fasteoi for the case when IRQs are threaded -- that ensues doing (2) as well.

    Note that all three aproaches lead to the effectively the same behavior 
WRT OpenPIC (except for the edge-triggered IRQs in 3rd case). Opinions?

> 	Ingo

WBR, Sergei

Re: [PATCH -rt] powerpc update

[Sergei Shtylyov]

Hello, I wrote:

> 3. Do the same as x86 APIC driver does and use level/egde flows instead 
> of fasteoi for the case when IRQs are threaded -- that ensues doing (2) 
> as well.

    Note that OpenPIC (as far as I could understand its h/w) is essentially
the same as IOAPIC/LAPIC couple on x86, i.e. a level-triggered IRQ remains 
effectively globally masked until CPU writes to its local EOI reg. However, 
the repetitive edge-triggered IRQ from the same line may still be detected 
while being serviced on a CPU (hm, the questions is, can it be sent to any 
other CPU in system while being handled on a certain CPU?)...
    I hope somebody corrects me if I'm wrong about both x86 and PPC. :-)

>>     Ingo

WBR, Sergei